Models
We refer simply as a Model to the set of Neural Networks Models trained to detect faces that we use within the application. We currently have 3 models implemented that use the Dlib Neural Network Models pretrained by Davis King (Author of dlib).
You can list and install the models using the command occ face:setup -m|--model [MODEL_ID].
This model uses a CNN face detector, and another model as 5-point landmarks shape predictor that allows aligning the faces to improve its descriptor. Finally use a Resnet model to get the face descriptor that is used to group similar faces.
- PDlib extension installed.
- 1 GB RAM assigned as maximum memory of PHP process. See the documentation about PHP Memory limit
- mmod_human_face_detector.dat
- shape_predictor_5_face_landmarks.dat
- dlib_face_recognition_resnet_model_v1.dat
It uses the same CNN network for face detection, but use a 68-point landmarks shape predictor model to align the faces. Having this data, you can infer more information as pose estimation, expressions, drowsiness, or 3D reconstruction, however, it is outside the objectives of the application, and was discovered that it negatively affects the quality of the clustering of faces (See https://github.com/matiasdelellis/facerecognition/pull/294#issuecomment-657324077). Thus, its use is discouraged unless you specifically want this information. Finally, it also uses the same resnet to get the descriptors of the faces.
- PDlib extension installed.
- 1 GB RAM assigned as maximum memory of PHP process. See the documentation about PHP Memory limit
- mmod_human_face_detector.dat
- shape_predictor_68_face_landmarks.dat
- dlib_face_recognition_resnet_model_v1.dat
Use a Histogram of Oriented Gradients face detector included internally in dlib, and same landmark predictor and resnet of Model 1. Note that the HOG model was training only with frontal and semi frontal faces, and find only faces with this position, therefore you will find less faces than with CNN models.
- PDlib extension installed.
- 128 MB RAM assigned as maximum memory of PHP process. See the documentation about PHP Memory limit
This model extends the model 1, doing a face validation with the Hog detector. Use the model 1 face detector to find as many faces as possible. However, we found that there are some faces that can cause problems, and these are correctly ignored by the HOG detector. Therefore, we go back to looking for faces using the HOG detector of Model 3, to avoid grouping the faces that can cause problems.
- PDlib extension installed.
- 1 GB RAM assigned as maximum memory of PHP process. See the documentation about PHP Memory limit
- Model 1 and Model 3 previously installed since they are dependencies.
- mmod_human_face_detector.dat
- Buil-In Dlib HoG
- shape_predictor_5_face_landmarks.dat
- dlib_face_recognition_resnet_model_v1.dat
For details read the article Comparative-analysis-of-the-models, but here we reproduce the main conclusions.
| Comparation | MODEL 1 (CNN5) | MODEL 2 (CNN68) | MODEL 3 (HOG) | MODEL 4 (CNN5HOG) |
|---|---|---|---|---|
| Faces Detected | Can find faces with very small images. | It uses the same face detector as model 1, therefore it has the same result. | Needs larger images to get the same results. | It uses the same face detector as model 1, therefore it has the same result. |
| Memory consumption | Memory consumption increases linearly according to the size of the image used. | Memory consumption increases linearly according to the size of the image used. Exactly the same as as Model 1. | Memory consumption is stable at around 100Mb regardless of the image analyzed. | Memory consumption increases linearly according to the size of the image used. Exactly the same as as Model 1. |
| Processing time | Processing time increases linearly according to the area of the image | It is exactly the same as Model 1 | It is slightly smaller than models 1 and 2 | The model analyzes the image twice, using CNN and complementing it with HOG, therefore it is practically twice as slow, |
| Clusters quality | High (Can have a few errors on some side faces) | Low (You can have more errors on faces that are not frontal.) | Very High (Only find fairly frontal faces) | Very High (Similar to model 3, but with many more faces) |
The model 5, is an external model for the Nextcloud Face Recognition application. It allows to run it on an external machine, which can be faster, and thus free up important resources from the server where you have Nextcloud installed. and it allows you to install our app, without installing the pdlib extension, which can be annoying in some cases. Theoretically, you can create any model, with any machine learning framework (tensorflow, ONX,et.), but we provide a reference model that implements the same models that we really explained above.
See:
