Facial recognition, between advanced biometrics and needs for privacy… – Part 2 of 4

Continuation of our series on facial recognition. After defining facial recognition among the biometric processes (see here), let’s see the techniques used for its application.

State of the art of the techniques used for its application

The technology on this topic is constantly evolving, driven in particular by the web giants, who directly publish their theoretical discoveries in the areas of AI and image recognition, to advance the state of the art as quickly as possible.

Main steps of the process

The facial recognition process can be performed from photos or videos. [6]

It takes place in five major steps:

1. Face detection: the system will isolate the faces present in the image from the rest of the image, to prepare them for processing.
2. Preparation of the images to align them to a precise standard: the goal is to make variables such as the position of the head, the size of the image and photographic qualities such as lighting or gray level as little influential as possible on the measurements that will follow.
3. Facial data extraction. Once the image is prepared, all the facial data that the AI will use to compare the information is extracted from the image.
4. A model, called “template”, which represents the biometric characteristics of the face appearing in the image (or video) is made.
5. The values of this template are then compared with templates calculated in real time from the stored biometric data.

For authentication, this template is made from the stored data for the identity that the person claims to be.
For identification, the template made in 4. is compared with the different people present in the database, and the AI selects the most matching person, provided that the similarity score is above a predetermined threshold. [6]

The different techniques that accompany the stages of facial recognition

Algorithms with “feature-based” approach

The first of the two main approaches for face recognition algorithms is to identify the different features of a face by extracting them from the image. The algorithms will retrieve from the image the different values associated with the criteria listed in the section above (parameters used for comparison).

This approach is also called “geometric”.

The criteria most often used by algorithms with geometric approach are:

• Eye distance
• Nose bridge distance
• Commissures of the lips
• Ears
• Chin
• Face shape
• Shape of the jaw

Algorithms with a “holistic” approach
Holistic approach algorithms aim at normalizing a gallery of face images, compressing the face data, and saving only the part of the data that is useful for facial recognition. This compressed representation of a face gives a template, and the different templates are then used for comparison. [7]

This approach is also called “photometric”.

Photometric algorithms include:

• Eigenfaces (oldest algorithm developed, 1991) [8]
• Fisherfaces
• Elastic bunch graph matching [9]
• Linear discriminant analysis [10]
• Hidden markov model [11]
• Local Binary Patterns Histograms (LBPH), which is one of the most popular today.

Remote human identification

To enable automatic human identification at distance (HID) (and thus at low resolution for the human in question shown in the photograph), the initial low resolution is enhanced using a process called “face hallucination”. [12]

This process then precedes the traditional face recognition steps, to prepare the image. It uses either:

• A machine learning AI, trained by face examples. The AI will trace the face in more detail to improve the resolution of the image based on these face examples to determine what that part of the face would probably look like in a more accurate photo.
• A k-nearest neighbor distribution, which is a statistical mathematical function often used in the AI world. This approach aims to mathematically deduce what the neighboring pixels of already known pixels “most likely” look like.

These processes can be enhanced by incorporating information about face characteristics based on age and different demographics into the AI, to help it make the right choice.
This is particularly useful on:

• Images from traditional video surveillance cameras, where the resolution is usually much too low for the image to be used as is for facial recognition.
• In the case of using facial recognition algorithms that require particularly high resolutions, the face hallucination process is also used to achieve a sufficiently high resolution for more standard resolution images, and thus widen the usable database.
• In case of hidden or partially hidden faces. This is one of the methods to recalculate the masked part of the face (by glasses for example).

3D recognition

The use of 3D sensors allows for a more accurate capture of information about the shape of the face. This method of capture has several advantages:

• It is not affected by changes in ambient lighting.
• It can identify more easily photographs taken in profile.
• Using 3D data gives much better performance for facial recognition AI.

Some facial recognition systems already in place use a 3-camera system to capture faces in 3D.
Note that the use of 3D faces makes facial recognition very sensitive to facial expressions that distort it; it becomes necessary to pre-process the image to compensate for this influence and allow good results.

Use of thermal imaging cameras

In order to completely bypass any attempt to hide one’s face, facial recognition techniques using thermal cameras have been developed. These methods are not very effective on their own for several reasons.

1. The databases of faces taken with thermal imaging cameras are very limited. Where other facial recognition methods can for example feed their databases via an aspiration of content found on the internet, infrared camera databases almost always have to be built from scratch.
2. This method does not currently work for photos taken outdoors. It needs a stable temperature environment.

On the other hand, researchers at the ARL (US Army Research Laboratory) have developed a method that allows to compare images taken in infrared with images taken by normal cameras. This solves the problem seen in 1.

Other techniques and associated technologies

Some other techniques are used in facial recognition to tailor it to specific needs.

• Match on card: biometric data is contained on a card in our possession rather than stored in a database. They can then be compared without any data leaving the card.
• Multi-modal biometrics and other authentication factors: when more reliability is needed than facial recognition alone, systems use multiple authentication factors.
• Data Anonymization: A biometric database can avoid linking data to an identity, and instead link it to a string of characters.

________________________________________________________________________

[6] Official site of the CNIL, Facial Recognition: https://www.cnil.fr/fr/definition/reconnaissance-faciale
[7] Wikipedia, Système de reconnaissance faciale: https://en.wikipedia.org/wiki/Facial_recognition_system
[8] Ravi S., 2013, A study on Face Recognition Technique based on Eigenface
[9] Wiskott L., 1997, Face Recognition by elastic bunch graph matching
[10] ETEMAD Kamran and CHELLAPPA Rama, 1997, Discriminant analysis for recognition of human face images: https://www.face-rec.org/algorithms/LDA/discriminant-analysis-for-recognition.pdf
[11] ARA V. Nefian et MONSON H. Hayes III, 1998, Face detection and recognition using hidden markov models: http://www.anefian.com/research/nefian98_face.pdf
[12] XIAOU Tang, 2015, Hallucinating Face by Eigentransformation: https://www.researchgate.net/publication/3421633_Hallucinating_Face_by_Eigentransformation