Facial recognition, between advanced biometrics and needs for privacy… – Part 1 of 4

Among today’s biometric applications, one in particular is the subject of much controversy and discussion: facial recognition.
One of the most controversial applications of facial recognition is face recognition based on a photograph, as done by the company Clearview AI to name one.

After defining facial recognition among the biometric processes in this article, we will give an overview of the techniques used for its application in a second part.
In a third part, we will give the rules to follow to allow its use, its field of application and the regulatory limits that this implies (in European countries in particular).
In a fourth and final part, and in a defensive concern (to try to protect oneself against the techniques of acquisition of these images for example, to lure them in particular), we will indicate if parades exist today, and will give proposals of implementation if necessary.

Introduction

Facial recognition is the most talked about biometric technique of the last decade. Touted as a high-potential technology since the 1990s, it has long been hampered by technological limitations such as computational capabilities, camera image resolution, limited storage capacity, and the lack of effective neural network models. With these technological barriers lifted, the web and computer giants have set about advancing facial recognition, notably by making the results of their work on artificial intelligence available in open source.

Facial recognition systems, or FRSs, are now at least as effective at recognizing an individual as a human and require only a few seconds and no action by the individual to perform authentication. The different technologies are combined to reduce the risk of error and fraud, and image processing techniques prior to facial recognition itself allow the use of an increasingly wide range of images (low-resolution, profile, infrared, masked face, photograph of a crowd…) in a way that is almost as effective.

But with this progression, human rights and the laws that protect them are more and more likely to be violated. The global scene is becoming more polarized and opinion is divided over this technology that allows Orwellian levels of surveillance of the civilian population. Some digital giants like Amazon and Microsoft are backing down in the face of protests, and say they want to wait for more specific laws before continuing to market their FRS solutions; others, like the Chinese government, are embracing the technology and monitoring the population, while still others are looking for methods to block, lure facial recognition systems and protect their anonymity… Or break the law.

In the midst of all these questions, the various states are struggling to establish an effective legal framework to protect individuals from the abuses of the technology; and companies like Clearview AI (USA) have taken advantage of the general hesitation to take the plunge, announcing that they have collected more than 10 billion images of faces from the entire Internet without any authorization, and are making their artificial intelligence services available to government bodies and the private sector using this database.

We will review what facial recognition is and briefly look at its history, before dwelling on the different techniques used today, the legal framework existing around the world, and ending with a discussion of the different attempts to fool the technology and their results.

Facial recognition among biometric processes

Biometrics – definition

Originally, the word “biometrics” referred to any analysis and measurement of physical characteristics strictly specific to a person (voice, face, iris, fingerprints…).

Nowadays, however, it is generally used to designate all computer techniques that allow for the automatic recognition, authentication and identification of an individual based on his or her physical, biological and even behavioral characteristics. Biometric data is therefore personal data, as it allows a person to be identified.

To achieve this goal, it is necessary that the characteristics used as criteria are:

• Universal: every human being must possess them.
• Unique: the criterion must not be identical between two given individuals, to limit authentication errors
• Invariant: the value of this criterion for a given individual must not change throughout his or her life, so that the identification remains reliable over time.
• Measurable: the current technology must allow a reliable measurement, so that the comparison is feasible.

The goal of biometrics is to make authentication, identification and recognition simpler, faster, and above all more secure.

The different categories of biometrics

There are three main categories of biometrics:

• Biological (identification via DNA).
• Morphological or morpho-physiological (hand, palm, fingerprints, venous network, face, iris, venous network of the retina, voice, gait, ear).
• Behavioral.

The biological category will use blood, urine or saliva for identification. It is obviously time-consuming and is mostly used in the context of judicial investigations.

The behavioral category will use voice recognition, signature dynamics (speed of pen movement, pressure exerted, etc.), gait, as well as keyboard strokes.

The morphological category is the only one that can easily be used on a large scale and in both private and public domains, due to the ease of acquiring the necessary data for comparison.

The position of facial recognition within biometrics

Facial recognition is a morphological type of biometric technique.

It is one of the three most widely used biometric recognition technologies today, along with fingerprint and iris recognition. It is the most efficient, reliable and easy to deploy technology at our current level of technology.

The advantages of facial recognition over other forms of authentication and identification are:

• For the best systems currently available, there is a strong resistance to fraud in all conditions of lighting, angles of the face or changes in it (motorcycle helmet, headphones, haircut, glasses, etc.)).
• Its use is very fast, the user hardly needs to stop.
• It is a contactless identification (interesting for several reasons, like the associated hygienic issue, particularly appreciated since the pandemic)
• Identification in the middle of a crowd is quite possible, just like in other dynamic and unstable environments. [1]
• Many of the required databases are very easy to populate in the case of simple photography. The Clearview AI example has shown that it is even possible to build up an effective international database simply by sucking up the images that people put on the web.

Facial Recognition Technologies (FRTs) can be used for:

• Identification (1:N verification): find out who the person in the photograph is by searching for similar data in one or more databases.
• Authentication (1:1 verification): Verify that the person is who they say they are by comparing them with pre-stored data for that person.
• Detection: simply verify that there is a face present.
• Verification: check that two biometric templates belong to the same person. The model does not need to know the identity of the person.
• Categorization: classify people based on their morphological characteristics, or classify photographs based on facial expressions for example.

The evolution of facial recognition

Although the first attempts and algorithms for facial recognition date back to the early 1990s, the technology requires a higher level of technology than other biometric technologies like fingerprint recognition, etc. The lack of sufficiently accurate images, sufficiently large databases, and sufficiently large computational capacities in particular have meant that FRT have long lagged behind biometrics.

2014: GaussianFace algorithm (University of Honk Kong) achieves facial identification scores of 98.52%

2014: Facebook launches DeepFace (97.25% accuracy). First facial recognition technology implementing deep learning.

2015: Google launches FaceNet, which achieves up to 99.63% accuracy. It leads to an integration of the technology with Google Photo to sort users’ photographs and becomes available in an open-source version called OpenFace.

2018: Amazon promotes Rekognition to law enforcement. The solution can recognize up to 100 people in a crowd photograph and has a database with tens of millions of faces.

2018: LFIS, the Thales solution, achieves 98% accuracy with less than 5 seconds spent per face on a test conducted by the U.S. Homeland Security Science and Technology Directorate on 300 volunteers.

2019: FRTs now receive the third largest share of global AI investment ($4.7 billion). [2]

2020: NIST tests show that the best FRT algorithms no longer have a racial or gender bias.

2020: Amazon places a one-year moratorium (since extended) on police use of Rekognition, for ethical reasons, pending more comprehensive US legislation. Microsoft does the same, and Axon announces that it is withdrawing from marketing TRFs to US police forces. [3]

2025 Predictions are that FITs will be used for smartphone payments by more than 1.4 billion users by 2025. [4] The necessary technologies are already in place on most mobile OSes today. [5]

It is estimated that facial recognition performance has increased 20-fold between 2013 and 2018, and accuracy continues to improve each year by 30-50%.

Who uses facial recognition?

Facial recognition is now used on a daily basis in the private and public sectors.

• Consumer applications: Smartphones, tablets, computers are equipped with facial recognition systems to authenticate their owners.
• Social media: Snapchat, Facebook etc. also use this technology to authenticate users
• Commercial applications: Identification of people approaching ATMs in banks, and anti-fraud banking checks, including on banks’ mobile applications.
• Surveillance and access control in physical spaces: Facial recognition systems are installed, for example, to control access automatically and replace manual identity checks; for example, in the Singapore airport, or at border controls.
• Identification in public spaces: Demonstrations, gatherings, or simply individuals moving normally in the street are identified via surveillance cameras. The system is already extremely developed and used in China, notably with SkyNet which monitors more than 1.4 billion suspects in their daily activities.
• Healthcare: FRTs are used for automatic patient sorting, for example.
• Elections: FRTs are also used for authentication during remote voting.
• Sentiment analysis: Sentiment analysis research is now completely dependent on FRTs. This research in turn feeds the progression of FRTs which integrate a correction of facial deformations via emotion to improve their efficiency.

________________________________________________________________________

[1] Official site of the Thales group, Facial Recognition: https://www.thalesgroup.com/fr/europe/france/dis/gouvernement/biometrie/reconnaissance-faciale
[2] Standford University, 2019, The AI Index 2019 Annual Report: https://hai.stanford.edu/sites/default/files/ai_index_2019_report.pdf
[3] SMITH Rick, 2019, The future of face matching at Axon and AI ethics board report: https://www.axon.com/company/news/ai-ethics-board-report
[4] i-SCOOP, 2020, Facial recognition 2020 and beyond — trends and market AND Fortune Business Insight, 2021, Facial recognition market, Global Industry Analysis, Insights and Forecast, 2016-2027
[5] Juniper Research, 2021, Mobile payment authentication: Biometrics, Regulation & forecasts 2021-2025 et Facial Recognition for Payments Authentication to Be Used by Over 1.4 billion People Globally by 2025.