Face matching, liveness and accuracy.

How face matching works

A face recognition system does not compare photographs the way a person does. It works in steps:

• Capture: a camera takes an image or short video.
• Detection and quality: software finds the face and checks whether the image is good enough, considering focus, lighting, pose and whether the eyes are open and the face uncovered.
• Feature extraction: the face is converted into a template, a compact set of numbers that represents its features. The template is not a picture, but it is still sensitive: researchers have shown that a recognisable face image can sometimes be reconstructed from an unprotected template. Well-designed systems therefore encrypt templates and may use template protection techniques (see ISO/IEC 24745) that make them hard to reverse and allow them to be cancelled and re-issued.
• Comparison: two templates are compared and the system produces a similarity score.
• Decision: a threshold turns the score into "match" or "no match".

Two different tasks use this machinery:

• Verification (one-to-one): is this person the same person as on this document or this enrolled record? This is what most identity checks do.
• Identification (one-to-many): who, among everyone in a database, is this person? This is used for finding duplicate applications, and in policing and border settings.

At the same threshold, a one-to-many search is more likely to produce a false match than a one-to-one check, because the face is compared with every record and each comparison is another chance of error. Large databases therefore need stricter thresholds and human review of candidate matches. One-to-many searches also raise greater privacy concerns, and some jurisdictions restrict them specifically.

Measuring accuracy

No face matching system is perfectly accurate. It makes two kinds of mistake:

• A false match: it says two different people are the same person. In identity checking, this lets an impostor through. The rate is called the false match rate (FMR).
• A false non-match: it says two images of the same person are different people. This rejects a genuine customer. The rate is called the false non-match rate (FNMR).

Systems can also fail before matching: a failure to capture or failure to enrol happens when no usable image is obtained, and these failures should be counted too.

The threshold trades one against the other. Raise it and false matches fall, but more genuine people are rejected; lower it and the reverse happens. Choosing the threshold is a business and risk decision, documented and reviewed, not a technical default.

Accuracy claims mean little without their conditions. Always ask:

• At what false match rate was the false non-match rate measured?
• On what images: good passport photos, or real selfies taken on cheap phones in poor light?
• On which people: does the test population resemble your customers?

Independent benchmarks help compare algorithms. The US National Institute of Standards and Technology (NIST) runs ongoing public evaluations of face recognition algorithms. Since August 2023 these have been called the Face Recognition Technology Evaluation (FRTE) and, for face analysis tasks such as liveness, the Face Analysis Technology Evaluation (FATE); they were previously known as the Face Recognition Vendor Test (FRVT). In the end, what counts is the accuracy you measure on your own customers, with your own capture process.

Fairness across groups

A system can perform well on average and still perform worse for some groups of people. Differences in error rates between groups defined by, for example, age, sex or skin tone are called demographic differentials.

NIST's 2019 study of demographic effects (NIST Interagency Report 8280) tested 189 algorithms from 99 developers. It found demographic differentials in most, but not all, of them. Differences in false match rates were much larger than differences in false non-match rates, often a factor of 10 to more than 100 between groups. With good-quality photos, false match rates were highest for West and East African and East Asian people, and higher for women, older people and children. The size of the effect varied greatly between algorithms; more accurate algorithms generally showed smaller differentials, and for some highly accurate one-to-many algorithms they were undetectable.

What this means in practice:

• Test on the population you actually serve, and measure error rates for each relevant group, not only overall.
• Mind capture conditions. Lighting, camera quality and exposure settings can affect some groups more than others, and good capture guidance helps everyone.
• Watch for age. Faces change over time, so comparing a selfie with a document photo taken many years earlier is harder.
• Never let the machine be the only route. People the system fails should be able to reach a trained person and another way of proving who they are.

Fairness is not only an ethical point. A system that rejects some groups of customers more often than others can exclude them from services. It can also raise legal issues under Kenya's constitutional protection against direct or indirect discrimination (Article 27) and the Data Protection Act's requirement to process data fairly (section 25).

Liveness and attacks

Face matching answers "is this the same face?" It does not tell you whether the face belongs to a living person who is actually present at the moment of capture. That is the job of liveness detection, one form of what standards call presentation attack detection (PAD).

A presentation attack is an attempt to fool the camera with something other than the real person present:

• a printed photo or a photo shown on a screen;
• a video replayed on a screen;
• a mask, from a simple paper cut-out to a realistic three-dimensional mask.

The international standard ISO/IEC 30107 covers presentation attack detection, and Part 3 (ISO/IEC 30107-3:2023) sets out how to test and report it. Two measures matter: how often attacks are wrongly accepted (the attack presentation classification error rate, APCER, measured separately for each type of attack) and how often genuine people are wrongly rejected as attacks (the bona fide presentation classification error rate, BPCER). Like matching, it is a trade-off, and liveness and matching each have their own threshold, so a journey's overall security and convenience depend on both together.

Another attack targets the reference photo itself: a morphed photo blends two people's faces so that both can match it, which is why the document photo's origin also matters.

Remote identity checks face a second kind of attack that never goes near the camera. In an injection attack, fake images or video, including deepfakes, are fed straight into the app or the data stream, for example through a virtual camera or a modified app. Defending against it needs checks on the device, the app and the integrity of the video stream, not only on the face. In 2024 the European Committee for Standardization (CEN) published a technical specification on detecting such attacks, CEN/TS 18099 Biometric data injection attack detection.

Ask any liveness provider which attacks were tested, by whom, to what standard and at what error rates, and test the whole journey yourself.

Handling biometric data

Biometric data is among the most sensitive personal data there is. Unlike a password, a face cannot be changed if it leaks.

In Kenya, the Data Protection Act, 2019 lists biometric data in its definition of sensitive personal data (section 2), which may only be processed under stricter conditions. The Office of the Data Protection Commissioner has published guidance notes on biometric data. Good practice, and in many cases the law, calls for:

• A lawful basis, a sensitive-data condition and clear notice before capture. You need a lawful basis under section 30, such as consent or performance of a contract, and, because biometric data is sensitive, a condition under section 45. People must be told what is collected, why, for how long and who will see it.
• A data protection impact assessment before processing that is likely to result in high risk (section 31). Kenya's Data Protection (General) Regulations, 2021 list processing biometric data among the operations considered high risk, so an impact assessment should be expected for any biometric identity check.
• Purpose limitation: use biometric data only for the purpose it was collected for. A selfie taken to open an account should not be reused to build a face search database.
• Keep templates, not images, where possible, protect them with strong encryption and strict access control, and consider template protection techniques that make a stolen template hard to reverse or reuse, and allow it to be cancelled and re-issued.
• Retention limits: delete biometric data when it is no longer needed.
• Human review: under section 35, people have a right not to be subject to decisions based solely on automated processing that significantly affect them. Exceptions include decisions necessary for a contract, authorised by law or based on consent; where such a decision is made, the person must be told in writing and may ask for it to be reconsidered, or for a new decision not based solely on automated processing. Good practice is to let anyone rejected by a biometric check reach a trained person.

Used this way, biometrics can make identity checks both safer and easier. Used carelessly, they create risks that last a lifetime.

Your certificate of completion

Enter your name as you want it to appear, then save the certificate as a PDF. In the print window, choose Save as PDF. A certificate is issued once per completion of the course.

Saolix does not record who takes this course, so it cannot verify these certificates. The certificate confirms completion of a free self-paced course and is not an accredited qualification.