In Fiado, to open an investment you have to prove you are the holder of the ID card: a liveness check with the camera and a match of your face against the photo on the document. I built it with face-api on the NestJS backend, on WebAssembly, with no GPU or native binaries. The first version worked perfectly with the sample photos and rejected almost every real ID card.
Two faces where I expected one
The rule was: exactly one face per image, so nobody verifies with someone else next to them. But the Ecuadorian ID card carries, besides the portrait, a smaller ghost portrait, so every card "had two people". On the card, then, the largest face counts: I detect them all and sort them by area.
const faces = await faceapi
.detectAllFaces(tensor, new faceapi.SsdMobilenetv1Options({
minConfidence: MIN_DETECTION_CONFIDENCE,
}))
.withFaceLandmarks()
.withFaceDescriptors();
return faces.sort((a, b) => b.detection.box.area - a.detection.box.area);In the selfie, when does a second face count?
In the live frame the rule does have to exist, but counting any face was too much: a picture on the wall, someone walking by in the background or a false positive brought the verification down. Now a second face only invalidates the capture if the detector is confident (0.7 or more) and it is at least a quarter the size of the main one.
const EXTRA_FACE_CONFIDENCE = 0.7;
const EXTRA_FACE_AREA_RATIO = 0.25;
export function relevantFaces<
T extends { detection: { score: number; box: { area: number } } },
>(faces: T[]): T[] {
const [main, ...rest] = faces;
if (!main) return faces;
return [
main,
...rest.filter(
(face) =>
face.detection.score >= EXTRA_FACE_CONFIDENCE &&
face.detection.box.area >=
main.detection.box.area * EXTRA_FACE_AREA_RATIO,
),
];
}Rotated photos and tiny portraits
Two more failures came from the photo, not the person. Photos taken with a phone sometimes arrived rotated and the detector found nothing: if no face shows up, I try rotating the image by 90° until it comes full circle. And the portrait takes up little of the card: searching for it in the image scaled down to 1280 px is fine, but taking the descriptor from there loses detail. The descriptor comes from a crop of the portrait at full resolution.
A measured threshold, not the usual one
face-api usually takes a distance of 0.6 between descriptors to mean "same person". With the sample photos, the same person landed between 0.05 and 0.35, and different people from 0.47. A real ID card (printed, with holograms, with a photo from years ago) pushes the holder away from their selfie, so the threshold ended up at 0.5: it accepts the holder without reaching 0.6. And it is set per environment, because with more real cards it would need measuring again.
const distanceAtThreshold = () => {
const raw = Number(process.env.FACE_MATCH_DISTANCE);
return raw > 0 && raw < 1 ? raw : 0.5;
};What it doesn't do
This liveness check stops a printed or on-screen photo, which doesn't turn or smile on request. It is not a replacement for a certified passive detector (ISO/IEC 30107-3) against masks or real-time deepfakes. That is why recognition sits behind an interface (BiometricProvider): swapping face-api for a commercial provider leaves the rest of the system untouched.