Face Gallery

After a campus event the photography team drops a few thousand photos into a shared folder and everyone scrolls through all of them looking for themselves. The fix is obvious: index the faces once, then let people search with a selfie. Getting there meant building the same feature three different ways to find out which one was actually viable.

The shape of the work, and where you put it

Face search looks like one problem and is really two. Detection and embedding - finding a face in a photo and turning it into a vector - is the expensive step, hundreds of milliseconds per image on CPU. Comparing two vectors is a subtraction and a square root, microseconds. Every design decision in this repo comes down to which of those two you put inside the per-query loop.

Get it wrong and the cost of a search grows with the size of the archive multiplied by the cost of a model forward pass, which is exactly the wrong thing to scale with three thousand photos and two hundred people all searching at once after an event. Get it right and a search is a few thousand vector distances, which is nothing.

Three implementations

• dlib embeddings in MongoDB. A Flask app uses face_recognition to extract a 128-value encoding per detected face, pickles each one and stores it as BSON binary against the image path with a timestamp. One document per image with a faces array, so a group photo with eleven people is one row holding eleven vectors. Search encodes the query face and compares it against every stored encoding at a 0.48 distance tolerance, breaking out of the inner loop on the first match so a photo is scored once. This is the version that got closest to usable, because the expensive work happens once at ingest.
• DeepFace behind FastAPI. Uploads are written to disk and mirrored into Mongo through motor, and /search/ takes a base64 data URL, decodes it with OpenCV, and runs DeepFace.verify against each stored image with enforce_detection=False, returning matches ranked by distance. Containerised on python:3.8-slim and served by gunicorn with four uvicorn workers.
• Entirely in the browser. A Create React App portal loads ssdMobilenetv1, faceLandmark68Net and faceRecognitionNet from a local /models directory, computes a descriptor for the query face and for each uploaded photo, and matches on Euclidean distance under 0.6. No image ever leaves the device.

Why the second one is the wrong shape

The FastAPI service is the clearest illustration of the mistake, and it makes it twice.

It stores the image bytes in Mongo, not an embedding - so every stored photo is a BSON document carrying a full JPEG, which caps out against the 16 MB document limit and doubles the storage because the file is also on disk. Then, for each query, it pulls every one of those documents, runs cv2.imdecode to turn the bytes back into pixels, and hands the result to DeepFace.verify, which runs detection and embedding on both images before it compares anything. The per-query cost is therefore the whole archive decoded and re-embedded, and the same photo is re-embedded on every search anyone makes. Nothing is ever cached because nothing was ever computed at ingest.

The dlib version inverts exactly that. Detection and embedding run once, at upload, and the artefact kept is 128 float64 values - about a kilobyte pickled, against a couple of megabytes for the JPEG. A search decodes one image, the query, and then does arithmetic. That is the right split even before you worry about how you search the vectors, because it turns a model-bound problem into a memory-bound one.

Supporting pieces

• Webcam capture in vanilla JS: getUserMedia into a <video>, a canvas grab on click, toDataURL to JPEG, and a blob upload. The Python CLI has the OpenCV equivalent, showing a live feed and saving the frame on s.
• Multi-file upload with per-file preview via FileReader before anything is sent.
• An Express and multer upload service writing to a timestamped filename and serving /uploads statically, so the JS portal did not need a Python backend to test against.
• A results gallery rendering matched photos with their similarity scores.
• A batch ingest script separate from the web app, so a folder of a few thousand event photos can be encoded overnight rather than through the upload form.

What the comparison taught

The browser version is the most privacy-respecting and the least scalable. It never persists a descriptor at all: the photos it can search are the ones in React state from the current session, and each search re-runs detection and embedding over every one of them in the tab. It is the same architectural error as the FastAPI service, on hardware with less to spare, and it buys something real in exchange - the photo never leaves the phone.

The DeepFace service is the worst of the three on cost, because it decodes a JPEG and re-runs face detection inside the comparison loop for every stored image, so the per-query cost grows with the archive and the expensive part is the part being repeated. The dlib version is the only one that separates ingest from query, and it still does a linear scan.

The real conclusion is that all three miss the same thing. Once you are storing embeddings, the search should be an approximate nearest-neighbour lookup over a vector index, not a comparison against every row. That is where this would have gone next, and it is why it is filed as a prototype rather than a product.

What else it would need

Neither service is deployable as it stands, and the reasons are worth naming. The FastAPI image declares Django, DeepFace and TensorFlow in its requirements and imports FastAPI, motor, OpenCV and numpy, none of which are listed - the container installs the wrong dependency set for the code it copies in. The Mongo insert on the upload path is a motor coroutine that is never awaited, so uploads to that service quietly do not persist. CORS is opened to every origin with credentials enabled, a combination browsers reject outright. The Mongo connection string is hardcoded in three scripts. And there is no notion of a user, an event or a permission anywhere - a search returns any photo containing the queried face, which for a tool whose entire input is other people's faces is the first thing a real version would have to answer for.