Facebook is getting smarter at faces—a lot smarter. If you’ve uploaded photos to the social network recently, you might have noticed an upswing in just how good Facebook is getting at guessing which of your friends appear in that photo from happy hour or a baby shower last weekend.
To test out what Facebook’s facial recognition algorithm has been up to lately, I decided to upload a test album of 15 photos. The photos were pictured I had recently taken of my friends, and I hadn’t previously shared any of them, to eliminate the possibility that Facebook was cross-referencing the photos with images already stored on its servers.
Of my test set, Facebook identified and automatically tagged faces in eight of the photos, and seven out of eight of the tags were correct. In the one it guessed wrong (a selfie of a friend and I eating corndogs, if you must know), it tagged my face as the with the name of the other person in the picture, so that’s at least half right in my book. Notably, two of the images it didn’t nail were taken in dark rooms, one was blurry, and one was of a friend who just buzzed her hair and dyed her eyebrows.
It’s worth noting that I didn’t ask Facebook to scan or tag these faces—the process happened automatically as I uploaded the photos from mobile. The tagging produced no delay whatsoever in the process of uploading the photo set to Facebook over Wi-Fi. It only took seconds. The automatic face scanning also prevented me from making a private test album by automatically making my photo set viewable to “people tagged” even though I’d selected the little lock icon and “only me” in the privacy settings menu.
From this set and others, I’ve noticed that once Facebook knows your face, it really knows it. A handful of my friends—not necessarily the ones who pop up in my uploaded photos the most often—are easy targets, apparently, and Facebook tags them every time with nearly total accuracy. Facebook also knows my face eerily well too, though it sometimes confuses me with one of my exes, which remains hilarious.
Facebook’s facial recognition technology is driven by a 2012 acquisition of an Israeli company called face.com, which built a facial-tagging system that Facebook had been licensing prior to the acquisition. Facebook owns the biggest repository of personal photos in the world, which should be no surprise, considering that sharing and tagging photos is a fairly unique opportunity for a social network built around real identities (1.39 billion, at last count). For scale, the world’s population is somewhere north of 7.125 billion.
Facebook’s recent practice of automatic photo tagging might feel a bit alarming (yes, a machine knows what you look like, and yes, it wants to make money off of you), but every untagged photo represents a missed engagement opportunity for a social network that thrives on its inherent stickiness. A user tagged in a photo is much more likely to view that image, Like it, and return to it as comments roll in, all of which adds up to $$$ in Facebook’s already insane engagement metrics.
According to a new report in Science Magazine, Facebook’s facial recognition technology—known, creepily, as DeepFace—already performs just as well as humans on some facial identification tasks. Facebook explains DeepFace’s algorithm in a paper published last year:
“In modern face recognition, the conventional pipeline consists of four stages: detect => align => represent => classify. We revisit both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network.”
DeepFace is working on identifying and tagging individuals in photos from crowds, which raises an entirely new, unsettling array of privacy concerns. As Facebook AI researcher Yann LeCun explained to Science, “You will get an alert from Facebook telling you that you appear in the picture. You can then choose to blur out your face from the picture to protect your privacy.”
Facebook’s DeepFace technology is so named in a reference to deep learning, a flavor of machine learning that’s particularly relevant to facial-recognition tasks—traditionally regarded as one of the more sophisticated processes conducted by the human brain. As Science explains:
“This is the “deep” in deep learning: The input for each processing layer is the output of the layer beneath. The end result of the training is a representational model of the human face: a statistical machine that compares images of faces and guesses whether they belong to the same person. The more faces the system trains on, the more accurate the guesses.”
Facebook’s DeepFace processing will grow to span the 400 million pictures that its users upload on a daily basis. That includes crowd shots and photos that users are otherwise unaware that they might appear in—photos from political rallies, music festivals… anything anywhere, really.
You might be creeped out that Facebook can now identify and involuntarily tag your selfies with ease, but know that Facebook’s big picture on facial recognition really is much, much bigger.
Illustration by Max Fleishman
