Facebook has not been secretive about its attempts to identify people, places, and things in the photos you upload to the social network. While some of its efforts verge on creepy, in recent months, Facebook has shown how it can use this technology to better allow visually impaired users to join the conversation around photos. And after downloading a simple browser extension, you can see how good (or poor) of a job Facebook is doing at this.
If you download this Chrome or Firefox extension from Github, it will show you the labels with which Facebook’s computer vision algorithm automatically tags your photos. For a photo of a mother and child on the beach, for example, this extension reveals that Facebook thinks the photo includes one person, one child, the ocean, the outdoors, and water, among other things.
For the visually impaired, these words can be read by a screen reader so individuals can have context about what their friends are posting online—and that’s a huge improvement for those users. However, it also reveals new insights into what sorts of information Facebook knows about you, even if you’re not offering other clues in the descriptions of your images. If you constantly post photos with a cat or a dog, it could infer you have a pet; ditto if your photos generally include a child or two. (And of course, it’s not just Facebook that’s doing this sort of image recognition— it’s every big internet company.)
Adam Geitgey, who developed this browser extension, says that the goal of this project is “simply to make everyone aware of the kind of information that is routinely extracted from your own images today.”