Facebook is using artificial intelligence to let millions of blind and visually impaired individuals around the world “see” what’s in photos.
The company introduced automatic alternative text, or automatic alt text, that uses Facebook’s AI tech to automatically generate descriptions of photos that can be read by screen readers. Screen readers are programs used by visually impaired people that read text on computers and mobile devices, either through a text-to-speech or braille interface.
Through Facebook’s object recognition technology, automatic alt text is created for each picture. Instead of just saying “photo,” the screen reader will pick up the automatically generated alt text, so it would describe a photo like: “Image may contain: mountain, sky, dog,” or other descriptions of recognizable objects in a photo.
Twitter recently launched a new feature for iOS and Android that lets people add alt text to photos manually.
Facebook image recognition software is trained on billions of photos uploaded to Facebook. Each day, people share two billion photos across all of Facebook’s properties including Instagram, WhatsApp, and Messenger. The company already uses its robust AI tech all over Facebook—for instance, it can automatically recognize your friends’ faces through your camera roll and send photos to them automatically, and is working on building a feature that would prevent you from accidentally uploading inappropriate photos.
The company has open sourced some of the tools it uses at the Facebook AI Research Lab so developers can create artificial intelligence software to add to apps and websites.
Other companies are using image recognition software to auto-tag pictures or know when you or friends appear in photos, but it’s not without hiccups. Last year, Google Photos tagged Black people as “gorillas,” prompting understandable outrage.
Facebook’s automatic alt text is only available in English on iOS devices. It’s still in early stages of development and can’t recognize all details within photos yet, but Facebook is working on adding the feature to other platforms and languages in the future.