Google Translate‘s ability to translate signs and menus in foreign countries just got a huge upgrade.
One of Translate’s biggest features—the ability to instantly translate scanned foreign-language text—is now available in 20 more languages.
When you hold your smartphone up to text and use this feature, Google crunches the data and immediately tells you what the text says. It even works without data connectivity, as long as you’ve downloaded the appropriate language pack. It’s perhaps one of the most compelling uses of augmented reality that’s changing our everyday lives.
Real-time photo-based translation now works in 27 languages total. Among the 20 new languages that can be translated back and forth with English are Danish, Filipino, Hungarian, Norwegian, Romanian, Turkish and Ukrainian.
Google’s translation software has improved dramatically since the company acquired augmented-reality translation service Word Lens in 2014. To demonstrate just how quickly and accurately the app works, Google made a video translating “La Bamba” into the new languages.
If you’ve ever tried the visual Translate feature, you’ll know that it feels a bit like magic. Suddenly, the whole world seems accessible in ways that language barriers previously prevented.
In order for Google to translate so fast and succinctly, it built the software using convolutional neural networks, a type of image recognition in which Google engineers teach machines to recognize different letters and numbers. The algorithm, which is similar to the machine-learning software that recognizes animals (and can turn them into creepy nightmare-fuel), can tell the difference between an “S” and a “5.”
Because so many of Google’s users are in countries that don’t have access to unlimited data or powerful smartphones, the company had to make the translation technology work offline. To do this, it limited the neural network’s handling of information and only taught it bare-bones language recognition.
It’s a rather complicated mathematical undertaking, but Google describes it more in-depth on its research blog.
Google isn’t the only company using augmented reality and real-time image recognition to transform the way we interact with and think about the world. Blippar, for instance, is a visual search engine that can recognize almost anything and tell you immediately what it is.
Through algorithms and neural networks, smartphones continue to get better at helping humans interact with the world around them.
Photo via Google/YouTube