You use augmented reality on a regular basis—even if you don’t realize it. If you’ve ever seen Who Framed Roger Rabbit or Space Jam, then you have an idea of what it looks like. The same goes for the yellow first down marker used in football broadcasts and virtual gauges in-flight training simulators. Augmented reality, or AR, blends the digital and real world in both subtle and profound ways. The technology is now taking off in the mobile industry, with popular apps like Pokémon Go, Ingress, and features on Snapchat introducing many to the futuristic concept for the first time, and it’s only going to become more prevalent. Here’s everything you need to know about it.
What is augmented reality?
Augmented reality (AR) is the process of enhancing the real world by superimposing digital images and sounds onto the environment. It’s important not to confuse it with virtual reality, which puts users in a fully digital computer-generated simulation.
The term “augmented reality” was coined in 1990 by Boeing researcher Thomas Caudell to describe head-mounted displays used by electricians when assembling wiring harnesses. But the first use of AR came much earlier and is widely contributed to computer scientist Ivan Sutherland, the “father of computer graphics,” who created the first AR head-mounted display in 1968. The aptly named “Sword of Damocles” was a primitive machine that looked like it came from the mind of an evil eye doctor and was so large it had to be suspended from the ceiling.
The next big development came in 1992, when Boeing researcher Louis Rosenberg built Virtual Fixtures for the U.S. Air Force. What some consider the first immersive AR platform, Virtual Fixtures consisted of an upper-body exoskeleton suit and used AR vision to replace a user’s real arms with two usable robotic arms. Then ARToolKit, an open-source software library used to overlay computer graphics on camera, was developed and would later find its way onto computers in 2009.
These creations all led to the troubled product most people associate with AR: Google Glass. First released in 2014, Google Glass was the first fully realized augmented reality headset made available to the public. Software bugs and privacy concerns forced the search giant to suspend sales after less than two years. Hope then shifted to Microsoft’s HoloLens mixed reality, an in-development headset that lets you see holograms in the world around you.
AR really started ramping up in 2017, when smartphone powerhouses Apple and Google announced new developer frameworks in ARKit and ARCore, making it easy for devs to build AR apps from scratch.
How does AR work?
There are many forms of AR, and they all work in different ways. We’re going to look at smartphones and headsets, the two gadgets that will most likely house the future of augmented reality.
Mobile apps use a camera and sensors to merge a view of the real world with virtual objects. First, your phone needs GPS, which tells an app exactly where you’re standing at any time. For some apps, a general location is all that’s needed. For example, Pokémon Go only requires that users stand at a marker to train or claim virtual Pokémon. Other apps, including those that scan QR codes, look for a tag on an item or surface.
Using your camera and software sorcery, AR apps place virtual figures on top of whatever your camera is pointing at. A great example is Snapchat’s filters, which overlay virtual objects onto a real-world image captured by a phone’s camera.
Virtual reality glasses and headsets like Google Glass and Microsoft HoloLens are far more complex and are designed to offer greater immersion. Companies claim these devices aren’t created to help you escape but are instead meant to connect the endless information on the internet with reality by transitioning the digital world from a 2D computer or smartphone display into 3D holograms.
We won’t dig too deep into the tech behind AR headsets, but the idea is to use a bunch of cameras and depth sensors to “see” the world, and send that information back to a computer. From there, the headset projects software onto translucent displays positioned in front of your eyes. The final product is a digital image that looks like it’s sitting in the real world. The more advanced headsets allow you to physically walk around a projection to see it from all sides.
AR glasses and headsets rely on a variety of inputs from voice-activated controls to built-in touch capacitive sensors and motion controllers.
What can augmented reality be used for?
Augmented reality is poised to invade nearly every industry, but we should expect it to follow a similar growth pattern to VR, first finding its footing in the gaming industry before branching out into other fields. There’s no point in listing the potential use cases as they are practically endless, but we’ll highlight a few to show just how flexible the technology is.
AR is a useful tool for training employees and is already used in factories around the world. In fact, after Glass failed with consumers, Google pivoted to business applications. AR can assist factory workers by superimposing virtual tutorial videos on top of real objects. For example, if a piece of machinery breaks, AR can guide a worker through the process of fixing it in real time. The technology is also being used in the medical field to help document patient’s records and guide surgeons through delicate procedures before they attempt them on humans.
On the consumer side, businesses are using AR to give customers a better look at their products. IKEA, for example, is using AR to place digital furniture into your house so you can see if a product throws off the feng shui of your living room before making your purchase. The same idea is being used by clothing and shoe stores.
With Google Lens, your smartphone camera won’t just see what you see, but will also understand what you see to help you take action. #io17 pic.twitter.com/viOmWFjqk1
— Google (@Google) May 17, 2017
And with Google Lens, anyone with an Android device will soon be able to point their camera at a restaurant and see a menu, reviews, and business hours overlayed on top of a building.
We should expect to see an explosion of new uses for AR in the years to come thanks to the attention it’s getting from giants Google, Apple, and Microsoft. Until then, download an AR game on your phone and enjoy the magic of this sci-fi-inspired technology.