Mark Zuckerberg is fascinated by Iron Man. But it isn’t Tony Stark he’s interested in. It’s his AI-powered sidekick, Jarvis. Yes, the voice that speaks to Stark in his home and does everything possible to make the superhero’s life stress-free.
In fact, Zuckerberg was so inspired by the Iron Man’s artificially intelligent companion, he apparently decided to build it himself.
Mark Zuckerberg
The Facebook CEO wrote a blog post on his site yesterday detailing the creation of an AI system that could run his home. What he built is not dissimilar to recently released products like Amazon Echo or Google Home—except there is no physical product. Commands are given to a smartphone through voice or text. Zuckerberg also wants his AI to learn, adapting to the CEO’s tastes and patterns instead of simply responding to already-defined requests. Think Nest Thermostat.
His side project resulted in the creation of smart software that could control his home’s lighting, climate control systems, appliances, music, security, and even entertain his daughter Max. It uses a number of AI techniques including natural language processing, speech recognition, facial recognition, and reinforced learning to teach devices how to react to commands.
Zuckerberg: ‘Jarvis uses face recognition to let my friends in automatically and let me know.’
Screengrab via Mark Zuckerberg/Facebook
Zuckerberg used Facebook Messenger to speak to or text his Sonos system with Spotify, a Samsung TV, and a Nest camera for his one-year-old daughter. The CEO claims one of his biggest challenges was figuring out how to connect devices that weren’t already “smart.” He proved he wasn’t only an inventor in the virtual space but also the physical world, rigging old appliances with new sensors.
“But one aspect that was much more complicated than I expected was simply connecting and communicating with all of the different systems in my home. I ended up finding an old toaster from the 1950s and rigging it up with a connected switch. Similarly, I found that connecting a food dispenser for Beast or a grey t-shirt cannon would require hardware modifications to work.”
Zuckerberg says his project did not take as long as he had initially thought it might. He only spent around 100 hours building Jarvis, but claims even 1,000 more hours wouldn’t be enough to reach his ultimate goal of building a system that could learn completely new skills on its own.
“In some ways, this challenge was easier than I expected. In fact, my running challenge (I also set out to run 365 miles in 2016) took more total time. In the longer term, I’d like to explore teaching Jarvis how to learn new skills itself rather than me having to teach it how to perform specific tasks. If I spent another year on this challenge, I’d focus more on learning how learning works.”
Machine learning and the internet of things (IoT) are hugely trending topics in technology. Analysts believe connected devices are on their way to seeing exponential growth. So what are the chances your home will soon be equipped with Jarvis-like AI? Not so great.
Zuckerberg says he considered open-sourcing his code, but the system is too closely tied to his home and network configuration at the moment.
“If I ever build a layer that abstracts more home automation functionality, I may release that,” Zuckerberg wrote. “Or, of course, that could be a great foundation to build a new product.”