Facebook wants to train computers to be smarter. To determine just how smart machines can be, the company’s researchers developed something of an IQ test that uses logical questions most humans can answer and put them to the test against artificial intelligence.
The tests determine a particular machine’s reading comprehension based on a series of simple questions that require only one-word answers. For example, “Mary moved to the bathroom. John went to the hallway. Where is Mary?”
Facebook scientists want to eventually build artificial intelligence that can communicate with a human being and respond to reasoning and emotional conversation. Though the questions and computers aren’t quite so advanced, they’re the first step in the eventual evolution of human-like AI.
By putting machines through these tasks, scientists can figure out where each system succeeds or fails.
The scientists described their work in a paper entitled Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks:
Our tasks are built with a unified underlying simulation of a physical world, akin to a classic text adventure game (Montfort, 2005) whereby actors move around manipulating objects and interacting with each other. As the simulation runs, grounded text and question answer pairs are simultaneously generated. Our goal is to categorize different kinds of questions into skill sets, which become our tasks.
In a blog post detailing some of the methodology in the so-called bAbI project, researchers said there are 1,000 questions for training and 1,000 for testing, though the goal is to use as little data as possible to complete the question or puzzle.
The beginning of the computer’s intelligence assessment gives two statements, and asks a question. And it gets more complicated from there. The numbers next to the answers correspond to where the correct answer can be found in the preceding story.
1. Mary moved to the bathroom.
2. John went to the hallway.
3. Where is Mary? bathroom 1
4. Daniel went back to the hallway.
5. Sandra moved to the garden.
6. Where is Daniel? hallway 4
7. John moved to the office.
8. Sandra journeyed to the bathroom.
9. Where is Daniel? hallway 4
10. Mary moved to the hallway.
11. Daniel travelled to the office.
12. Where is Daniel? office 11
13. John went back to the garden.
14. John moved to the bedroom.
15. Where is Sandra? bathroom 8
While some of the tests are readable by humans, researchers also created a “shuffled” version, meaning the letters of the text are completely unreadable, and therefore require the machine to rely on previously-learned puzzles to correctly answer the question.
Sbdm ip im vdu yonrckblms.
Abf ip im vdu bhhigu.
Sbdm yigaus ly vdu hbbvfnoo.
Abf zumv vb vdu aivgdum.
Mduku ip vdu hbbvfnoo? A:yonrckblms
Mduku znp Abf fuhbku vdu aivgdum? A:bhhigu
While Facebook is trying to make machines smarter through research and open-sourcing its AI and machine learning tools, some scientists warn that it’s impossible to predict what could happen when machines eventually think for themselves. Stephen Hawking and Elon Musk have both discussed potential threats and necessary risk-avoidance when pursuing artificial intelligence.
For now, computers haven’t presented us with this dilemma. But clearly, Facebook plans to make them smarter.
H/T New Scientist | Photo via IntelFreePress (CC BY 2.0)