Over the weekend, a computer successfully convinced a panel of judges that it was human, passing what is known as the Turing Test, a question laid out by famed mathematician Alan Turing regarding the capacity for thought in machines.
The Turing Test applies Turing’s question by assessing whether a computer can be mistaken for a human more than 30 percent of the time during five-minute conversations with judges.
At the Turing Test competition held by the University of Reading, the organization announced that, for the first time, a computer had passed the true Turing Test. A computer called “Eugene Goostman” developed by Russian succeeded, tricking the judges 33 percent of the time.
This begs a question: Does Mitchell Kapor owe futurist Ray Kurzweil $20,000? In 2002, Kapor and Kurzweil established that hefty wager over whether a computer would pass the Turing Test by 2029. They called it “A Long Bet.” Kapor, who co-founded the Electronic Frontier Foundation and was the first chair at Mozilla, bet against a computer passing the Turing Test by 2029. Kurzweil, a well-known futurist who works for Google, believed (correctly, as we now know) it would happen.
So who has to pay up?
No one, at least not yet. While “Eugene Goostman” did indeed successfully trick people into thinking it was human, Kapor doesn’t have to shell anything out to Kurzweil because the experiment didn’t meet all of the criteria laid out in the rules for the bet. For instance, the test this past weekend required five minute conversations, but Kurzweil and Kapor’s test was far more stringent, requiring a total of eight hours of conversation between the judges and the test participants. Then, instead of a 30 percent success rate, the computer would need to convince two out of three judges.
So while “Eugene” is getting acknowledged as the first computer to pass a Turing Test, the version of the test laid out by Kurzweil and Kapor requires a different set of standards, ones not met by any machine yet.
But Kapor still has plenty of reason to worry. Kurzweil has another 15 years until he loses, and if computers are already passing different iterations of this test, it’s likely developers will make one capable of more advanced trickery in a decade and a half. Better start saving up.
Update: According to the EFF, the bet is most certainly still not settled. “Perhaps the recent development means it would be a good time for Kapor and Kurzweil to hold a competition. But I doubt that the recent test victory is what even Kurzweil originally had in mind,” the EFF’s Dave Maass wrote in an email.
Joseph Bonneau, a fellow at Princeton’s Center for Information Technology Policy, was sceptical that this recent experiment even counts as a Turing Test “Most certainly not, and by any reasonable definition this should not count as a “pass,” he wrote. “Ignoring the 30 percent bar they used instead of 50 percent (which was based on the accuracy Turing guessed would be doable by the year 2000) they used cultural and age limitations to make the test much, much easier. It’s easy to “pass” a Turing test by claiming to be an infant or young child barely capable of comprehensive speak. It also would be very easy to “pass” a Turing test by posing as a human who only speaks Bengali to English-speaking evaluators. This test passed by claiming to be a 13-year old Ukrainian kid which is a big loophole. Passing certainly requires simulating a literate adult with no language barrier,” he continued.
Photo via Flickr/ell brown (CC BY 2.0)