HE MAY have helped to crack the Nazis鈥� Enigma codes, but Alan Turing鈥檚 test for computer intelligence is an out-and-out failure, say artificial intelligence experts. They reckon IQ tests designed for three-year-olds are probably a better test of a machine鈥檚 capacity to think.
The British mathematician proposed his eponymous test in 1950 at the dawn of the computer age. His idea was that if a machine can fool a panel of human judges into believing that it is human 鈥� through a casual electronic conversation 鈥� then it can be said to be intelligent. But recently, software based 鈥渃hat bots鈥� have proven more than good enough at everyday chit-chat to regularly fool the judges.
So critics say being good at chatting is no real test of human-like intelligence, especially since most of the best bots deliberately make all-too-human goofs like spelling and grammatical mistakes to fool judges. 鈥淚t鈥檚 just too easy to cheat,鈥� says Sam Adams, an AI researcher with IBM in Research Triangle Park, North Carolina. Adams leads a project, code-named Joshua Blue, which is aiming to build a computer-simulated mind with the common sense of a three-year-old child. The idea is to emulate the developmental skills that infants must acquire to develop adult intelligence.
Advertisement
But Adams had no test that would let him know if he was making progress. The Turing test only assesses intelligent behaviour, rather than the processes believed to constitute human intelligence. 鈥淲e need to be able to measure incremental progress,鈥� he says.
Fortunately, tests that assess human-like intelligence do exist 鈥� in psychology. Psychometric tests are often given to children, for example to look for learning disabilities, and to assess their understanding of space, logic, language, social situations and the goals of others, says Adams鈥檚 colleague, psychologist Nancy Alvarado at the University of California in Santa Cruz. 鈥淚 don鈥檛 think these tests are all you need for intelligence, but if you can鈥檛 do them at all, you鈥檙e not getting very far,鈥� she says.
One example of a psychometric challenge a bot might take is the Wisconsin Card Sorting Test, in which cards must be sorted into four piles. Each time a card is put down, they鈥檙e told if it鈥檚 in the 鈥渞ight鈥� or 鈥渨rong鈥� pile. People quickly learn the rule 鈥� that the choice of pile depends on what shape is printed on the card, for example. But as fast as people get the rule, the tester changes it to see whether they can adapt.
Another example is a multiple choice comprehension test 鈥� of the type designed to see whether children have understood something they鈥檝e read. Adams says this would test much better whether a bot can distinguish between subtle shades of meaning.
The pair are now trying to adapt these and other tests for bots. But they鈥檙e wasting their time, says philosopher John Searle of the University of California at Berkeley. He says we鈥檒l never know if there鈥檚 genuine understanding going on inside a bot, no matter how clever the tests get. 鈥淚 think the whole idea of the Turing test is confused, and efforts to improve it typically compound the confusion.鈥�