Some artificial intelligences can perform tasks with superhuman ability, but just how clever are they overall? As smart as a honeybee? A Labrador? Or a chimp? A competition called the Animal-AI Olympics will pit AIs against tests normally used to study animal intelligence.
From April, AIs will battle it out in a virtual playground for a $10,000 prize pool. All the tasks involve retrieving a piece of food, but the skills needed to succeed vary in complexity. This mimics real-life experiments used to measure animal intelligence.
Entrants will complete tasks they haven’t seen before to eliminate the opportunity to swot up beforehand, says Matthew Crosby at the Leverhulme Centre for the Future of Intelligence, UK.
Advertisement
The Animal-AI Olympics will test a range of cognitive skills, such as the ability to reason, navigate and learn from past experiences. “One key concept is object permanence, the understanding that objects continue to exist even when they’re out of sight,” says Crosby.
Although it won’t be in the competition, the A-not-B task is one such test of this ability. In the task, an animal is repeatedly presented with two cups, A and B. For the first few iterations, cup A always contains a piece of food.
However, once the animal is trained to understand this, the experimenter switches the food to cup B in plain sight. Some animals, such as dogs, continue to persevere with cup A, but others, such as macaques, instantly switch to cup B.
Tested to the limit
The variety of tasks in the competition will challenge one key limitation of many AIs: once they learn something, it is very difficult for them to adapt that knowledge to a similar but different situation. For example, one AI can outperform humans at the video game StarCraft and another beats us at the board game Go, but they are both useless at most other tasks unless completely retrained.
A will be provided during the competition, which runs until November.
The big hurdle will be to build AIs with general intelligence: systems with common sense and the ability to do a wide range of tasks based on limited data, says Chris Summerfield at the University of Oxford.
Many AI algorithms mimic certain functions in animal brains, such as the visual system in primates, he says. This is why image-recognition software, such as used by Google’s reverse image search, has been so successful.
But AIs lack many other brain features that contribute to cognitive ability, including short‑term memory or future planning. This may explain why AIs are good at specific tasks, but struggle to adapt to others.
Testing AI systems in unfamiliar environments is an important step to creating AIs that can solve a wide range of problems beyond those they were initially designed for, says Crosby.
“We expect this to be a hard challenge,” he says. “A perfect score will require a breakthrough in AI, well beyond current capabilities. However, even small successes will show that it is possible, not just to find useful patterns in data, but to extrapolate from these to an understanding of how the world works.”
