DeepMind has developed an artificial intelligence that can solve tasks it has never seen before as fast and as accurately as humans – a possible step towards generally intelligent AI that could master an array of jobs in the real world.
The AI, called Adaptive Agent or AdA, works in a 3D virtual world where it is asked to solve tasks that involve navigating, planning and manipulating objects.
Humans are excellent at solving new problems in very different environments, including ones they haven’t seen before. For instance, once you have learned to drive a car in one country, it is relatively straightforward to drive in another.
Advertisement
AI systems, however, tend to be able to solve only the specific problems they are trained to do – when the rules of the game change, they can struggle.
Much of DeepMind’s success with game-playing AIs, such as its chess-playing AlphaZero, has come from what is called reinforcement learning, where an AI is taught what success looks like and is left to figure out the rules, and how to succeed, using trial and error. It is essentially told to do whatever it can to win.
The AI would have to be trained all over again to be able to play a different game, though.
To get around this limitation, DeepMind has been developing an AI that can succeed at games it hasn’t seen before. In 2021, it that could solve new tasks in a virtual 3D world, called XLand. This environment contained more than 700,000 games, from well-known examples like hide and seek and capture the flag to more abstract games that had been automatically generated and existed only within the virtual world.
The team has now developed this AI by training various versions of it on a set of billions of tasks of slowly increasing difficulty – allowing between one and six tries on each task, whether they succeed or not – for the equivalent of 100 human years.
The virtual world AdA lives in has 1042 tasks, a number far larger than the number of observable stars in the universe, and it has to learn a system of rules, similar to laws of physics, that can be adjusted and changed.
When the DeepMind team tested the AI on a set of 30 custom-designed test tasks it hadn’t been trained on, it performed as well as and as quickly as 100 human participants in solving them, and could solve some tasks that the people couldn’t. On a larger, automated set of 1000 tasks that it hadn’t seen before, it could solve about 90 per cent of them.
The model also seemed to display some unexpected aspects of intelligence. On tasks that required multiple in-game agents to solve them, the system could control several agents, displaying coordination and cooperation, kinds of intelligence that are hard to produce in AI models.
As well as creating tasks that only work within the system, such as matching two different coloured shapes to produce another specific block, the DeepMind team translated several levels of a popular cooperative cooking game, Overcooked, in which players have to work together to produce food in a kitchen. The AI could cooperate to solve these, too.
DeepMind declined to comment on the work, but at the University of Bath, UK, says: “This work is quite a step forward, showcasing an agent that can develop, in some sense, a deep understanding of its environment and quickly learn how to solve new problems in this environment.”
An AI that can solve tasks in constantly changing environments could be useful for a range of real-world uses, from a humanoid robot carrying out manual labour to a self-driving car learning new driving environments.
The fact that DeepMind’s AI can solve unseen problems in a similar amount of time to humans is impressive, says Şimşek, but it doesn’t mean it can handle life outside its virtual environment. “It’s a closed world, lacking the types of uncertainty that we generally have to deal with in the real world.”
Reference:
