You are dropped in an unfamiliar New York neighbourhood, with no idea where you are. All you have at your disposal is a guide on the end of the phone with a map. Together you must communicate to first work out where you are, and then how to get where you are meant to be going.
This isn鈥檛 the start of a new orienteering game, but a challenge created by Facebook for artificial intelligence. To master the task, AIs must look at the world around them, comprehend it, and communicate effectively. The aim is that by doing this, AIs will be able to better interact with humans, and each other.
For the challenge there are two AIs, one plays the role of a tourist and the other a guide. The tourist is dropped into one of five New York neighbourhoods and has a 360-degree view of the surroundings. It can move around, but doesn鈥檛 have access to a map nor its target location.
Advertisement
The guide has a map, dotted with landmarks including bars, banks, shops, theatres, hotels, and restaurants. The guide has to communicate with the tourist to work out where it is and direct it to the final destination.
To set the initial benchmark for the challenge, Facebook created several of their own pairs of AIs. The AIs were significantly better than humans 鈥� but only when communicating in a machine language, incomprehensible to people.
These AIs correctly reached the final destination nearly 90 per cent of the time, compared to humans that only managed to do so around 75 per cent of the time.
However, when the AIs had to communicate in English their performance drastically dropped to 50 per cent.
This shows just how hard communicating in natural language still is for machines. 鈥淚f you look at how humans communicate there are a lot of implicit assumptions about what we think other people already know,鈥� says Douwe Kiela at Facebook鈥檚 artificial intelligence research lab.
Currently, most AIs learn language through massive collections of text, like Wikipedia. But there is plenty we learn from the context in which language is used too, such as the way people say things or what else is going on at the time. 鈥淔or example, the smell of coffee is very easy to imagine, but very hard to describe in words,鈥� says Kiela.
As a result, when AIs communicate they often misunderstand what鈥檚 really being said because they lack the relevant knowledge. Facebook hope that by training AIs in context, such as meandering through the streets of New York, they would learn in a way more similar to humans.
鈥淭he goal is for AI systems to understand the world around them and to understand what people are actually talking about,鈥� says Kiela.
But anyone who has ever attempted a conversation with a digital assistant or chatbot will know, smooth talking AIs still have way to go. 鈥淭hat goal is very far away,鈥� says Kiela.
