Telling a robot where to go without having to speak like a robot – that is, with so-called natural language commands – is now a lot easier, thanks to a new model based on how people actually speak when giving directions.
Currently, many robots use simultaneous localisation and mapping (SLAM) to know where they are in a given landscape. They have to concurrently keep track of their location on a map, while constantly updating their knowledge of the environment.
“SLAM is powerful and is great,” says Jason Corso at the University of Michigan. “The challenge for humans to interact with SLAM-based machines is we need to think on their terms. It’s really rigid and we have to adapt to the robots. The goal is to flip that and have the robot adapt to the human language.”
Advertisement
Corso and his colleagues moved a robot around a 2.5-by-2-metre tabletop maze configured in 116 different ways. They had one person, a driver, remote-controlling the robot without being able to see the maze. Another person, a navigator, gave them directions about how to solve the maze from another room using an online chat.
As the robot moved around, a natural language processing model translated the orders the navigator sent to the driver. Once a list of phrases used was developed, the model that translates them was trained in a simulation – the team couldn’t test this in a lab, as access was restricted due to coronavirus measures. The model learned to follow plain-text commands.
“We collected a huge amount of data on real robotic systems with real human input,” says Shurjo Banerjee at the University of Michigan. “This is data we want the field as a whole to look at.”
It is a significant step, says William Wang at the University of California, Santa Barbara. “Language is the most natural form of communication with robots.” He says this work adds to previous research in this area by testing natural language commands in a physical environment and creating a “robot dialogue data set” that could be used in other settings.
Wang hopes that the findings in this paper can be used to improve the use of robots in real-life situations, such as disaster relief, healthcare support and in-home assistance.
Reference:
