A humanoid robot that can perform actions based on text prompts could pave the way for machines that behave more like us and communicate using gestures.
Large language models (LLMs) like GPT-4, the artificial intelligence behind ChatGPT, are proficient at writing many kinds of computer code, but they can struggle when it comes to doing this for robot movement. This is because almost every robot has very different physical forms and software to control its parts. Much of the code for this isn鈥檛 on the internet, and so isn鈥檛 in the training data that LLMs learn from.
Advertisement
at the University of Tokyo in Japan and his colleagues suspected that humanoid robots might be easier for LLMs to control with code, because of their similarity to the human body. So they used GPT-4 to control a robot they had built, called Alter3, which has 43 different moving parts in its head, body and arms controlled by air pistons.
Ikegami and his team gave two prompts to GPT-4 to get the robot to carry out a particular movement. The first asks the LLM to translate the request into a list of concrete actions that the robot will have to perform to make the movement. A second prompt then asks the LLM to transform each item on the list into the Python programming language, with details of how the code maps to Alter3鈥檚 body parts.
They found that the system could come up with convincing actions for a wide range of requests, including simple ones like 鈥減retend to be a snake鈥� and 鈥渢ake a selfie with your phone鈥�. It also got the robot to act out more complex scenes, such as 鈥淚 was enjoying a movie while eating popcorn at the theatre when I realised that I was actually eating the popcorn of the person next to me鈥�.
鈥淭his android can have more complicated and sophisticated facial and non-linguistic expressions so that the people can really understand, and be more empathic with, the android,鈥� says Ikegami.
Though the requests currently take a minute or two to convert into code and move the robot, Ikegami hopes human-like motions might make our future interactions with robots more meaningful.
The study is an impressive technical display and can help open up robots to people who don鈥檛 know how to code, says at the University of Manchester, UK. But it doesn鈥檛 help robots gain more human-like intelligence because it relies on an LLM, which is a knowledge system very unlike the human brain, he says.
arXiv
