A robot dog chased down a ball and clambered over obstacles after learning the skills from images and video generated by artificial intelligence.
at the Massachusetts Institute of Technology and his colleagues developed their training platform 鈥淟ucidSim鈥� by taking a popular computer simulation software that follows the principles of real-world physics and inserting a generative AI model to produce artificial environments such as a stone pathway.
Advertisement
They also used OpenAI鈥檚 ChatGPT to generate thousands of text descriptions that were fed into the AI image generator to create the various images, from an outdoor concrete staircase framed with stone walls to a storage trunk topped with an orange traffic cone. The images also included different weather and lighting conditions, such as the sun shining dimly through low-hanging fog.
鈥淭his allowed us to generate 10 million images that are super-realistic looking but also physically correct,鈥� says Yang. Next, they transformed the AI-generated images into short videos to provide virtual training experiences for the robots, such as a short stairway with cones placed on various steps.
Their training programme analysed each scene鈥檚 3D geometry and calculated changes from the robot dog鈥檚 perspective as it moved through the virtual world 鈥� shifting each image detail within the scene to create a sense of motion.
The team then trained a robot to memorise actions in a simulation while relying on a map of the landscape, before letting it also explore AI-constructed visual environments by itself.
Despite never being able to 鈥渟ee鈥� the real world during training, the robot successfully chased real-world balls and climbed over objects 88 per cent of the time after the AI-enhanced training. When the robot relied solely on its skills learned while having access to the simulated landscape map, it only succeeded in the real world 15 per cent of the time.
The training shortcut could ultimately eliminate the need for traditional, often painstaking learning methods that involve large amounts of training data, while also boosting accuracy.
Article amended on 15 November
We clarified the type of training the robot initially received.
arXiv
