快猫短视频

Rewarding robots for good behaviour

MARVIN the Paranoid Android is made flesh. Well, almost. The new bot isn鈥檛 as pitiful as Douglas Adams鈥檚 brainchild in the Hitchhiker鈥檚 Guide to the Galaxy. But it is capable of feeling disappointed.

Its creator designed the robot to mimic the 鈥渉appy鈥 feeling you get when you鈥檙e rewarded. And just like its living cousins, the robot feels let down when it expects a treat and doesn鈥檛 get one.

Olaf Sporns at Indiana University in Bloomington has wired his robot so it can see, approach and grip coloured objects, and then test their conductivity. If the conductivity is low, the robot considers the find a reward. If it is high, the bot feels the find is a punishment. But the robot does more.

Its 4000 neurons are organised in a similar way to an area of the brain called the ventral tegmental area. When monkeys are rewarded with a banana or a toy, the neurotransmitter dopamine is released in the VTA, leading to speculation that a similar release in people corresponds to feeling pleased or happy.

Sporns designed his robot to generate an electronic version of this chemical response, and found that he could reproduce one of the classic results in animal experimentation. When animals are taught to associate their reward with a particular stimulus, the pattern of dopamine release changes. Eventually, the stimulus triggers dopamine release even before the reward appears.

The robot learned to do the same. And if the reward wasn鈥檛 forthcoming quickly enough, certain circuits slowed down, just as dopamine levels are known to drop in monkeys who expect a reward during an experiment and don鈥檛 get one. In effect, the robot was disappointed, Sporns told the International Conference on Machine Development in Cambridge, Massachusetts last month.

Sporns hopes his creation will help us understand how animals learn in the wild. The more complex the environment he taught his bot in, the less depressed it got.

More from 快猫短视频

Explore the latest news, articles and features