Monet鈥檚 paintings of gardens can make a viewer feel content, while Dali鈥檚 melting clocks elicit fear or confusion. Now, an AI art critic can predict the emotions famous paintings will evoke and often explain them as convincingly as a human.
AI image analysis often focuses on describing what is going on in pictures, but the subjective feelings that works of art arouse have just as big an effect on human behaviour, says Panos Achlioptas at Stanford University in California. 鈥淭he way we understand and interpret our world involves a lot of emotional reactions,鈥� he says.
Being able to predict and even emulate these responses could help machines interact with us more seamlessly, says Achlioptas, so his team built a massive data set of human reactions to art using online surveys.
Advertisement
They asked more than 6000 participants to choose the dominant emotion elicited by 81,000 paintings in and write a caption describing what it was about the artwork that guided their decision, with each painting analysed by at least five people. The images, emotion labels and captions were used to train an AI, which was then challenged to predict what emotion paintings it hadn鈥檛 seen before would evoke and provide short explanations.
Judging the output is inherently difficult, says Achlioptas, because there is no right answer. A majority of annotators agreed on the dominant emotion in just 45 per cent of paintings. So the researchers carried out a form of Turing Test by showing human evaluators a painting alongside a caption from the AI and one from an annotator, and asked them to guess which was written by a human. The AI鈥檚 captions passed as human 50 per cent of the time.
Achlioptas admits the AI鈥檚 captions aren鈥檛 as diverse or creative as human ones, but says the early results are promising and the data has been made open source so others can improve on the models.
鈥淚t should come as no surprise that with a sufficiently large amount of appropriate data and certain deep learning trickery, a model could be trained to generate decent captions with emotional content,鈥� says Simon Colton at Queen Mary University of London, who wasn鈥檛 involved in the work.
But the new data set is an important contribution to the field, he adds. An exciting prospect could be to use the data to train creative AI that generates visual art so it can better convey particular feelings, he says.
Reference:
