Photobomber be gone. Editing objects, people, or unwanted details out of photos has long been a tedious, manual process – but an algorithm can do it at the swipe of a brush.
To remove an object from a picture, an artificial intelligence algorithm has learnt to fill in the gaps that are left behind. The system is able to recognise what content is present in photos it is asked to retouch, having been trained on photos including human faces, animals and landscapes. When filling in gaps left after editing, the system checks that the resulting image looks acceptable.
“Honestly I didn’t expect this to work so well,” says at Nvidia, a computer graphics card firm that created the system.
Advertisement
Picture perfect
The technique used by the AI is called partial convolution, and allows the system to check at successive stages whether its attempts to smooth over editing relate accurately to the rest of the image. It takes about a quarter of a second using a powerful graphics chip, says Catanzaro, but he thinks this could be made much quicker in future work.
Because the system relied on data from images it had been trained on when filling in, there were some unexpected results. At one point, the researchers removed a man’s eyes from a photograph – the system responded by replacing them with very feminine eyes.
“That’s because we trained it on a dataset that had more female eyes in it,” says Catanzaro.
And while a casual observer might not detect any signs of editing in one retouched image of a beetle, it would not fool an entomologist, as its mandibles were missing in the final version.
The technology may be of interest to social media firms, keen to encourage more image and video sharing on their platforms. Two years ago, Twitter acquired a start-up called Magic Pony that had developed image-enhancing AI.
But algorithms that make photo editing so easy may have darker uses. Authoritarian regimes sometimes remove people who have fallen out of favour from photographs. They will likely continue to rely on humans to do this – but automatic editing, which may similarly distort reality, could be abused in the hands of those wishing to spread misinformation online, says at the Georgia Institute of Technology. “If there’s a danger, it’s about the scale.”
Read more: Face-faking AI isn’t just for porn – it will change the world
Journal reference:
