Tricking artificial intelligence has never been easier. One way is to fool an AI into misclassifying an image by misinforming it about what that file shows. Such 鈥渁dversarial examples鈥� can now be generated a thousand times faster than before.
at the Massachusetts Institute of Technology and his colleagues have created an algorithm that produces a thousand adversarial examples to fool Google鈥檚 image recognition system 鈥� Google Cloud Vision. One was an image of a dog that was wrongly identified as a picture of two people skiing. In principle, each attack could be completed in several minutes, Athalye says.
The system works by taking an image and changing it pixel by pixel into a different one. Throughout the process, it pings Google Cloud Vision to see how it classifies the image, and makes sure that it still thinks it鈥檚 looking at the original, even though the final image is completely changed to a human eye.
Advertisement
Image recognition relies on algorithms called deep neural networks and they can often make mistakes that look silly to humans. For example, earlier this year the MIT team showed that the same principles apply in the physical realm by tricking a neural network into thinking that a 3D printed turtle was a rifle.
What is most mysterious is how an image that fools one network can also fool another, says at Wyoming University. He has produced an image that more than one deep neural network was over 99 per cent confident was a starfish, yet to humans it just looked like a load of squiggly lines. 鈥淚t is as if all the deep neural networks in the world are sitting around saying 鈥榃hy don鈥檛 these silly humans realise this is a starfish?鈥欌�� says Clune.
The potential impact of adversarial examples on any software that relies on image recognition, like self-driving cars or facial recognition security technology, is worrying. If a driverless car failed to spot a pedestrian or a security camera misidentified a gun the consequences could be incredibly serious.
However in reality, an attack like this might not be so easy to execute. If the attacker has to query the system thousands of times from the same IP address to generate the example, they would be detectable, says at Vanderbilt University. 鈥淚t remains to be seen how plausible such an attack would be in practice,鈥� says Vorobeychik.
And Clune is sceptical that Athalye鈥檚 approach is much faster than other methods for producing adversarial examples. 鈥淲hat is new here is that they use a different technique to generate the samples, but it is not clear that is more efficient as they do not compare these methods,鈥� he says.
Reference: ArXiv:聽
