Why did the machine think the turtle was a rifle? No, this isn’t a bad joke, but one of many recent examples of machines being tricked into seeing or hearing things that aren’t there.
Artificial intelligence is easily confused by so-called adversarial examples and like many others, at the Massachusetts Institute of Technology had thought they were bugs that would disappear with better algorithms or ways to train them.
But he and his colleagues have now discovered that the problem is more fundamental – adversarial examples seem to arise from features in images that we can’t perceive, but machines can. Early indications are that by understanding these features, it may be possible to stop adversarial examples wreaking havoc in the future.
Advertisement
Most adversarial examples can seem baffling to an onlooker with two images that look identical being interpreted in very different ways. For example, with two apparently identical images of a cat, an AI will insist one of them is actually a dog.
As a research experiment, fooling AIs can be amusing, but if a medical AI confuses a clearly obvious tumour in a medical scan, the results could be devastating.
Madry and his colleagues appear to have confirmed a long-held suspicion that AIs do not view images in a similar way to humans. Rather than solely relying on details like ear shape or nose length to classify images of animals, say, they use features that are imperceptible to humans.
“We don’t actually know what these features are — they may be big, or small — but the human brain doesn’t pick up on them,” says Madry.
The team call these patterns non-robust features because they seem particularly vulnerable to adversarial images.
Finding the culprits
To identify that non-robust features were causing the problem, Madry and his colleagues took a standard collection of images of cats and dogs and generated a series of adversarial examples. They then trained an AI on the adversarial examples, with each image mislabelled, in other words labelled as if an AI had been confused by the image.
Rather than resulting in something completely useless, when the AI was shown non-adversarial examples it classified them correctly, meaning the non-robust features it picked up in the training set, did actually help it to correctly identify cats and dogs – just not for adversarial images.
The team then trained an AI on a set of images with the non-robust features removed, essentially forcing it to use more human-like methods to make its choice. The result was an AI with significantly improved resistance to adversarial examples , reaching a level normally only seen when painstaking effort is used to correct an AI’s mistakes.
“Analysing how the properties of training datasets can affect a learnt system’s robustness to adversarial examples is a good step towards developing machine learning models that can be safely deployed in the real world,” says at research firm DeepMind.
Even so, the very fact that these features are imperceptible to humans, makes it difficult to see how this would be applied in practice, says at the University of Oxford.
Madry agrees that there is still a lot of work to be done. “It is not a solution, but an explanation and guidance towards the solution,” he says.
Reference: arXiv,Ěý
