Apple will inspect every photo uploaded to the cloud by US users of iPhones and iPads to detect images of child sexual abuse, and will report any found to a nonprofit that investigates cases of child exploitation. The new measure has been praised by child welfare charities but condemned by privacy campaigners, who believe it opens the door to other types of surveillance from authoritarian governments.
Rather than examining the photographs themselves, will include an algorithm that creates a string of characters – a hash – unique to each photo on the device. It will then compare these to hashes of known abuse images from child safety organisations. It says that this will be done on the customer’s device rather than company servers so Apple will never see the contents, even when flagged as possible abuse images.
Any positive matches will trigger a human double-check of the photo’s hash within Apple and if this confirms a match with the hash of a known image containing child sexual abuse, the information will be reported to the US nonprofit organisation , which will pass them on to law enforcement to decide whether to make a legal request either to see the images or for information about the device’s owner. The feature will launch later this year in the US as an update to iOS and iPadOS.
Advertisement
Andy Burrows at the UK says he would welcome Apple launching the system in the UK, too. While Facebook refers millions of images of child sexual abuse to authorities each year, Apple refers just a handful because its messaging system and image storage systems are encrypted and it cannot tell what content is being sent or stored. “It’s an innovative and very welcome step, and it’s one that we’d like to see extended to the UK as quickly as possible,” says Burrows.
Of all the technology giants, Apple is considered to be the strongest on privacy. Its business model is more about selling high-end devices and services than collecting data on customers and advertising to them, so it can differentiate itself by taking a strong stance on protecting its customers’ data. But neuralMatch has sparked a backlash among campaigners and academics. An open letter calling for the technology to be scrapped has been signed by privacy groups, academics and 6000 individuals.
at the Electronic Frontier Foundation says the tool is “a backdoor to encryption” that will ultimately allow governments to snoop on the photos we take, with users not necessarily aware it is happening.
“We all agree that child exploitation is a serious problem,” she says. “But it’s only a matter of time before lawmakers and governments seek to use this backdoor technology to filter out speech they don’t like.”
at Privacy International agrees that the tool risks “opening the door to mass surveillance” around the world. “By opening the floodgates, even for something as important as protecting children, Apple and the rest of the industry will inevitably be unable to resist doing the same for other reasons and for other governments,” he says.
Apple didn’t respond to a request for an interview but said that it would refuse any requests from governments to turn the technology on images other than those of child sexual abuse.
“We have faced demands to build and deploy government-mandated changes that degrade the privacy of users before, and have steadfastly refused those demands. We will continue to refuse them in the future,” it said.
But there is certainly cause for concern about what authoritarian governments would request Apple tracks using neuralMatch if the service launched globally. Apple has at times from handsets sold in a few countries including Saudi Arabia by government request, and a recent New York Times investigation revealed that have been made for the Chinese government also.
Currently neuralMatch will be used only in the US, but at law firm decoded.legal says it is unlikely that Apple would build such a complex system only to use it in one country. Many believe it will gradually be turned on around the world.
Brown says that the feature is likely to be allowed under EU law, which demands that anyone collecting data on individuals either seeks their consent or uses only what is absolutely needed to run a service for that person. It is possible that Apple could make the case that preventing images of child sexual abuse on its cloud service is an essential measure in providing that service, he says.
Brown, too, points out that the system could easily be misused by governments. Alternatively, media companies may be keen to use a version of the software to catch people sharing copyrighted content. “Where does it stop?” he asks.
There are also fears that neuralMatch could provide false positives. Although Apple says that the system will see less than one in a trillion false positives every year, at Princeton University says it is difficult to see how it arrived at that number.
“Even if Apple did test on real accounts, there aren’t a trillion iCloud users – and you’d actually need even more accounts than that to have statistical confidence,” he says. “Ultimately a system like this depends on trust in Apple, NCMEC and other involved parties.”
Mayer has urging caution on systems like neuralMatch because of issues with privacy and false positives. “It’s disappointing that, just as we were urging caution and further research, Apple was barreling forward with implementation. And it’s even more disappointing that Apple hasn’t meaningfully acknowledged those problems, let alone explained how it’s going to address them,” he says.
