Singer-songwriter and billionaire Taylor Swift shares something in common with a growing number of non-celebrities: the appalling experience of featuring in fake sexual images made by strangers using artificial intelligence.
Barely a week after explicit AI‑generated images of Swift went viral online last month, more AI-manipulated images and videos appeared that falsely portrayed her expressing support for US presidential candidate Donald Trump. These incidents were preceded by a scam campaign that cloned the voices of Swift and other celebrities and used lip‑syncing AI tools to depict them promoting a fake prize giveaway.
Swift’s experience reflects the full range of troubling trends involving AI-generated images, audio and video called deepfakes. Now that they are trivially easy for anyone to create at little to no cost, is it too late to stop their spread?
Advertisement
Trouble, trouble, trouble
AI tools have already fuelled the rise of non-consensual deepfake pornography and financial scams that have duped people and corporations out of millions. Deepfakes are a growing threat to elections through misinformation and disinformation, too. “It is something of real concern that is happening now and is going to get worse fast,” says at the University at Buffalo in New York.
The first wave of deepfakes gained traction with the development of generative adversarial networks in 2014, a technique that pits AI models against each other, in this case to gradually improve the realism of fake images. Yet this generation of deepfake technology was time-intensive and required creators to have technical expertise in AI. Lyu says that making a 30-second video typically needed weeks of digital post-processing work and large amounts of computing power, while still falling short on realism.
But the rapid commercialisation of newer AI technologies in the past two years has made deepfakes easy to churn out. Anyone who can write a text prompt can create AI-generated images in seconds using products such as Midjourney, DALL-E made by OpenAI, and Stable Diffusion from Stability AI. Other tools let users clone a voice by simply uploading or recording an audio sample. And while some AI services have implemented safeguards against generating certain content – for example, an OpenAI spokesperson says their tools deny requests that ask for a public figure by name or for explicit content – many haven’t.
I’m afraid the ship has sailed on deepfakes, and it’s not just a ship – it’s a large flotilla, an armada
A , a team of online security experts, found there were more than 95,000 deepfakes online in 2023, up 550 per cent since 2019. It also revealed that 98 per cent of deepfake videos were pornographic, with 99 per cent using a woman’s likeness.
The pornographic Taylor Swift deepfakes were created by an anonymous group on the internet forum 4Chan in an effort to bypass safeguards on Microsoft Designer, a mainstream AI tool (Microsoft has since fixed the exploited vulnerabilities). “The [4Chan] community is really desensitised to the impact that this has on real humans,” says Cristina López G. at social media intelligence firm Graphika. She and Santiago Lakatos, also at Graphika, found that these deepfakes then spread to social media platforms Telegram and X (formerly Twitter) in under two weeks. One X post racked up more than 45 million views before the user’s account was suspended. In a statement, Telegram says the firm “has actively moderated harmful content on our platform, including illegal pornography” and has “removed every group sharing this illegal pornography that we know of”.
Not all non-consensual deepfakes involve celebrities. Teachers and school-age girls have also been targeted in and in the US, as well as countries including , , and . “It’s definitely being used to target women specifically and to intimidate, to threaten,” says at the University of Bologna in Italy. “It’s the same idea as revenge porn, but now you don’t even need actual intimate photos – you can just make them up.”
Deepfakes are also complicating elections. Last month, a robocall campaign used audio deepfakes of US president Joe Biden’s voice to discourage voting in New Hampshire. The that any such calls made with AI-generated voices are illegal. Deepfakes also targeted a candidate in in February 2023 as well as last month.
Just a single deepfake could disrupt national politics. “Even if it’s only once that you’re on the eve of the election in the United States and some deepfake of Joe Biden comes out… that could have a huge impact,” says at Cornell University in New York.
One such event may have already occurred during the in September 2023. An audio deepfake falsely depicting a party leader as planning to rig the election was posted to Facebook just before voters went to the polls. A rival party won the election.
Deepfakes could also be a national security threat if used to extract information or spur actions from government staffers or military personnel, says Kreps. “This technology is so prevalent now that we almost can’t scope the number of scenarios that is possible,” she says.
Troublingly, deepfakes of the political, pornographic or scam variety may become even more commonplace (see “Scammers use stolen voices”, below). “I’m afraid the ship has sailed, and it’s not just a ship – it’s a large flotilla, an armada,” says at Northwestern University in Illinois.
Reining it in
Solutions to the problem so far have been overly narrow. The European Union’s Digital Services Act requires tech platforms to identify and label AI-generated content. The US is developing similar standards. And the tech industry has embraced a standard called C2PA that would encode digital content with information about where it came from and who created it. Earlier this month, on its social media platforms, while Google said it would integrate a for its generative AI tools, .
Watermarking efforts from the big tech firms can be helpful, but they don’t stop malicious actors from using open-source AI tools, says Hany Farid at the University of California, Berkeley. They also fail to address violations of privacy and consent for those affected by deepfake pornography – especially when the aim is to harass and intimidate, says at Cornell University. “The focus on… whether the media was or wasn’t manipulated using AI is almost irrelevant to the actual issue,” she says.
At least six US states and several countries have made it illegal to create or distribute harmful deepfakes, and the US is considering a similar federal rule. But showed that places where sharing or viewing deepfake pornography is outlawed, such as Australia and South Korea, have some of the highest prevalence of creating, sharing or viewing such images and video. That is why tech platforms may have the most power and responsibility to stem the deepfake tide.
In a , Microsoft president Brad Smith wrote that the firm aims to protect freedom of expression without shielding distributors of non-consensual pornography. Google, Midjourney, Stability AI, X and 4Chan didn’t respond to a request for comment. Meta shared its existing policies on deepfakes, which its own Oversight Board this month called “incoherent”.
at the University of Pennsylvania says it is time for a societal reckoning over sexualised deepfakes – whether of Swift or anyone else. “The solutions aren’t necessarily rocket science. They just require us to reorient our values around the bodily autonomy of everybody and the privacy of everybody, including women.”
Scammers use stolen voices
Deepfakes are supercharging traditional scams. In 2023, the US Federal Trade Commission warned the public about “grandparent scams” that use the cloned voices of family members to trick people into sending money to help a loved one in a fake emergency.
Ambitious scammers have aimed for even more lucrative payouts by targeting businesses with the help of deepfaked executives. One of the first such incidents reported and involved the CEO of a UK-based energy company being tricked by deepfake audio of his boss. The CEO ultimately sent $243,000 to the scammer’s bank account.
Such scams have become more elaborate in recent years. This month, Hong Kong officials revealed that an employee of a multinational company was fooled into transferring more than $25 million in company funds. The scammers used deepfakes to create a Zoom call featuring a convincing video of the company’s chief financial officer along with other employees.
