Fake videos created by artificial intelligence are now so realistic that they are becoming useful.
Israeli tech firm Canny AI is one of several companies cashing in on so-called deepfakes, using the technology to dub videos into different languages. The firm is currently using it to dub advertisements or messages from celebrities for audiences in different countries and plans to use it for television shows and films in the future.
Deepfakes make it easy for people with a bit of technical know-how to create fake videos. So far, they have mostly been used to make pornographic films involving celebrities or to make videos where well-known figures appear to say something that they haven’t. Canny AI, for example, created a satirical deepfake of Facebook .
Advertisement
The firm’s technology requires a voice actor to provide replacement audio, unlike other algorithms that have learned to synthesise convincing fake speech, such as that of UK-based company Faculty, . The AI needs about a minute of speaking footage of both the person being deepfaked in the video and the voice actor saying the words that will be edited in.
The system learns to transfer the movements of the lower half of the dubber’s face and their neck into the video being edited. It works for both footage where the speaker is facing the camera and when they are side-on. The result is a video in which the speaker looks and sounds like they are saying the new dialogue.
The algorithm trains on the footage scene by scene, so dubbing five languages into one scene requires less processing power than dubbing for five separate scenes, says co-founder Omer Ben-Ami.
UK tech firm Synthesia also offers a dubbing AI, based on facial mapping. Their deepfakes were behind a malaria awareness campaign video in which David Beckham .
Using such deepfake AIs commercially brings up legal questions, says Lilian Edwards at Newcastle University in the UK. “There’s an underlying issue there about what parts of yourself you own,” she says. “Do you own your face, do you own your image, do you own the voices coming out of your face?”
These are questions that companies and celebrities will have to consider when entering into entertainment contracts, says Edwards.
“Every client we’re working with has to declare they are responsible for copyrights and liability issues,” says Ami. The AI is an example of how deepfakes can be used positively, he says.
“Any technology is a double-edged tool,” says Edwards. The danger is that as AI video-editing becomes more user-friendly and widespread, people could use deepfakes “to put words in the mouths of politicians, public figures, people they hate,” she says.
Canny AI co-founder Jonathan Heimann likens the growing awareness of deepfakes to the widespread acceptance that photographs can be easily edited. “The world will understand that videos can be faked too,” he says.
