Spies can eavesdrop on conversations by using radar to detect tiny vibrations in smartphones and employing artificial intelligence to accurately transcribe them. The trick even works in noisy rooms, as the radar homes in on the phone鈥檚 movement and is entirely unaffected by background hubbub.
Millimetre wave sensing is a form of radar that can measure movements of less than 1 mm by transmitting pulses of electromagnetic wave energy and detecting the reflected beams.
at Pennsylvania State University and his colleagues used a commercially available sensor operating between 77 and 81 gigahertz to pick up the tiny vibrations in a Samsung Galaxy S20 earpiece speaker playing audio clips. They then converted the signal to audio and passed it through an AI speech recognition model, which transcribed the speech.
Advertisement
Although the radar signal is unaffected by other noises in the room, Basak says extracting the smartphone鈥檚 vibration from extraneous movements produced by the person holding it is still like finding a needle in a haystack. The user鈥檚 breathing causes vibrations of a few millimetres and their heartbeat creates vibrations an order of magnitude smaller. But the movement of the earpiece from the conversation itself is another two orders of magnitude smaller: typically just 7 micrometres.
Because of this, the researchers had to employ statistical error correction techniques to tease out the signal from the noise, and even then the audio is scratchy and difficult to understand. But by using an AI model to transcribe the sound, they were able to achieve a word accuracy rate of 50 per cent and a character accuracy rate of 67 per cent from a range of up to 125 centimetres.
鈥淗umans typically don鈥檛 sit still, so in that situation, isolating everything and then taking down the vibrations is difficult,鈥� says Basak. 鈥淚t鈥檚 very difficult. That鈥檚 the caveat. There are very weak vibrations, but there are certain conditions in which you can tease them out.鈥�
The idea for the research came when Basak was rehearsing with his band. In order to play songs for the other members to hear, he held his phone up to the contact microphone installed in his acoustic guitar 鈥� a tiny device that creates an electrical signal when distorted by vibration 鈥� so that it was broadcast from his amp. By replacing the microphone and amp with a radar sensor and an AI model, the team has achieved something similar at a distance.
Basak declined to comment on whether government security agencies and militaries are likely to be working on such technology, or if he has been approached by anyone in the sector, saying only that 鈥済overnments are keen on new security technologies鈥�.
arXiv
