Beware stealthy sound waves (Image: Sam Falconer)
No need to upgrade: with just its microphone, your phone can pay for a taxi or join Lady Gaga’s light show. But who else is whispering in its tiny ear?
IT WAS four years ago when Dragos Ruiu noticed that some of his computers were behaving bizarrely. First, software on a laptop changed unprompted. Then settings on other machines mysteriously altered – and data vanished. Most alarming of all, Ruiu claimed his computers seemed to be communicating with each other, even after he had severed every connection between them he could think of. It made no sense.
Advertisement
Ruiu, a respected information-security expert based in Edmonton, Canada, began to suspect that a powerful new computer virus was at work. Yet all efforts to find it failed. Then he discovered the only way to regain full control of a computer was to disconnect its microphone and speakers. In October 2013, he announced on Twitter that the virus, which he dubbed badBIOS, appeared to use high-frequency sound waves to send commands from one machine to another. Making use of speakers, microphones and some very smart programming, this malware seemed to be communicating through inaudible whispers.
It was an extraordinary claim. Many organisations, including the security services and those in charge of critical infrastructure, rely on so-called “air gaps” to protect key computers from intruders. In response, some pointed the finger at state-sponsored hackers. Others noted the lack of real evidence, and suggested that it was either a hoax or that Ruiu was paranoid.
In December last year, Ruiu’s claim began to look more credible. German researchers announced they had found a way to send data covertly between computers using inaudible sound waves. Perhaps mysterious sonic signals really were controlling Ruiu’s machines.
“Sonic signals take control of an audiences’ cellphones, making them part of the show”
Whatever the truth, there are good reasons why we can’t afford to ignore the idea. It turns out that old-fashioned sound technology is returning to play a key role in beaming data between all manner of devices, including cellphones. Want to pay for dinner at a restaurant or a cab ride home, share photos with friends, receive discount shopping offers or information at concerts, exhibitions and sporting events? A host of new services use speakers and microphones rather than wireless signals to make payments or swap information. Broadcasters are getting involved too – beaming out high-frequency audio during your favourite TV quiz so that you can play along at home on your smartphone. Could sound even deliver the all-purpose electronic wallet that will finally see off credit cards and cash?
But just suppose Ruiu is on to something. Given that the security systems protecting our devices are not designed with sound in mind, how secure can sound-based technology be? Could our gadgets become vulnerable to malware embedded in a voice call or music broadcast, say? And what about privacy? Will you have to disable your smartphone’s mic whenever you leave home, or risk a barrage of acoustic spam?
It is easy to forget that sound was once a favourite way to communicate over long distances. For centuries, booming cannons or clanging bells were used to send important messages. Even the dawn of the internet age was accompanied by a chorus of clicks, hisses and squeaks, as dial-up modems squirted data along phone lines.
Since then, a relative silence has descended as Wi-Fi and Bluetooth have taken over along with other technologies based on radio waves. Yet the incredible number and diversity of mobile gadgets means that engineers are taking a fresh look at acoustics.
Whenever manufacturers want to introduce new communications technology or even update old ones, they have to spend money on new hardware. Even then, their advanced gadgetry won’t be able to communicate with every other device.
However, pretty much every one of the billions of phones, tablets and computers we use already have speakers and microphones built in. To get them talking a common language, all that’s required is a software update. “There are lots of great ways to move data, but none are as universal as sound,” says Patrick Bergel of Animal Systems, based in London. “There are more loudspeakers on Earth than there are humans.”
In 2012 Animal Systems launched Chirp, an app that allows people to share files and other information via short sequences of audible birdsong-like calls. It’s just one of dozens of new systems that allow devices to sing, chirrup or whisper to each other.
Although Bergel believes we will take more easily to audible sounds like electronic warbling, other companies are opting for inaudible, ultrasonic sound waves. Among the earliest adopters are TV broadcasters in Europe and North America. They see audio as a simple low-cost way to add . So far, programmes including the BBC’s Antiques Roadshow and MTV Spain’s reality show have used inaudible signals buried in the soundtrack to synchronise software on viewers’ gadgets, allowing them to play along with quizzes or receive behind-the-scenes video.
If it works in the living room, why not at public events too? That’s the idea of New York City-based Sonic Notify, which hopes the technology will spark new kinds of interactions with fans at concerts or sports venues. The company’s software can translate ultrasonic chirps into digital codes which release web content on to smartphones. It even created an app that allowed fans at a Swedish House Mafia concert to make their phones part of the event’s light show. The technology has been tested at concerts by other artists including Lady Gaga and Katie Perry (see “Work the crowd“).
Hear that money
Compared with wireless communication, Chirp and its ilk must overcome serious limitations. First, they are vulnerable to interference from background noise, especially over longer distances. Yet this is less of a problem if you want to deliver content over short distances at specific locations. For instance, the Golden State Warriors, a basketball team based in Oakland, California, has launched a network of Sonic Notify beacons around its arena to deliver seat upgrade offers to fans in specific parts of the stadium along with extra web content and ads.
The major headache for acoustic transmission is data rate. “If we compare it with Wi-Fi or Bluetooth, sound is orders of magnitude slower,” says Ramarathnam Venkatesan of Microsoft Research India, which has developed a prototype sound-based mobile data-transmission system. It will never allow you to send large amounts of data, he says: audio signals can carry data at speeds of up to 2.4 kilobits per second – wireless networks are thousands of times faster.
Yet transmission speed isn’t a big issue for the new wave of sound-based apps for money transfer and shopping that are in development or have just launched. These use audio signals to simply pair up your smartphone with a pay point. Last year, for instance, Verifone released an app called that allows you to pay for a cab in New York City by tapping your cellphone on a reader. Your phone listens for inaudible sound cues emitted by speakers in the cab which identify the taxi and the customer, and then you confirm the payment.
Filling niches like taxi payments is just the start. Some hope audio will be the key to a cashless utopia, where even wallets and payment cards are history. At least that’s the bet being made by investors including PayPal-cofounder Peter Thiel and entrepreneur Richard Branson, who have stumped up around $30 million to back Clinkle, an all-purpose smartphone payments system due for launch this year, which uses Way2ride-like technology. Clinkle’s founder, Lucas Duplan, is a former computer science student at Stanford University in California. He plans to begin offering Clinkle on US college campuses, using incentives and reward schemes to get the ball rolling. He points out that store owners won’t have to buy new hardware – they can use their own laptops or tablet computers, unlike competing electronic payment systems such as Google Wallet or EE’s Quick Tap scheme.
Yet if we’re going to exchange sensitive data like bank account details using sound, will these transactions be secure? Could someone nearby simply listen in and steal your sensitive information or cash?
This wouldn’t be straightforward, says security expert Apu Kapadia from Indiana University in Bloomington, principally because audio transfer only works over short distances, so a hacker would need to be close. However, he concedes that it is possible to buy or make specialised eavesdropping equipment that works over long distances.
Alternatively, hackers could infect your smartphone before trying to grab your audio data. A estimated that 42,000 apps in Google’s Play store contain spyware and data-stealing trojan programs. Such malware could in theory be used to remotely activate microphones and steal data.
As a warning, Kapadia and his colleagues constructed proof-of-concept malware for Android phones capable of stealing valuable audio data. Called Soundcomber, the trojan virus can record the audio signals when someone phones a retailer and taps in their card details. To help disguise the theft, these details are then encoded using rapid changes to the device’s vibration, volume and screen settings. A second trojan, connected to the victim’s phone via the internet, decodes the signal and transmits the details to a hacker. “With any communication channel there is potential for eavesdropping,” says Kapadia.
Worse, today’s security software simply isn’t designed to spot illicit sound recording or transmission, warn Michael Hanspach and Michael Goetz from the Fraunhofer Institute for High Frequency Physics and Radar Techniques in Wachtberg, Germany. Last year the researchers built an acoustic communication system that linked computers into a covert network (). They argue that this kind of inaudible sound transmission sidesteps conventional security systems.
Without proper protection, might it even be possible for your cellphone to become infected by a virus hidden in music or a voice call? This seems unlikely, since software capable of decoding such incoming malware would have to be on the device already, so why bother to install more? “It’s feasible to send any kind of data in the background, and have a system at the receiving end turn it into something meaningful,” says Graham Cluley, a UK-based computer-security consultant. “However, you would need to be running dodgy software already. What would be the point of delivering another payload via sound?”
“Could your cellphone be infected by a virus hidden in music or a voice call?”
Indeed, the computers used by Hanspach and Goetz were loaded with software capable of receiving audio signals prior to creating their network. Nor did Ruiu ever claim his computers were initially infected via their microphones, just that they could communicate via audio signals once they were already infected.
Ear this, or not
In principle, it isn’t too hard to protect devices. Fitting a phone with software that filters out inaudible signals can reduce the risks, say Hanspach and Goetz, though this would also stop the device working with some new audio technologies. Meanwhile, researchers at Pennsylvania State University in University Park have developed that vets apps before you install them and alerts you if the app requests to access particularly risky combinations of device features – both the microphone and internet access, say. At the moment, new devices do not have security systems with this kind of protection.
Then there is privacy to worry about. Facebook, for example, recently announced a new feature on its app that to record a 15-second audio clip. It then matches this against a library of TV shows and music tracks, so it can tell your friends what you are listening to. However, more than half a million people have objected online, worried it will record or store personal information. Audio’s limited range is also perfect for firing information at individuals within buildings, making the technology a golden opportunity for location-based marketing businesses hoping to hit you with ads. Last year Sonic Notify’s beacons were installed in 34 shopping malls in the US, where they deliver personalised promotions and advertising to consumers via a smartphone app.
This may not be popular: according to a recent US survey, 93 per cent of those questioned did not want their location to be harvested for tailored advertising. However, Sonic Notify argues that limits can be put on how much content is delivered to users. It’s not in the interests of companies to bombard users with information as they’ll quickly delete the app or turn it off, says Alex Bell, the founder of Sonic Notify. “Users choose to have a store’s app to start with.”
Whether or not audio advertising becomes a pain in the ear, it sounds like sonic technology is here to stay. Developed thoughtfully, it could bring a whole world of possibility, Bergel says. We have the potential to create new sonic languages for machine-machine and human-machine interaction, he says, something that can give people the ability to do novel things. “Birds sing to communicate, so why not machines?” he says. “But it could also be abused. It’s up to us to determine which it is.”
Listen to the sonic techology of yesteryear: ““
Work the crowd
“One of the great things about audio is that it can be attached to existing media like radio, video or live music events,” says Alex Bell. “If you have speakers, you don’t need separate beacons.” Bell’s company, Sonic Notify, has provided musicians with apps that use sonic signals to transform an audience into part of the show, by taking control of their cellphone screens to create a coordinated light display (see main story).
Meanwhile, researchers at Disney Research and the Swiss Federal Institute of Technology in Zurich have looked at ways that inaudible signals could enhance the movie experience. Thomas Gross and his colleagues have developed an app called on the audience’s smartphones.
To improve data transfer rates and reduce transmission errors, the team’s software links up the microphones to form a distributed array so that the audience’s phones collaborate to download and share messages. The researchers suggest this could be used at any indoor movie or outdoor festival to transmit games or questionnaires, offer vouchers, or even create new approaches to interactive storytelling.
This article appeared in print under the headline “Sonic boom”
