People could soon let their ears do the talking when using a virtual assistant thanks to an ear-reading device. When we speak or mouth words, our facial muscles move and our ear canals change shape. The new earphone technology detects those changes, allowing people to issue silent speech commands.
鈥淥ne key problem for using voice assistants right now is that every time [we use them], we need to say some activation words, so this will feel very awkward in the public scenarios,鈥� says at the University at Buffalo in New York. 鈥淵ou don鈥檛 want to sit in public and say, 鈥榟ey Siri, go ahead and do something.鈥欌��
Jin and his colleagues created the hands-free 鈥淓arCommand鈥� system, which could provide privacy for people interacting with their virtual assistants in public, while also avoiding voice command interference stemming from background noise or muffled speech from wearing masks.
Advertisement
EarCommand works by emitting near-ultrasound signals into the wearer鈥檚 ear and detecting the reflected echoes through an inward-facing speaker and microphone. An artificially intelligent algorithm analyses the patterns in those reflected sound waves and learns to associate warping of the ear canal鈥檚 shape with particular words such as the names of popular smartphone apps like 鈥淭ikTok鈥� and 鈥淪napchat鈥� and commands such as 鈥渨hat鈥檚 the weather鈥� and 鈥渃all mom鈥�.
The technology could also prove more convenient than other silent speech interfaces that require users to make certain gestures, hold their phone in a stable position or wear extra sensors on their faces or bodies.
The system currently recognizes 32 single-word commands and 25 sentence-length commands. It makes mistakes about 10 per cent of the time when recognising commands composed of single words and 12 per cent of the time when interpreting sentences.
The researchers hope to gather more speech samples to train the algorithm so they can reduce the word error rate to about 5 per cent, in line with the rate typically sought by commercial voice recognition systems. They also hope to expand the vocabulary recognised by the system.
The team also envisions the system鈥檚 accuracy improving as the AI adapts to individual users. That may require patience from consumers when devices don鈥檛 have the best accuracy straight out of the box, says at the University of Washington in Seattle who was not involved with the project. Still, he says the EarCommand concept is promising.
鈥淰oice [command] comes at the cost of disruption to others in the physical space and at a loss of privacy,鈥� says Dey. 鈥淎t a high level, this concept of detecting silent speech is really valuable as it addresses both of these challenges.鈥�
Journal reference: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies,
