People who find it difficult to speak due to a stroke or Parkinson鈥檚 disease could communicate more easily with the help of artificial intelligence. A new model constructs what a person is trying to say based on tiny vibrations in their throat, but also takes into account other factors, such as what time it is and the emotions they may be experiencing.
Some neurological conditions can result in dysarthria, where people lose fine control over their voice box, jaw or tongue. Previous solutions using brain-computer interfaces have yielded promising results, but users needed invasive surgery to place electrodes on or in their brains.
Now, a group of researchers 鈥� including academics from the University of Cambridge, University College London and Beihang University in Beijing 鈥� have used textile strain sensors to measure the movement of throat muscles, via vibrations, as well as the pulse in the carotid arteries, to shed light on whether a user鈥檚 emotional state is neutral, relieved or frustrated.
Advertisement
That data is then fed into two large language models, each based on GPT-4o-mini, the model behind some instances of ChatGPT. The first, known as the token synthesis agent (TSA), aims to tease out the intended words mouthed by the user and group them into sentences.
The second, the sentence expansion agent, takes sentences from the TSA and uses contextual information like the time and weather, as well as the user鈥檚 emotional data, to expand the sentences into what the researchers describe as 鈥渓ogically coherent, personalised expressions that better capture the patient鈥檚 true intent鈥�, compared with when the sentences are created without contextual and emotional clues.
The researchers declined to speak to 快猫短视频 but claim in their paper that in tests with five people with dysarthria as a result of a stroke, their system achieved sentence error rates as low as 2.9 per cent. They also found that using emotional and contextual clues to add to sentences increased user satisfaction over straightforward reconstruction of sentences by 55 per cent.
at the University of Birmingham, UK, says the technology could be hugely positive, but that interpretation of intended communication comes with risks. 鈥淚t could be a bit frustrating for people if that language model is saying things in a way that they wouldn鈥檛,鈥� he says. 鈥淚f you had someone who was massively articulate before and used lots of long words, if the large language model is using lots of small words, people might find it a bit frustrating 鈥� but that鈥檚 probably much less frustrating than not being able to communicate at all.鈥�
Beale says that mannerisms, accents and patterns of speech could conceivably be customised to the model to create a more familiar voice for the user, but even now, the idea shows promise. 鈥淸Users] don鈥檛 have to think about doing anything differently,鈥� says Beale. 鈥淭hey just do what they did before, and magic happens. And that鈥檚 what good [social] interaction should be like.鈥�
arXiv
