żěè¶ĚĘÓƵ

Google Glass app uses emojis to help children with autism read faces

Many children with autism find it hard to decipher other people’s facial expressions. An interactive system that uses Google Glass may help
child wearing Google Glass
Julian Brown took part in the trial
Jeff Chiu/AP/REX/Shutterstock

Many children diagnosed with autism find it hard to decipher other people’s facial expressions. An interactive system that uses Google Glass may help them out.

“I can see the difference this will have in people’s lives in a significant, immediate and meaningful way,” says Donji Cullenbine, whose child took part in the study. She said enrolling on the study was “one of the best choices that I have ever made for him”.

Using technology such as Google Glass to support individuals with autism has been a promising area of research. The idea is they can provide a game-like environment to practise life skills without being overwhelming. But it has never been used outside the lab before.

Dennis Wall at the Stanford University School of Medicine and his colleagues have now demonstrated that the effects work outside of the lab. They gave 14 children with autism a system called to try at home for an average of 10 weeks each.

Superpower Glass is a Google Glass headset combined with an Android app. The system detects expressions in real-time and then displays emojis that match the facial expression in the corner of the Google Glass, as well as speaking via the headphones. The idea is that the children find it easier to spot the emotion when it is displayed in this way.

To find out if the setup had any effect, the team asked the children’s parents to assess their children before, during and after wearing the glasses using the Social Responsiveness Scale, which is often used in autism research.

The children’s scores on this questionnaire decreased by 7.38 points during the study, indicating less severe symptoms. None of the participants’ scores increased. Six of the 14 participants had large enough declines in their scores to move down one step in the severity of their autism classification.

The results should be interpreted with caution since the study didn’t have a control arm, says Wall. However, the findings are promising, he says.

His team is now completing a larger, randomised trial of the therapy. In addition, it also plans to test the system in children who have only recently been diagnosed.

Gaze avoidance

Cullenbine and her son Alex had previously used regular sessions of applied behavioural therapy with a therapist over many years, but Alex’s gaze avoidance remained a significant issue for him. This was associated with anxiety, especially with other children, who can be unpredictable.

Cullenbine noticed a difference in Alex’s eye contact abilities within just two weeks of using Superpower Glass. What she finds remarkable is that even after the study, when he was no longer using it regularly, Alex’s eye contact still became commonplace. “Previously, it would be two or three times a day. Then I noticed I wasn’t counting the times he looked at me anymore,” she says. She says this is still true almost two years after participating in the trial.

Access to early interventions is crucial, says at Durham University, UK. “I would also hope to see this intervention compared to other types of social skill and emotion recognition training packages so that we can see whether the benefits are specific to this intervention,” she says.

While understanding emotion recognition in people with and without autism has long fascinated scientists, this topic has limited relevance to the everyday concerns of autistic people, says at the University of Edinburgh, UK.

“While I am unconvinced by emotion training programmes, I do think that creating an opportunity for parents to talk with their children about what feelings are and how they are expressed has value,” she says. “However, this could be achieved with much less expensive and complex resources.”

npj Digital Medicine

Topics: Autism