A bit less virtual, a bit more reality (Image: Pedro Lopes)
MY OPPONENT bobs and weaves in front of me, fists cocked, ready to attack. Gingerly, I settle into my own stance and prepare to fight. He closes in straight away and throws a few jabs, testing my guard. My forearm jerks back as his fist connects. Feeling my arm physically move is strange, because this boxing match is happening in virtual reality.
I’m experiencing this mash-up of real and virtual in the Hasso Plattner Institute, southwest of Berlin, Germany. This lab, run by , is where the future of our interaction with computers is playing out. I felt the virtual punch because my arm is wired to an electrical circuit that fires in sync with the virtual world. The electricity stimulates the muscle into involuntary contraction, simulating the force of a landing punch.
Advertisement
is investigating this phenomenon, known as electrical muscle stimulation (EMS), as a channel by which computers and brains could communicate. By talking directly to the nervous system through our muscles, Lopes and other researchers like him can guide a person’s body movements, providing a novel way to interact with technology.
Electricity is already being used to control the body in medical applications. For example, pacemakers help to regulate the heartbeat by delivering timed electrical pulses. Lopes and Max Pfeiffer, who is working on EMS at the University of Hannover in Germany want to bring this mode of interaction to consumer technology, although without the implants for now, and on less crucial muscles than the heart.
Lopes keeps electrodes on his forearm all day while he’s working in the lab. Wires dangle loose as he explains his work. He has made EMS a part of his life. He and a student kept forgetting to water the plants, so they wrote code that would jolt their arms to perform a watering motion periodically, as a reminder. It worked for a while, but now the plants sit brown and dry on Pedro’s windowsill. Outside, a robot lawnmower with a plastic sheep attached to the top whirs by.
“They wrote code to jolt their arms to perform a plant-watering motion, as a reminder”
A diagram of the muscles in the human hand hangs above Lopes’s head. His most sophisticated system, called , can control a number of these using electricity, guiding hands to operate a drill or play an instrument, for instance. Another, called Pose IO, tweaks arm muscles to change the position of the limb and relay information.
Pfeiffer thinks of it as a supporting technology that simplifies the way we interact with our devices so we waste less time. He is using electrical stimulation to guide a person’s legs along a predefined path. “For instance, I hate when I’m running around and need to look at my map. My purpose is to go there, help me do that,” he says.
EMS’s first application as a computer conduit is likely to be in virtual reality, helping to heighten the immersion of a simulation by moving the body in conjunction with other stimuli. Making me feel like I got punched is a good start.
“Someone hitting you turns out to be really hard to simulate,” Lopes says. The sensation is split into multiple parts. First, a small plastic arm just barely touches me, then the electrode provides a little jolt of electricity – “but put the two together and your brain gets a little bit tricked. It feels like, in VR, that this boxer was hitting you,” says Lopes.
Uncanny valley
But EMS faces a substantial obstacle: it feels really weird to have a computer take control of your muscles. At one point, Lopes wires up my forearms and sits me down in front of a can of spray paint. He asks me to pick it up, then taps a few commands into his computer. My hand gets the unmistakeable instruction to shake the can, and my muscles obey.
It is possible to resist the electric commands. I hold my hands stiff as Lopes shocks me again, and am able to keep dead steady. “I could go to a level where that fight gets more complicated, where you would have to push even more,” he says. “The opposing muscle would have to be really strong.”
This willing handover of control might prove too much for EMS to really take off as a mainstream technology, while the electrical sensation takes a while to get used to. But tweaks such as custom electrodes could soften the “zappy” feeling, says Pfeiffer, which might help reduce the weirdness.
Zappy feeling
Using implants would give finer control over the EMS sensations. Lopes is already talking to a group from the Max Planck Institute in Germany who are making implants out of thin metal hairs. These reach through the skin into different layers of muscle in the arm and can fire at different depths, offering high-precision control of specific muscles. The implants are designed to help those with motor neuron disease, but Lopes says the Max Planck group is casting around for other applications too.
He views these kinds of implants as inevitable, even if they don’t become mainstream for many years. If and when they do, the fine-grained muscle control they offer could let those with the skills for a specific task, say soldering, take control of the arms of those who don’t know how.
Pfeiffer sees applications in sports, with electrodes guiding athletes’ muscles during training, allowing them to focus on the higher-level mental challenges of a certain move or play. “If you’re in training and the coach wants to train a specific running pattern” they would simply draw it on a tablet and have their players guided through the move, he says.
“The canonical way of learning a new skill is YouTube tutorials,” says Lopes. “My vision was that instead of watching a video, a tutorial would play EMS, designed by someone to teach you.”
Electric threads
Stimulating our muscles with electricity could provide a novel interface between us and our computers. However, getting electricity into muscles non-invasively poses a challenge (see main story). Help may be at hand.
łŇ´Ç´Ç˛µ±ô±đ’s aims to weave electronics into the fabric of normal clothes, working with clothing brands to embed circuitry early in the manufacturing process. “LEDs, haptics and other embedded outputs provide feedback to the user, seamlessly connecting them to the digital world,” explains łŇ´Ç´Ç˛µ±ô±đ’s page on the project.
Future interfaces may even stick to our skin. John Rogers’s lab at the University of Illinois has developed thin, stretchable electronic films that can be worn for days at a time and communicate with other devices wirelessly. The films are light enough to stick to the skin with Van der Waals force and could deliver electricity to muscles, with batteries and microprocessors built in.
This article appeared in print under the headline “Body-hack with a zap”
