żěè¶ĚĘÓƵ

Artificial touch: The new tech making virtual reality more immersive

The feelings of touch and temperature are complex biological processes. Now everyday chemicals like menthol and capsaicin are being used to simulate them – and create more realistic VR experiences

YOU open a door and it hits you – a flare of warmth on your skin. You brace yourself to go inside, battling smoke and heat. Flames flicker around you as you make your way through a burning building. You find what you came for and escape. Outside, it is so cold you start to shiver, while your hands and feet go numb.

But then you remove your headset and it all stops. You just finished an incredibly realistic training exercise. None of those sensations were caused by changes in your surroundings, although they felt real. Instead, chemicals carefully selected to mimic different feelings were pumped onto your skin.

Such stimulants have long been useful for understanding touch, the most complex of all human senses. In the 1990s, studies of capsaicin, an extract of chilli peppers, and menthol, found in peppermint, helped us pin down how our bodies react to hot and cold conditions. Now, are using this knowledge to create chemically induced sensations, to make virtual environments astonishingly realistic.

In a technology dubbed chemical haptics, they have built a wearable device that, when placed on the skin, can cause the wearer to experience a range of sensations – hot or cold, numb or tingly – on demand. Its uses could include creating intensely realistic virtual worlds for gamers to explore or for training firefighters. But will we ever be able to fully replicate the experience of touching something real, and what might we lose if we can’t? Amid growing talk about metaverses, such questions are increasingly important. “How we sense the world around us is critical for pretty much everything in life,” says Thomas Perlmann, a biologist at the Karolinska Institute in Stockholm, Sweden.

The word haptics officially means anything related to the sense of touch. Today, it is mostly used as a shorthand for haptic technology, the devices we use in daily life that help replicate a touch feeling using force, vibrations or motion. On your phone, your home button may not be a physical button at all, but . Next time your device is switched off, see if you can still press it.

But the applications for haptics go beyond phone buttons. Haptic devices have been used to help and provide realistic feedback to . In 2019, researchers in Hong Kong used tiny motors to create a virtual skin to “hug” relatives across the world.

The limit with these devices, though, is they only make use of one type of touch – pressure. Our skin can feel so much more than that. “Our sense of touch is mediated by various receptors in our skin,” says Lu. “We have thermoreceptors for sensing hot and cold, mechanoreceptors for sensing vibration, pressure and force, and nociceptors for sensing pain.” Why not try to make the most of all of them? This way, says Lu, you could put people into simulations of dangerous situations, like burning buildings, to train them in what to expect without any physical danger.

The idea for chemical haptics began not with touch, but with different senses altogether. Lu’s colleague at Chicago, , was interested in the way chemicals add to the richness of daily life, particularly through smell and taste. Brooks designed a headset that released chemicals like mint and pepper into the nose’s trigeminal nerve, a large tract of neural fibres that carries pain, touch and temperature information to the brain. When people wore the headset in a virtual reality environment, pumping menthol into their nose made them feel cooler, and . Not only were the users experiencing a smell they associated with a cold feeling, like mint, but the trigeminal nerve was also telling their brains the room was a different temperature.

When Lu joined the lab, she wondered whether there might be a way to bypass the nose and go directly to the skin. “I realised there were a lot of other chemicals that can achieve different sensations on the skin,” she says. “Then our team began looking at all other chemically induced sensations that have been studied, expanding beyond just hot and cold.” She started reading about work that had been done decades earlier.

A library of touch

In the early 1990s, , San Francisco, wanted to find an alternative type of painkiller to the opioids that were starting to become widespread in the US. But first, he realised, he had to learn more about . He and his team wanted to understand the signalling pathways that underpin our sense of touch.

As a starting point, they created a library containing millions of DNA fragments, each corresponding to genes expressed in the neurons linked with pain, heat and touch. Julius knew capsaicin made skin feel hot and burning. They spent years trying out thousands of fragments to see which ones caused receptors in our cells to respond to capsaicin. After a long search, in 1997, they finally identified a protein called TRPV1. Receptors for this protein are found in cell membranes, mostly in nociceptive neurons in our skin. These are nerve cells responsible for feeling a certain kind of pain. TRPV1 alerts the brain to both physical and chemical stimuli, such as burning capsaicin and higher temperatures – anything above 43°C. In response to these triggers, TRPV1 opens an ion channel that sends electrical signals to the brain.

Julius and others went on to spend decades examining the intricate web of receptors, other neurons, proteins and ion channels that together give us the complex and varying sense of touch we experience. , California, was one of the key researchers, helping to discover the way we feel cold sensations and Under pressure. This work was vital in getting us closer to a complete understanding of the sense of touch.

Devices worn on the skin can create more realistic virtual worlds
Human Computer Integration Lab/UChicago

When Lu read about Julius and Patapoutian’s work, it made her reconsider the way her lab was using stimuli. “Before, I had just imagined doing it externally – creating heat, providing force feedback, generating vibrations,” she says. “But now, I was thinking of how to more directly interact with the specific [cellular] channels that regulate the perception of these sensations.”

In 2021, Julius and Patapoutian were awarded the Nobel prize in medicine for their work. On the same day, Lu and her team published their . “It was actually quite a coincidence,” she says. “Their work on detailing the receptors that correspond to these sensations of hot, cold and pain is foundational to our approach.”

In their paper, Lu and her team used chemicals ranging from sanshool, a component of spicy Sichuan pepper, to create a tingling feeling, to capsaicin, to mimic warmth. Menthol was used for cold, while a local anaesthetic, lidocaine, numbed the skin. Each was pumped through a wearable device to the skin.

The the . In a video of a virtual reality scenario, someone wearing the chemical haptic system is seen escaping from a nuclear power plant on the brink of meltdown. With sparks flying, sanshool is pumped into channels on the arm and face to create a feeling of tingling, as if they were hitting the skin. When the person tries to unlock a door using an arm-worn interface, it fails, and lidocaine numbs the area, giving the impression they have lost the use of the limb. As the door to the reactor opens and heat rises, capsaicin flows onto the skin to simulate the warmth coming from a fire, and when they exit the power plant and enter a snowy scene, menthol is released onto their cheeks to mimic the feel of a cold wind.

“How we sense the world is critical for pretty much everything in life”

While this is the cutting edge, the chemical pathways involved in touch have been exploited in various ways before. Medical creams use concentrated levels of minty wintergreen oil to create a thermal reaction on the skin that helps relieve pain. Some skincare products use capsaicin to promote blood circulation, and mouthwashes use menthol to generate a fresh sensation. But such chemicals have never before been used in conjunction with virtual worlds.

Not everyone is excited about this development. “Honestly, it scares me,” says , a neuroscientist at Liverpool John Moores University in the UK. He has spent decades studying (see “Emotional touch”) and believes virtual reality will never be able to replicate this. “We know that this digital world is going to take over,” he says, “but we need to find ways to ameliorate the negative consequences of not having physical contact.”

A new haptic system pumps chemicals onto the skin to provide virtual sensations
Human Computer Integration Lab/UChicago

“Touch is a matter of life and death,” says . Compared with other mammals, humans can do very little when we are born. “Our very survival in those early days relies on caregiving from our parents and all of this is mediated by touch,” she says. It also lies at the heart of our social development. “Touch has a power unlike that of the other senses,” she says.

Lu is quick to stress that her work with chemical haptics is not, and in her view never will be, a replacement for real touch. “Touch is a really complex sense, which is why using touch to interact with our world in real life is such a wonderful experience,” she says. Instead, she sees it enhancing digital experiences. “I don’t tend to think of VR as a vehicle for escaping our reality, but as a medium that can empower us to do and experience things we can’t normally.”

And while chemical haptics can replicate some sensations, others elude it. “I can’t simulate the softness or texture of my cat’s fur when I pet her,” says Lu. This is why she and her team see the most important applications of VR being experiences and sensations that augment rather than substitute our day-to-day lives.

Chemical haptics was made possible thanks to decades of research using natural stimulants. Next, Lu wants to create new molecules, to see how they react. If all goes well, it could even lead to more discoveries about our most complex sense. “This is the first exploration of generating skin sensations using an interactive device that dispenses chemicals,” she says. “We don’t know how far this can go.”

EMOTIONAL TOUCH

Touch a burning hot iron and you will immediately know to pull your hand away. This sense, called discriminative touch, is communicated to the brain using nerves known as A fibres, which provide almost instantaneous information. But there’s another group of nerve fibres, called C fibres, that act more slowly, taking around a second to carry a signal from your foot, say, to your brain. These communicate different types of pain, such as throbs and aches, rather than stings or burns.

In the late 1990s, Åke Vallbo at the University of Gothenburg, Sweden, discovered a specific type of C fibre called C-tactile or . “It’s a lovely nerve,” says Francis McGlone at Liverpool John Moores University in the UK. “It responds to exactly the velocity of stroking you would say is nice.” The CT fibres only responded to slow, gentle touch – 5 centimetres per second – and they are only found on hairy skin. The type of touch that triggers the CT nerve is called affective touch, because it is used to create social bonds.

Not getting enough of this kind of touch has been linked to depression and anxiety. McGlone and his team published a study in which they stroked rats every day for 10 minutes. One group was stroked at the speed the CT fibres respond to, while the animals in another group were stroked six times faster. Then, they were put through situations that would provoke mild stress. . Whether this works in people is yet to be seen, but McGlone says initial results are promising.

UNDER PRESSURE

After finding a protein that helps our bodies sense cold (see main story), Ardem Patapoutian at the Scripps Research Institute in California didn’t stop there. To study how we feel pressure, Patapoutian and his colleague Bertrand Coste designed a new experiment. After spending the best part of a year and a half switching off sensory genes one by one, Coste identified two genes that control our body’s sensitivity to pressure. The pair called these PIEZO1 and PIEZO2, from the Greek word for “pressure”.

They found that when they are silenced, it renders tissues incapable of feeling force and reduces their ability to feel pain. The genes code for proteins that form ion channels that open in response to mechanical pressure on the skin and internal membranes. They help our bodies detect blood pressure, bladder pressure and breathing. But they play a much bigger role too.

Proprioception is the sense of self-movement and bodily location, sometimes likened to our body’s GPS. People without PIEZO2 in the nerve cells that supply muscles and tendons tend to lack coordination and can end up in a wheelchair. PIEZO1 and PIEZO2 channels regulate important physiological processes, including blood pressure, respiration and bladder control, while PIEZO2 plays a role in pain-sensing neurons. Researchers are now thinking about targeting force-sensing proteins with medicines to treat, for example, chronic pain.

Topics: Biotechnology / Technology