DRESSED in a shiny leotard, Carla Pulvermacher first bounds, then creeps onto centre stage in slinky, feline fashion. So begins her dance to a rhythm of hums, twangs and airy swishes that respond to her every move. Seconds later the mood has changed, and the sounds seem to be pursuing her, as she glances left and right in response to notes from an encroaching harp. Soon the faint plucks grow into a crescendo of loud, frenetic plinks and clicks and she writhes wildly, trying to keep up with the ever-increasing rhythm.
If the dance seems eerily connected to the dancer鈥檚 movements, it鈥檚 because the soundtrack is no ordinary score. She is dancing to the music of her own brainwaves. 鈥淵ou don鈥檛 know what鈥檚 going to be thrown at you in the next 10 seconds,鈥� says Pulvermacher, a student at the Conservatory of Dance and Performing Arts in Frankfurt, Germany.
She may not know what鈥檚 coming next, but Thilo Hinterberger, the conductor of Braindance, has some idea. He is no stranger to reading people鈥檚 minds. For much of the past decade Hinterberger, a physicist at the University of T眉bingen in Germany, has been developing ways of reading the brainwaves of people with severe disabilities 鈥� those who have poor eyesight and cannot talk or even move a muscle 鈥� to allow them to communicate using their thoughts alone.
Advertisement
Hinterberger straps electroencephalograph (EEG) electrodes to their heads to read the jumble of electric signals produced by their brain activity. The signals are then filtered and processed by a computer to isolate low-frequency brainwaves. Hinterberger has developed software that converts these brainwaves into a visual representation such as a movable cursor on a computer screen, so that patients can use this visual feedback to learn to control their brain waves and move the cursor. A few have learned to surf the web in this way.
However, some patients with failing eyesight found they could no longer focus on the screen. To overcome the problem, Hinterberger is modifying the system so that it instead produces simple sounds that can be used by patients to communicate. This latest work has renewed a long-standing personal interest for Hinterberger. 鈥淔or many years I鈥檝e wanted to create music using brainwaves,鈥� he says.
Early in 2004 he met Ottmar Gendera, a journalist who interviewed him about his research. The pair discussed Hinterberger鈥檚 private passion and came up with the idea of reading a dancer鈥檚 brainwaves to create music for the dancer to perform to. Gendera put Hinterberger in touch with Pulvermacher, and in October 2004 the two gave their first public performance of Braindance. Since then, Pulvermacher and Hinterberger have performed three more times in public and have practised their peculiar performance dozens more.
鈥淚t takes just a few hundred milliseconds to convert her brainwaves into sound鈥�
The music is created in a simple feedback loop. As Pulvermacher dances, her brainwaves are recorded by scalp electrodes connected to a portable EEG monitor strapped to her hip, which beams the signals to a computer. Hinterberger sits at the computer and chooses in real time which waveforms from Pulvermacher鈥檚 brain will trigger which instruments, mixing them to change the mood of the music as he sees fit.
The raw EEG signals the software receives are a jumble of superimposed brainwaves of different frequencies. So Hinterberger鈥檚 program filters these signals into the six distinct frequency bands used by neurologists to classify the different types of waves produced in the human brain. Brainwaves are usually a mixture of all these frequencies, but different wavelengths dominate at different times, depending on what the person doing (see 鈥淲hich wave?鈥�).
Hinterberger can select different aspects of each wave to trigger a sound. For example, for higher-frequency waves such as theta waves, wave peaks above a threshold chosen by Hinterberger produce individual notes on a simulated instrument or pre-recorded sound stored in the computer鈥檚 memory (see 鈥淢usic of the mind鈥�). Low-frequency waves might be made to control the pitch and volume of a continuous sound whose pitch rises and falls with the wave form.
Hinterberger can choose from 128 simulated instruments and sounds, including harp, piano, bongo drums, hums and hisses, though when he uses the high-frequency waves to trigger these sounds, the instrument or source of the sound is often unrecognisable. Crucially, it takes just a few hundred milliseconds for Pulvermacher鈥檚 brainwaves to be converted into sound, 鈥渟o the dancer can really get into her own rhythms鈥�, says Hinterberger.
At first it was a wild and unpredictable performance, but slowly the pair are learning how they can have some control over the direction the of the performance and even choreograph short bits of it. 鈥淚鈥檓 beginning to be able to predict some aspects of the music,鈥� Hinterberger says. 鈥淏ut unexpected things still happen because EEG signals are kind of spontaneous. There鈥檚 always some sort of surprise,鈥� he says.
Pulvermacher is also still learning how to dance to her own rhythms. 鈥淚 try to listen to what the music sounds like it will do next and think about how my brain will respond to the movements.鈥� She says she can exert a small degree of control. 鈥淚 can speed up the tempo of the music and make it louder, but calming it is more difficult.鈥� That鈥檚 no surprise. The brain is a constant hive of electrical activity and Pulvermacher would have to calm her mind to a near sleep state to soften the rhythm.
But Pulvermacher has experienced startling moments of familiarity. 鈥淥ne day during rehearsal we were talking, and Thilo told me to listen to the music. There was this rhythm, bum-bum-bum, bum-bum-bum. It was a waltz,鈥� she says. Nevertheless, she says she does not practise controlling her brainwaves. 鈥淭hat is not Braindance鈥榮 objective. It is supposed to be more spontaneous.鈥� Controlling an instrument with brainwaves could be a real possibility in the future, though. 鈥淚n principle it could be possible,鈥� Hinterberger says.
But we won鈥檛 be watching a jazz pianist jam with his saxophone-playing brainwaves anytime soon, says Jonathan Berger, a composer at the Center for Computer Research in Music and Acoustics at Stanford University in California. 鈥淚t鈥檚 all about control,鈥� says Berger. 鈥淚f you鈥檙e talking about broad improvisatory gestures, we can do it now. If you鈥檙e talking about the finesse that a violinist can evoke playing a violin, then we鈥檝e got a long way to go.鈥�
