WHEN Sarah Chesters-Thompson paid a surprise visit to her college’s end-of-year art show this summer, she found half a dozen children pushing and fighting over her exhibit – and not an irate curator in sight. Chesters-Thompson, who had just graduated from London’s Royal College of Art, was pleased: she’d designed a device that lets children make music out of coloured patterns, and the fact that the kids were willing to go to war over it told her she was doing something right.
Chesters-Thompson’s device is simple enough – a child paints a picture with six colours, then rubs a fluffy, squishy device shaped like a half-football over the painting. Buried in this fluffy mass are sensors that detect the colours by the different shades of grey they “see”, and others that sense when they’re squeezed. As the football moves around, a computer translates the signals into motley notes or noises. The child, in effect, “plays” her painting.
Now swap London for San Francisco. You only have to watch the children swarming around Japanese artist Toshio Iwai’s computer exhibit at the city’s Exploratorium museum, to sense the allure that such new tools can have. Jostling the controls, a young boy slaps different coloured paints on a “brush”, then covers the computer screen with patterns and squiggles. As he paints, tiny “insects” march hither and thither across the screen, triggering music as they cross the child’s artwork – a different note for each colour, a different instrument for each insect. Children aren’t the only ones to be enchanted by Iwai’s invention: “insects” was also a smash hit with a group of Tibetan monks, who visited the museum not long ago to carve a traditional butter sculpture.
Advertisement
For all kinds of reasons, artists, musicians, composers and scientists are devising ingenious devices that people can use to make music in unconventional ways. Some turn colours and shapes into sounds; others track the movement of the body, the tug of a muscle. Some even tap into the electrical buzzing of our brains to set off strings of notes. You could even say plants are making music, depending on your definition of music.
In part, people are doing it because computer technology now allows them to. Today, a musician wearing a Dataglove in Innsbruck can play drums in Salzburg, while a dancer in Salzburg plays a violin in Innsbruck. People can wander through virtual worlds inside computers, “touching” the objects they encounter, triggering tunes as they go. Or, if they prefer, they can do the opposite: use music to change the patterns, colours and shapes that they see through their virtual reality goggles.
Quality of life
But there is more to be had here than fun. Many of the inventors hope these new tools will improve our lives, that they will rekindle our involvement in music by allowing people to cut down the years of mind-numbing drills designed to teach “technique”. They also might offer us new entertainments that are more creative than the majority of CD-ROMS and videos around today. People with disabilities, who may lack the dexterity to master a piano or violin, might gain a musical voice from their body movements. Dancers might finally control the music that accompanies their performances. And musicians and composers can add extra qualities to traditional instruments, or reinject their own expression into electronic performances – expression that electronic music, for all its wonders, has whittled down.
In fact, any one of us stands to gain from these strange new instruments, since few of us get to make music these days, and many of us would probably like to – witness the phenomenal success of karaoke bars. “Here’s a forum where people basically get up and make fools of themselves singing in public,” says Tod Machover, professor of music and media at the Massachusetts Institute of Technology’s Media Lab. “Who would have thought that they would want to go out and do that?” Exhibitionism, for sure, is part of the reason, but so, too, is that untapped musical urge.
It wasn’t always untapped. “For centuries, people would sing together, whether in religious services or around the fire – and even past that, most middle-class people pretty well into the 19th century probably learnt the piano,” says Machover. “The way they’d experience music would be to get together on a Saturday evening and play through popular songs, or four-hand versions of Beethoven pieces. It was all very active.”
Today, though, we can hear fine music at the push of a button. We have radio, TV, CD players, videos and computer games – a plethora of distractions that make years of practising scales or scratching away at the violin seem less and less worth the payoff. Nowadays, says Machover, the average Westerner gets to hear more music than ever, yet isn’t much engaged in making it.
No return
Since there’s not a chance that we’ll return to the old days, Machover thinks the true answer lies with new instruments – ones that don’t have such a daunting mountain to be scaled before aesthetically pleasing sounds emerge. At the same time, such instruments can’t be so facile and easily fathomed that we toss them aside two weeks after Christmas like the latest gimmicky kid’s toy.
Getting the balance right is deceptively difficult, says Machover, who is certainly no slouch when it comes to building innovative instruments. He has developed a number of them with MIT colleague Neil Gershenfeld. The Hypercello (Netropolis, èƵ supplement 16 April 1994) – a kind of electronic cello that allows subtle movements of the wrist and bow to trigger complex new sounds – has been performed by the cellist Yo-Yo Ma, to rave reviews. For non-virtuosos, meanwhile, the Media Lab feels that the perfect, user-friendly instrument might use weak electric fields to track the movement of the body through space.
There’s some history to this approach. Early this century, the Russian Leon Theremin invented an instrument (the “theremin”) that was played by moving one’s hands in the space around two metal antennas: different notes would sound as the performer changed hand positions. Concerts for theremin and orchestra were even performed in New York’s Carnegie Hall, while at the other end of the artistic spectrum, theremins provided the perfect, spooky music for many a horror film.
Decades on, the Media Lab’s inventions are rather different. Take, for instance, the Gesture Cube, which looks like a giant cube standing on one of its points. As you draw near it, you notice that it is murmuring softly, and then – as soon as your body gets close enough – the noises begin to change. You swing your right arm forward, and new notes are triggered; you move it back and then forward again, and the sounds change once more. Waving your left arm in the air, you notice that the quality of the notes is shifting – growing richer or more hollow, perhaps, lasting for a longer or shorter time depending upon whether your arm is higher or lower in space.
Then picture the Sensor Chair – a much more elaborate device which Media Lab researcher, Joe Paradiso, built for the popular North American magicians Penn and Teller, who have been on the road with it since 1994. Sitting in this chair, the skilled performer can choose one of 400 percussion sounds by touching a specific place in the space around him. Or he can play any melody or sound at any volume by carefully choreographed wiggles of the hands and feet. The Artist Formerly Known As Prince thought the Sensor Chair magical enough to try to get it for his own performances. Instead, Machover’s team built him two other instruments, the Gesture Frame and a shop mannequin that can be played when one moves near to it – and both will no doubt show up in some future concert.
Both the Sensor Chair and the Gesture Cube use similar, but not identical, ways of mapping the body’s movements in space. The Sensor Chair, for instance, has a flat, copper electrode on the seat. When a person sits down, they become an extension of that electrode, and a tiny low voltage, low radio frequency current flows through the body. In front of the chair are four receivers, positioned at four corners of a large rectangle. If the sitter moves a hand close enough to these receivers, the signal injected into the body is picked up by the receivers. The closer he moves his hand, the more current he delivers; the farther away his hand, the lower the current until it drops off altogether. (Similar receivers lower down pick up movement of the feet.)
Hand signals
Since the hand will be always nearer some of the receivers than others, it transmits different amounts of signal to each of them, thereby enabling its position to be mapped in three dimensions. Machover’s software then reads the input values from the four receivers, calculates the hand’s position, and triggers instructions to a synthesiser via the universal electronic music control system, (MIDI). What happens next – the clash of a cymbal, the roll of a drum – depends on how the computer has been programmed to respond, when a particular coordinate in space has been “touched”.
In years to come, might happy families gather round Gesture Cubes or something like them in the evening, and play away to their hearts’ content? Machover doesn’t think he’s yet invented that perfect instrument for the people, but he is talking with the Japanese musical instrument giant Yamaha about turning some of his inventions into commercial products. And next summer, at New York’s Lincoln Center, the general public will have a chance to use a new generation of Machover’s hyperinstruments in a giant project called the Brain Opera.
Among many other oddities, visitors will get the chance to play a Rhythm Tree, a sort of circular sculpture. The player stands in the centre, surrounded by a range of suspended objects which when tapped send signals ricocheting to neighbouring objects, producing curious echoes. Or, if they prefer, participants can form musical ensembles and play the Gesture Wall, where every movement made by a group of up to five people is converted into sound.
And anyone who can’t make it to the Lincoln Center, or to any of the other gigs on the Brain Opera’s world tour, can always make a contribution via the Internet. Just log-in to the designated site on the World Wide Web then take your pick: send in your own sounds, play Internet versions of the instruments or change the on-line score, which will be playing continuously. Everyone – from the thousands of people expected to wander through the exhibit to the person sitting at their home computer – can help shape and transform Machover’s composition, becoming very real collaborators in a musical creation.
If instruments with brand new interfaces could give us all a musical voice, they might also give back to professional musicians what some feel they’ve lost when they play electronic instruments: the power to express themselves in performances, via artful shifts in tempo, dynamics, orchestral mix and more. Take the example of a singer. Performing with an orchestra or pianist, he or she can really shine, calling the shots by slowing down, speeding up or pausing for breath, adding richness and expression to the performance.
Marking time
It’s quite another kettle of fish with electronic accompaniment (something many singers opt for – synthesisers do all sorts of interesting things, and flesh-and-blood musicians demand wages, don’t forget). “The singer is girdled into the exact tempo of the synthesiser – and if you stray it’s obvious you’ve made a mistake,” says Maureen Chowning, a singer based in Palo Alto, California. “Sometimes, singers even resort to using stopwatches.” The soloist, in effect, has been demoted to accompanist.
Chowning doesn’t have that headache any more: she is one of the lucky few to own a very special conductor’s baton that lets her keep her own, electronic orchestra in line. With a wave of her hand, she can alter the tempo and volume of her electronic accompaniment – whatever she deems artistically right. “It’s very liberating – I’m now in control of what I do,” she says.
Richard Boulanger, a composer and professor at the Berklee College of Music in Boston, also owns a baton, which he’s used in everything from more traditional classical performances with symphony orchestras and synthesiser, to New Agey pieces that trigger sounds and lights as he waves his hands through space.
“It’s fantastic to use – it’s fantastic to have this power,” he says. “The baton allows me to take to the stage with a computer in a virtuosic way – lean into a note, add more vibrato, more shades, lengthen a line, or detune a note a little bit, so that it has a more yearning quality to it – things that singers do all the time. It allows me to play as a virtuoso, to play with some of the greatest clarinetists and sopranos in the world.”
The Radio Baton, as it’s called, was developed by Max Mathews of Stanford University, who wrote the first music for a computer around 40 years ago. There are actually two batons, each of them acting as a radio antenna transmitting a different radio frequency, and each attached by wires to a transmitter that sits inside a large flat instrument, affectionately nicknamed a “pizza box”. Ten receivers – five tuned to each baton – are also inside the pizza box. As the performer waves both batons around in the space above the box, the five receivers pick up the signal, and – as with the Sensor Chair – the computer inside the box calculates the position of the baton from the strength of signal received by each receiver. It then uses that information to amend the electronic score, which is also loaded in the computer, and finally, the synthesiser plays the amended music.
Of course, there is no “finally” until the last note of music is played, because the conductor is continually waving his batons around, and the computer is constantly number-crunching and changing the next bit of score. Just what those changes are, meanwhile, depends on how each composer decides to program the software. Mathews, for example, generally has the right-hand baton beat time, and the left baton control the loudness and mix of different instruments. As he stands before the pizza box, his right baton swings downwards then up again, and the music plays along in rhythm. His left baton dances and twists through the space above the box, bringing out the higher or lower-pitched instruments as it travels left or right, and changing the volume as it moves from back to front. If all goes as planned, the radio baton will be on sale by Christmas.
Body music
Then again, perhaps you’d like your whole body to be your instrument – in which case, you may be tempted by the Very Nervous System, developed by David Rokeby, an interactive artist living in Toronto. Point a video camera at a dancer and VNS will compare each frame with the one before, converting the shifting shapes into sounds.
Dancers and other artists everywhere are using the VNS: they are sometimes amazed by what it can do for their performances, says Derrick de Kerchove, director of the University of Toronto’s Marshall McLuhan Program in Culture and Technology, which explores ways that artists are embracing new technology. One group of dancers, he remembers, were singularly unimpressed when first shown the system. “They were pretty haughty – what is this silly thing?, you know,” he says. “But then they got into it – and we had to kick them out at 11 pm. They’d been dancing the whole day.”
Meanwhile, the Yacov Sharir Dance Company, which is based at the University of Texas in Austin, is excited by another invention, a touch-sensitive floor invented by Russell Pinkston, professor of music at the university. Forcesensing resistors set off notes, melodies, lines of verse, you name it, when triggered by the dancer’s well-aimed leaps and rolls on the floor (This Week, 3 December 1994). At last, dancers don’t need to worry about keeping absolute time with a melody – now, they can make the music happen.
And today, with some of these new tools, people who’ve been shut out from making music by their physical inability to grasp a bow, or, to press a piano key, might also see doors swing open. Boulanger will soon be working with a disabled drummer. The man is too weak to crash out a roof-rattling drum solo, but he hopes that something like Mathews’ radio baton can keep him performing.
With the help of VNS, meanwhile, two severely disabled people, one moving her fingers and the other holding a drumstick in his mouth, performed in a band in front of a Vancouver audience. To the south, in Seattle, a child with cerebral palsy was rigged up with a device called Thing One by its co-inventor, David Warner of Syracuse University in New York state. As she smiled, electrodes on her cheek sensed the firing of her muscles and triggered musical notes.
But these days, you needn’t even move a whisker to make some kind of music. Two companies – Interactive Brainwave Visual Analyzer Technologies in New York, and Palo Alto’s BioSystems Control – have developed devices (IBVA and the BioMuse) that delve right into our brains. Electrodes placed on the user’s head measure brainwaves, and pass the signals on to a computer. This tells a synthesiser to trigger a note, depending on which frequency of brain wave is most abundant at the time. Think serene thoughts and your alpha waves go up; focus on a task and beta waves predominate. As the brain waves shift, so does the music.
The resolution isn’t so great – how well can any of us control the precise frequencies of brain waves we fire? But crude or not, artists are having a ball experimenting. Sylvia Pengilly, a performance artist based in New Orleans, has used the IBVA system in multimedia shows. And another artist even rigged up IBVA electrodes to a plant, recorded its electrical “song”, then played the resulting melody to a canary, which tweeted its response. If nothing else, IBVA’s manufacturers say that the plant-IBVA combination could be turned into a fine burglar alarm, since plants will change their electrical activity when people enter rooms.
A “green” burglar alarm? Maybe. Fun? Most definitely. But whether all these instruments make “music” depends on individual tastes and how one defines music. The composer John Cage held that any sound could be considered music – it was just a matter of sloughing off all those ingrained prejudices. Others think that the rules for what is “musical” are hard-wired in our brains to some degree.
Whether the results will be musical depends, also, on which instrument one is talking about – there’s a lot more sophistication and musical control in a Sensor Chair than in a brain wave analyser, for example. But will it continue to depend upon who is playing, if an instrument’s interface is so user-friendly that anyone can tackle it? Could we all become virtuosos? “There’s a lot of loose talk about how ‘art is dead’, now that anyone can go into an interactive art installation and wave their hands around,” comments Rokeby, who isn’t much impressed by such talk.
Nor is Machover. “I don’t believe in this idea that you build these tools and everybody’s going to become Beethoven,” he says. “There are always going to be people who take these tools and do something really great with them – something that everybody else, given the same tools and the same opportunities, couldn’t do in a million years.”
But what might change, they say, is the time and effort sucked up into physically mastering the idiosyncrasies of an instrument: the demand for unfaltering dedication from an early age, the gruelling decades of arpeggios, scales and exercises – all that concentration on technique rather than musical messages and meaning. If tomorrow’s strange instruments can bring all of us closer to music, they might also help tomorrow’s great artists get to the meat of what is musical a little sooner.