WHEN Teresa Marrin Nakra slips into her Lycra jacket and begins to conduct, a
stream of note-perfect orchestral music pours forth. But if she hands the
sensor-stuffed garment to an untrained friend, the music slurs, as if the whole
orchestra was suddenly drunk.
There are no musicians in Nakra鈥檚 laboratory at Immersion Music in Haverhill,
Massachusetts鈥攐nly a cyber-orchestra, made up of groups of instrumental
sounds inside a synthesiser. But just like the real thing, the cyber-orchestra
only plays well if it鈥檚 conducted properly, with the conductor鈥檚 right arm
signalling volume and the left arm beating time.
鈥淪he plays the system just like an instrument,鈥� says Gary Hill, a teacher of
orchestral conducting at Arizona State University. Hill discovered Nakra鈥檚
invention when he was looking for a better way to teach the discipline. Like all
professional conductors, Hill鈥檚 students begin their careers conducting silently
in front of a mirror. The only music they produce is in their head.
Advertisement
Nakra developed her conductor鈥檚 jacket while doing research at the Media Lab
at MIT. It contains sensors over the arms, back and torso that pick up
electrical activity from muscles and detect the wearer鈥檚 heartbeat. It is
connected to a PC though wires emerging from the tail of the jacket.
To calibrate the system, Nakra used the jacket to analyse hundreds of hours
of action by six conductors. The six included Benjamin Zander of the Boston
Philharmonic, Keith Lockhart of the Boston Pops Orchestra, and Nakra herself.
Software running on the PC correlated data from each conductor鈥檚 muscle and
heart activity with the volume, tempo and emotional responses each one used for
particular musical passages. The clingy Lycra material of the jacket holds the
sensors close to the skin, and the conductors felt comfortable wearing it, Nakra
says. 鈥淭he point was to sense movements without changing them.鈥�
When worn by a student the jacket and its PC are linked to a music
synthesiser that automatically replays various pieces. As the student conducts
the piece, the computer interprets their movements to give the music emotional
highs and lows.
In Hill鈥檚 classes, students conduct a series of study pieces designed to test
particular skills, like increasing and decreasing the volume, or beating in a
particular dynamical style. The computer responses enable them to associate the
right muscle memories with musical sounds, and penalises them for tensing in
ways that professionals usually avoid. 鈥淚f they generate in a weird way, they
get a weird sound,鈥� says Nakra.
Early indications are that the system appears to be halving the time students
take to progress from novice to intermediate conductor, from six months to about
three. Hill says it is especially good for teaching students to relax their
upper body. Barry Kraus, one of Hill鈥檚 students, agrees. 鈥淵ou learn to make sure
you relax enough so your gestures stand out.鈥�