EVERYONE knows there are five basic senses. But try separating them one from the other in your daily life and suddenly they don鈥檛 feel so distinct.
Eat a banana, for instance, and try to taste it without smelling it and experiencing that banana-y texture on your tongue. Can you really just taste, or must you sometimes taste-smell-feel? Try talking to your lover. Listen to what is said without watching the mouth move or feeling the caress of a hand. Can you simply hear, or is there always an element of hear-see-touch? Even on the phone, can you hear a voice without imagining a face? Hard, isn鈥檛 it?
The prevailing view of the brain still holds that there are five separate senses that feed into five distinct brain regions preordained to handle one and only one sense. The yellowness of the banana skin, the texture of its flesh, its smell and taste-each of these elements is parcelled up and analysed in isolation. Some theories of consciousness suggest that these dedicated brain areas somehow stamp each sense with a unique 鈥渇eeling鈥. Then, the theory goes, the brain pastes the fragments back together, calls on memory to give it a name and recall what it鈥檚 for and, voila, a banana.
Advertisement
But perhaps it鈥檚 time for a radical rethink of how the brain works. Tasks we鈥檝e long assumed were handled by only one sense turn out to be the domain of two or three. And when we are deprived of a sense, the brain responds-in a matter of days or even hours-by reallocating unused capacity and turning the remaining senses to more imaginative use. All this begs the questions: are the senses really so segregated? Are they separate at all? Indeed, is it possible that our senses are continuously developing and merging so that each one of us has our own private view of the world?
It might be a big shift in thinking, but it began with a simple finding-the discovery of 鈥渕ultisensory鈥 neurons. These are brain cells that react to many senses all at once instead of just to one. No one knows how many of these neurons there are-maybe they are just a rare, elite corps. But perhaps there are no true vision, hearing or touch areas dedicated to a single sense, after all. Perhaps all the neurons in our brains are multisensory-and we mistakenly label them 鈥渧isual鈥 or 鈥渁uditory鈥 simply because they prefer one sense over the others.
That鈥檚 the view of Alvaro Pascual-Leone at Harvard University. He made a splash five years ago when he showed that people who were born blind use the visual cortex when they read Braille. He wondered if rather than lie idle, parts of the brain meant for seeing just started helping out with touching. His more recent work has convinced him that not only blind people but everyone has the capacity to swap senses if they have to. He thinks that the brain is much more versatile than most researchers would have us believe.
To test the idea, Pascual-Leone blindfolded healthy, sighted volunteers for five days running, taught them Braille and watched how their brains responded. He even fitted their blindfolds with photographic paper-just to be sure volunteers weren鈥檛 tampering with them. Before, during and after the blindfolding, they had a series of brain scans while they were set different tactile and auditory tasks-feeling either Braille characters or brush strokes on their fingertips and listening to tones or word fragments. Before the blindfolding began, the 鈥渧isual鈥 areas were not switched on by the touching and hearing tasks. But as the week wore on the visual regions became more and more involved in routine touching and hearing.
If a person isn鈥檛 seeing, Pascual-Leone found, parts of the 鈥渧isual鈥 cortex are roped in to help out in tasks involving other senses. In fact, the newly recruited regions soon become indispensable. When he tried temporarily disrupting the workings of the visual areas, using a technique called trans-cranial magnetic stimulation, or TMS, the blindfolded volunteers found it hard to read their Braille.
Taking the blindfolds off for just a day, though, was enough to undo the changes; suddenly touching and hearing tasks no longer triggered visual areas, even though volunteers were blindfolded again briefly for the scan. 鈥淩emoving the blindfold and being exposed to the seeing world for 12 to 24 hours is sufficient to revert all changes induced by the five days of blindfolding,鈥 says Pascual-Leone.
What was astonishing was how quickly the brain seemed able to recruit new areas and equally effortlessly reverse that process. It was far too quick to be the result of new connections forming from scratch reasoned Pascual-Leone. 鈥淚t must be assumed,鈥 he says, 鈥渢hat tactile and auditory input into the 鈥榲isual cortex鈥 is present in all of us and can be unmasked if behaviourally desirable.鈥
Pascual-Leone now feels the brain is not organised into 鈥渧isual鈥 and 鈥渁uditory鈥 and 鈥渢actile鈥 regions at all. Instead he thinks it is split into units that have specific jobs to do or particular problems to solve-calculating distance, for example, or timing intervals. These problem-solving units simply use the best information available. Sometimes they may prefer certain senses to others, based on how suitable they are for the assigned computation, and sometimes they may use more than one, if that helps. Vision, for instance, might be the preferred way to judge distances. But if you can鈥檛 see, hearing or touch can certainly fill in.
The preference of a particular problem-solving unit for a specific sense may explain the notion of sense-specific regions, he says. Just because an area tends to call on vision doesn鈥檛 mean it can鈥檛 process other senses, only that it may not bother if its first choice sense is on hand. This may have tricked neuroscientists into thinking that the brain is structured in parallel, segregated systems processing different types of sensory signals, says Pascual-Leone.
There is some good evidence that the brain can mix up the senses to solve particular problems. One of the main benefits of sensory integration may be better clarity and detection, says Barry Stein, at Wake Forest University in Winston-Salem, North Carolina, one of the first researchers to identify the brain鈥檚 multisensory capabilities. Even weak signals should be taken seriously if they鈥檙e picked up by more than one sense.
We are, for example, much more sensitive to a chemical when we combine smell and taste. Pamela Dalton, at the Monell Chemical Senses Center in Philadelphia, asked 10 people to smell benzaldehyde, a cherry-almond odour that has no taste, and to taste saccharin, a sweetener that has no smell. Before each testing session, she worked out the point where each volunteer could no longer detect each substance and prepared even weaker samples. Then she asked them to slosh the solution around in their mouths and sniff the odour at the same time. Combining taste and smell made both substances much more apparent, she found. 鈥淭en minutes before, they hadn鈥檛 been able to detect it,鈥 says Dalton.
A brain combining senses can also make better sense of ambiguous information. David Lewkowicz at the New York State Institute for Basic Research in Developmental Disabilities on Staten Island shows this nicely with a visual image of two balls moving from opposite sides of a screen, merging briefly in the centre, then continuing along their merry ways (see 鈥淏rain Games鈥). But when a beep sounds at the moment the two balls merge, what you see changes completely. Now, instead of passing through each other and continuing along the same trajectory, the two balls bounce off each other and return to the side they came from.
Combining hearing with vision can lead us to draw different conclusions about what we鈥檝e seen too. A single flash of light, can appear to be two flashes when it coincides with two beeps, says Ladan Shams and her colleagues at Caltech in Pasadena. Even when we know there is just one flash, we can鈥檛 help perceiving it as two. Apparently the brain won鈥檛 let us draw contradictory conclusions from two different senses.
Increasingly, scientists are discovering that even everyday activities may actually make use of more than one sense. Consider the task of running your fingers over a pattern of raised ridges and deciding in what direction they are running. What sense do you call upon? Most of us would guess the obvious: touch. But a group at Emory University in Atlanta has demonstrated that in perfectly normal people parts of the 鈥渧isual鈥 brain are also essential for perceiving touch.
They started by scanning people鈥檚 brains to see what regions were activated when they were trying to decide the orientation of some grating patterns on a touch pad. They found that a part of the brain that鈥檚 involved in recognising objects by sight was active while people felt the gratings, even though they couldn鈥檛 see them. 鈥淲hat excited us was what our subjects told us,鈥 says Krish Sathian, a lead member of the team. 鈥淲hen they were doing the tactile task, they were actually visualising in their mind鈥檚 eye the orientation of the grating.鈥
Did visual imagery just provide a convenient aid, or was it essential to the task? To find out, they used the TMS technique to disrupt the activity in the 鈥渧isual鈥 region the volunteers had been using. Suddenly, their volunteers could no longer tell the direction of the pattern.
The researchers concluded last year in the journal Nature (vol 401, p 587) that the 鈥渧isual鈥 cortex is closely involved in certain tactile tasks. They claimed it was the first time that visual processing was shown to be instrumental in ordinary tactile perception. But Sathian admits that the activated region may not really be visual at all. It could be a part of the brain that helps us visualise what鈥檚 being touched. 鈥淲e certainly can鈥檛 rule out that what we鈥檙e seeing is multimodal processing in an area previously thought to be just visual,鈥 he says.
Pascual-Leone鈥檚 bold interpretation, that the brain is organised by task rather than by individual sense, is by no means the accepted one. Even most scientists who study multisensory processing consider it extreme. 鈥淎t least some areas are exclusively unisensory,鈥 says Sathian. There鈥檚 very clearly a primary visual cortex with strong inputs from the eye, he says, and a primary somatosensory cortex getting information from the body. But that鈥檚 not to say that the map of the brain is static-far from it. New multisensory areas are being found all the time. 鈥淭he boundaries are being pushed back,鈥 says Sathian, 鈥渏ust not pushed back all the way.鈥
Those boundaries were seriously tested by an experiment that involved 鈥渞ewiring鈥 the brains of ferrets. The findings called into question the well-guarded notion that certain brain areas can only dedicate themselves to certain tasks. They suggest that, although the brain may tend to develop in a particular way, with vision processed at the back of the head and hearing on the sides, it doesn鈥檛 have to be that way.
A group at MIT in Boston wanted to know how much they could override innate developmental pathways. 鈥淚f we put the retina into the auditory cortex, will it see?鈥 asks Sarah Pallas, a member of the team, now at Georgia State University in Atlanta. The researchers surgically rearranged one brain hemisphere in a handful of newborn ferrets, so that the nerves from the retina, which normally go to the visual thalamus and then on to the visual cortex, now connected to the auditory thalamus and eventually to the auditory cortex.
To their surprise, they found that the auditory cortex on the rewired side arranged itself like a visual cortex: the cells showed selectivity for orientation and motion, and they encoded a two-dimensional map of visual space. The rewired animals also seemed to behave perfectly normally. Using only the untouched hemisphere the researchers trained the animals to go to a food spout on one side of a test room if they heard a sound and one on the other if they saw a light. Amazingly, even after the visual cortex on the healthy side was completely destroyed, the animals found their way to the food.
鈥淲e were able to turn the auditory cortex into a visual cortex,鈥 says Pallas. 鈥淢aybe they couldn鈥檛 recognise their grandmother with that, but they certainly could detect light.鈥 In fact, the young ferrets seemed so normal that the researchers had to mark them to tell them apart from their siblings.
The experiment revealed just how multimodal the brain may be. The amazing rewired auditory cortex was not only seeing-it was hearing at the same time, Pallas told a meeting of multisensory scientists in New York last autumn. Though the finding has not yet been published, she said that preliminary testing showed that the rewired auditory cortex was responding well to sound.
What鈥檚 more, the study shows that what goes into the brain can have a lot of influence on how it鈥檚 ultimately organised. Although some parts of the brain may be predisposed to become one thing or another, the rewiring shows they aren鈥檛 predetermined. 鈥淪ensory inputs can influence the regional identity of the cortex,鈥 says Pallas.
But how far does this go? We can fairly assume that people deprived of sight early on will have their brains wired up differently from people who see. But what about someone who has been nearsighted since birth-could that person have a quite a different brain from someone who鈥檚 experienced the world through sharper eyes? Is someone born into the high rises of Hong Kong wired up differently from a person growing up in the Gobi desert?
Pascual-Leone thinks that, both at the functional and the anatomical level, our brains are quite unique. 鈥淏lind people are not experiencing the world like a sighted person with eyes closed,鈥 he says, 鈥渂ut rather, they have a dramatically different world representation and hence consciousness.鈥 Indeed, maybe each of us has our own very personal take on the world, sensed by our own unique brain.
Alas, we only know how it feels to be ourselves, so it鈥檚 impossible to know. And we can鈥檛 ask those ferrets whether they were really seeing, or somehow hearing the light. It makes you wonder all over again about bananas-is the divine yellow fruit the very same to you as it is to me? Probably not.

Brain games
- Balls seem to bounce when a sound is added http://neuro.caltech.edu/scheier/BouncingIllusion/BouncingIllusion.html
- Can your brain detect a single flash if two beeps sound? http://neuro.caltech.edu/~lshams/demo.html
- Find out what you perceive when you hear a voice say: 鈥淢y bab pop me poo brive鈥 and you see a mouth say: 鈥淢y gag kot me koo grive鈥 http://mambo.ucsc.edu/~course/dad.mov
- The McGurk effect: http://www.media.uio.no/personer/arntm/McGurk_english.html