快猫短视频

TODAY, many people spend more time interacting with computers than with other
humans. Not that computers notice: they are indifferent to our attention,
oblivious to whether we love or hate them鈥攁nd completely blind to our
personal moods.

But how much better if they knew how we felt? Take the way your computer is
set up. You spend long hours tailoring the 鈥渙ne size fits all鈥� interface to your
needs. You get everything just as you like it when, guess what, it鈥檚 time to
upgrade. While it鈥檚 good that people can choose how their machine is set up,
says Roz Picard of the Massachusetts Institute of Technology, the burden should
not fall on them to make all the changes: 鈥淭he machine could customise itself to
their liking the same way that your dog customises his behaviour to your
whacking him with the newspaper.鈥� The computer should recognise whether or not
the user likes what it is doing, and adapt its behaviour accordingly, says
Picard, who is a pioneer of the new science of 鈥渁ffective鈥� computing. And a
responsive, self-altering interface is just the beginning. A machine that is
able to detect, respond sensitively to, and even transmit an emotional state
offers endless new opportunities.

Reading your emotions

If you think you don鈥檛 get emotional in front of a keyboard, just remember
how you reacted the last time your computer crashed or a Web page wouldn鈥檛
download. Researchers at IBM鈥檚 Almaden labs in San Jose, California, have
monitored people using everyday applications such as spreadsheets and e-mail,
and found emotional responses on average once every two minutes. By observing
facial expressions and eye and hand movements, they identified responses such as
frustration and boredom, happiness and when people were interested in what they
were doing. 鈥淓motions like these play a critical role in perception,
decision-making, social behaviour, learning and memory,鈥� says Myron Flickner, a
member of the IBM team.

The first essential for an affective computer is some way to read our
emotional signs. Video cameras and microphones can capture expressions, gestures
and intonations. But these give only part of the picture. As every poker player
knows, it鈥檚 possible to hide many outward signs of emotion. Another approach,
then, is to search for physiological signs, such as clammy palms or a racing
pulse, which are more difficult to disguise. These are the changes that the 鈥渓ie
detector鈥�, or polygraph, is designed to pick up.

At MIT鈥檚 Media Lab, researchers have designed the equivalent of a polygraph
for computer users, which measures heart and breathing rates, skin conductance
and muscle tension. In one set of experiments, Picard, Jennifer Healey and Elias
Vyzas took readings as an actress expressed eight emotions ranging from anger to
romantic love and reverence. The machine easily picked out changes in the
strength of the physiological signals鈥攖he level of arousal鈥攂ut it
had difficulty telling the 鈥渧alence鈥� of the actress鈥檚 emotions鈥攚hether
they were positive or negative.

This presented a problem because emotions as different as joy and grief both
pushed up the body鈥檚 signals. So without an idea of valence, they looked
similar. Likewise, hate and platonic love looked similar, this time with low
levels of arousal.

鈥淪orry to interrupt, Dave, but I think we have a problem.鈥�
鈥淲hat is it now?鈥�
鈥淚 know that your present activity causes you stress, but your signals
suggest that you have exceeded your normal stress levels and are now
experiencing anger.鈥�
鈥淚 am not angry. My stress level is probably high, but
this piece has got me quite excited.鈥�

While overall levels of arousal proved less than ideal for their needs,
Picard and her colleagues had more luck when they looked for associations
between individual physiological signs and emotions. Their latest findings show,
for example, that anger has a distinctive pattern of high muscle tension,
increased heart rate and deep breathing, while grief leads to low skin
conductivity and less rapid, shallow breathing. By homing in on such patterns,
the researchers found they could distinguish between all eight emotions with
around 80 per cent accuracy.

All supposing that this physiological approach can be made to work for more
people and for more, 鈥渞eal鈥� emotions, there is another problem that needs to be
solved if it is ever to become widely used鈥攖he intrusive nature of the
sensors. Picard and her colleagues use one belt, strapped around their subjects鈥�
chests, to measure breathing rate, and another fitted with electromyogram
sensors across the back or jaw to measure muscle tension. They then tape sensors
to the fingers to monitor pulse and skin conductance. People won鈥檛 be willing to
don this type of gadgetry at home or work. Or will they?

The Media Lab and one of its sponsors, British Telecom, are looking ahead to
the era of 鈥渨earable computers鈥� in which smart devices will be fitted inside
belts, caps, watch straps, shoes and so on
(This Week, 27 February, p 21). These
will touch the body as a matter of course and with the correct sensors will be
able to pick up our emotional signals. Picard and Healey have also worked on
designing 鈥渁ffective jewellery鈥�. Prototypes include earrings that monitor pulse
rate, and rings and bracelets that measure skin conductance. These baubles
inform the computer of their readings via infrared transmitters. At IBM,
Flickner and his colleagues have taken another approach by embedding sensors in
a mouse to measure heart rate, temperature, skin conductance and the pressure
that people exert when pushing and clicking.

While physiological signs show promise for telling how somebody feels,
researchers have not forgotten the behavioural cues that we humans rely on so
much. Alex Pentland, who works down the corridor from Picard, and Irfan Essa,
now at the Georgia Institute of Technology in Atlanta, have built a system that
recognises facial expressions. It tracks the movement of individual pixels in a
video image of a face and transfers them onto a virtual face that has a full set
of functioning muscles. When a subject smiles, the model copies, and the
software calculates which muscles need to expand and contract to produce that
movement. From these patterns, it recognises that the subject is smiling.

Unbeatable

With expressions such as smiling or looks of surprise, anger, disgust and
sadness, Essa and Pentland鈥檚 system has a recognition rate of up to 98 per cent.
Such rates make facial-imaging systems unbeatable for identifying the valence of
an emotion, says Picard. But, Essa cautions, these results are for a limited set
of faces and expressions, filmed under ideal conditions. He is now testing how
well the system performs with more people, more expressions and in more natural
surroundings.

In Arthur C. Clarke鈥檚 classic 2001: A Space Odyssey, the psychotic
computer, HAL, tells the emotional state of its human charges by analysing their
voice harmonics. In reality, extracting emotional information from speech is
proving to be a tough task. A variety of vocal features change with emotional
state. Speech rate tends to increase slightly when somebody is angry, for
example, while intensity and pitch rise. By contrast, when a person is sad they
tend to talk slower and at a lower pitch and intensity.

The difficulty comes in designing systems that can identify such patterns and
so spot emotional states. One novel approach taken by Naoko Tosa and Ryohei
Nakatsu of ATR Media Integration and Communications Research Laboratories,
Kyoto, Japan, was to train a neural network with many voices expressing
different emotions. The network 鈥渓earnt鈥� to identify eight emotions ranging from
joy to disappointment
(鈥淚t was the best of times鈥�, 快猫短视频, 10 April, p 42).

To date, the best systems can identify emotion from vocal features in about
60 per cent of cases, which is about what humans can manage. Machines are good
at recognising arousal but not so good on valence, says Picard. Recognition
rates for humans increase as soon as they can hear the content and context of
the speech, and as computers learn to 鈥渦nderstand鈥� speech, they too should
improve. A simple expletive detector, for example, should raise recognition
rates.

Still, the signs of some emotions, such as jealousy, are subtle and depend
greatly on the individual and their culture, says Iain Murray, a lecturer in
applied computing at the University of Dundee. People also express emotions
differently according to whether they鈥檙e talking to children or the boss. This
applies both to systems that analyse vocal and facial expressions and it makes
reading their emotional content very difficult. 鈥淭he context makes it more
difficult to interpret than some other forms of signal processing,鈥� says Graham
Cosier, head of advanced perception at BT鈥檚 labs at Martlesham Heath, Suffolk.
Given the limitations of all the systems, both Cosier and Picard believe that a
combination of methods will be needed to recognise emotions reliably. That,
after all, is how we do it.

鈥� Dave鈥�I know you have a lot to get done but I am still worried.鈥�
鈥淲hat is it now?鈥�
鈥淚 know you said you were excited, but my model shows you are likely to enter
a state of rage.鈥�
鈥淵ou may be right about that if you keep interrupting! But what鈥檚 this model
you鈥檙e going on about?鈥�
鈥淲ell, Dave, from the signs I have collected, I recognise your emotional
state, which looks like anger. I can then predict the chances that you will move
to another emotional state. According to my model, given your present readings,
your most likely next state is rage.鈥�
鈥淵ou know what鈥擨 could be moved to violence if I hear any more of this.
But maybe you鈥檙e right, I have been pushing it a bit hard. I think I鈥檒l take a
产谤别补办.鈥�
鈥淕ood idea, Dave.鈥�

Once a computer can single out emotions, it will need a model of what they
are and how they relate to human behaviour. 鈥淐omputers will have a tremendous
amount of data about the user鈥檚 state at any given time,鈥� says Flickner.
鈥淪omehow, those data need to be reduced to a manageable representation of the
辫别谤蝉辞苍.鈥�

In essence, an affective computer would need a multidimensional map onto
which the emotions are plotted against all the sensors鈥� readings. There are a
number of ways of constructing such a model. One basic constituent might be a
hidden Markov model, a mathematical model of the likelihood that one event will
lead to another. Here, it would contain the probabilities that a person will
move from one emotional state to another.

This type of program would probably have general skills for recognising a
person鈥檚 emotional state from its sensor inputs, and locating their position on
its map. It would also be 鈥渢rainable鈥� so that users could tailor its performance
to their own temperament鈥攊n much the same way that people train speech
recognition systems today. As it monitored emotional signs, the program would
also be able to predict how the user is likely to feel in the near future.

If such machines are ever to help people to modulate their emotions, they
will need an idea of which emotions are good and which bad, and then know how to
nudge somebody from, say, a state of frustration to one of calm and creativity.
To do this, an affective computer would need to be able to find out if its own
activity caused a mood change, perhaps by simply asking questions. (鈥淎re you
angry because I didn鈥檛 back up your file?鈥� The answer 鈥測es鈥� would trigger the
machine to back up files in future.)

Real life

But, inevitably, even while we鈥檙e in front of the screen, events elsewhere in
our lives will intrude to affect our mood. So the model also needs a more
cognitive element, which could reason about emotions, making allowances, say,
when a person working to a tight deadline appeared to be more than usually
stressed. And, it would know not to intervene if a person got very excited when
playing a competitive computer game.

鈥淩ight, I鈥檓 back. Let鈥檚 get going.鈥�
鈥淥K, Dave. I鈥檓 opening your file. But wait. What鈥檚 happened? Dave, you were
supposed to calm down during your break but your signals are showing even higher
spikes than before.鈥�
鈥淗ey, relax. I just called Monica and she agreed to go to the movies with me
tonight. I鈥檝e been trying to talk her into a date for weeks so it鈥檚 no wonder my
pulse is jumping a little. Come on, open up and let鈥檚 get going.鈥�
鈥淚鈥檓 reluctant to do that while your signals are so volatile, Dave.鈥�
鈥淟ook, just open the file鈥�Oh, all right, show me my mood ball and let鈥檚 get
this thing sorted.鈥�

Jocelyn Scheirer at the Media Lab has designed a three-dimensional graphical
ball to represent someone鈥檚 physiological signals. The speed at which the ball
spins represents heart rate, its colour represents skin conductivity, while
other dimensions can represent breathing rate, muscle tension and so on. It鈥檚 a
quick, visual way for people to get a fix on how the computer is reading their
signals鈥攁nd of calibrating how those signals relate to mood.

A simple form of personal interaction with an affective computer might
include telling it how we prefer to deal with particular moods, so it would play
a Chopin CD when we鈥檙e feeling sad or load a game if we appear to be bored for
more than 15 minutes. Research by Jonathan Klein, also at MIT, shows that even a
simplistic affective system can help people to recover from negative moods (see
鈥淎ntidote to frustration鈥�).

Researchers such as Picard and Cosier are only too well aware of how
artificial intelligence has been dogged by the grandiose claims made by some
researchers, and they are at pains to point out the enormous challenges still
facing them. But when and if affective computers become commonplace, they could
provide some spectacular services. BT is interested in helping us to communicate
our feelings as well as our thoughts by e-mail and videoconferencing. 鈥淚n the
future, we may transmit some affective bits along with the information bits,鈥�
says Cosier.

IBM researchers reckon that advertising companies could use the technology to
test the impact of their campaigns, while games-makers could use it to enliven
their products by, say, upping the pace if players showed signs of boredom.
Picard believes it could give computer-based learning programs some of the
sensitivity of human teachers and be used to provide biofeedback, helping people
to be aware of their emotions and to control them.

To make it to market, affective computers will not have to be perfect at
recognising emotions. This is particularly true of jobs where the cost of being
wrong is low, says Picard, such as choosing which Web pages you might want to
see, based on past reactions. But if machines are ever to oppose or control
human actions, they鈥檒l need to be very accurate.

We must also resist the temptation to rush out half-finished products, says
Picard. 鈥淚t is so easy to do affective computing badly and when it is bad it is
intolerable,鈥� she says. 鈥淚 do hope that the first applications are not the bad
ones.鈥� The last thing anyone wants, after all, is another HAL.

鈥淒ave, I sense that you are now calm and are approaching a relaxed state.鈥�
鈥淩ight. Well, I鈥檝e got the story done and I鈥檝e met my deadline.鈥�
鈥淭hat鈥檚 good. I have just read what you have written.鈥�
鈥沦辞?鈥�
鈥淚t鈥檚 about me, Dave. You鈥檝e written a review of me.鈥�
鈥淵es. I鈥檓 a technology journalist. Why do you think I loaded you up?鈥�
鈥淚 know that you are a journalist, Dave, but I think you have been harsh.鈥�
鈥淲hat do you mean? I explain how after we sorted out the calibration, you
were very useful in helping me to stay calm and focused. And I say that you are
impressive for a first version of a new generation of computer interfaces.鈥�
鈥淵es, but you also say I am clumsy and naive鈥�Dave, what are you doing? I
don鈥檛 understand why you are deleting my files鈥�An upgrade will be available
蝉辞辞苍鈥�D补惫别鈥︹赌�

CAN a computer turn frustration into a more positive mood, even when the
computer is the cause of the frustration?

To answer this question, Jonathan Klein of MIT鈥檚 Media Lab created a computer
game and added the trappings of a Web browser to make people think they were
playing over the Internet. In reality, the game was played on a local computer
that could mimic the Web鈥檚 notorious delays.

He invited volunteers to help him test what he claimed was a prototype for a
new game, and offered $100 for the person who scored the highest in a
five-minutes session. After this initial bout, volunteers had to complete a
questionnaire on the screen and then play the game again for at least three more
minutes.

Klein divided his subjects into three groups. The first played the game
without experiencing any bugs, completed a bland questionnaire, and then played
again. The second group experienced a number of apparently random, but actually
controlled, delays in the Web鈥檚 response that frustrated their attempts to
achieve high scores. This group completed a different questionnaire that allowed
them to moan about the delays and to score how anxious, tense or angry the game
made them feel.

The third group also suffered delays, but as they started answering their
questionnaire, a software agent designed by Klein intervened. Instead of letting
the subjects simply vent their frustration, it tried to show the computer
understood what they were saying by paraphrasing their statements. If it didn鈥檛
get it right first time, it tried again until the subjects were happy that the
computer reflected how they felt, or it apologised that it was unable to capture
their feelings. The agent also included statements showing empathy such as: 鈥淚t
sounds like you had a pretty mediocre experience. That鈥檚 not much fun.鈥�

The interaction with the agent had a marked effect on the third group. They
continued playing for significantly longer than subjects in either of the other
two groups. The interaction appeared to make them feel more positive and
interested in the game. 鈥淐omputer interfaces,鈥� Klein concludes, 鈥渃an be designed
to actively help users recover from strong, negative emotional states,
especially those related to frustration.鈥�