快猫短视频

IN CASE you hadn鈥檛 noticed, neural computers have come of age. Over the past
few years they have begun dispensing hard-learnt wisdom on everything from
medical conditions to financial investments. A simple neural network is even
likely to take command of NASA鈥檚 space shuttle as it docks with the
International Space Station.

The forte of these machines is learning to identify patterns鈥攚hether
they come from pixels in a video image or signals from a robot鈥檚
sensors鈥攁nd changing their outputs depending on what they detect. So
neural nets become face recognition systems or robot controllers. These
abilities stem from the very structure of the machines, which are modelled on
what is inside your head. But while your brain has 100 billion neurons
interlinked in a huge electrochemical mesh, neural networks are made up of
purely electronic neurons, known as 鈥渘odes鈥濃�攖ypically about 100 of them.
And while every neuron in the brain connects to some 10 000 others, nodes in a
neural network are lucky if they are wired to a few dozen of their peers.

Clearly this leaves plenty of scope for computer scientists to make these
machines smarter, more adaptable and altogether more brain-like. One of the
chief routes towards this goal has been to increase the number of nodes and the
richness of their interconnections. So how is it that researchers in Britain, at
the University of Sussex, have managed to create devices with the capability of
large, complex neural nets that consist of just a handful of nodes and sometimes
not a single interconnection?

The answer, which is inspired once again by the workings of the human brain,
lies in a virtual gas. The researchers鈥� approach, which they presented last
month in Sk枚vde, Sweden, at the International Conference on Artificial
Neural Networks, opens the way for a new generation of powerful, lean computers,
which they call 鈥済as nets鈥�. The notion of gas nets is also giving
neuroscientists a way to improve their simulations of the workings of the brain.
鈥淚t represents a considerable step forward in understanding biological and
artificial neural networks,鈥� says Dario Floreano of the Laboratory of
Microcomputing at the Sony Computer Science Laboratory, Tokyo.

Though neural computers are based on the brain, they are actually pretty
imperfect models鈥攁nd not just because they have such paltry numbers of
nodes and interconnections. A brain cell fires off an electrical impulse when
the sum of the signals it receives from other neurons reaches a certain
threshold. This much is copied by neural networks. A node carries out a
mathematical procedure鈥攚hich may be simple addition or something more
complicated鈥攐n the inputs it receives from other nodes. If the result is
above a certain threshold, then it fires an output.

In the brain, neurons are separated by gaps, called synapses, and
communication across these tiny chasms is carried out by chemical messengers,
called neurotransmitters. So an impulse from one neuron must first be converted
into a neurotransmitter, which is then converted back to electricity by the
receiving neuron. To complicate the picture, different synapses have different
effects on the receiving neuron鈥攕ome may stop it firing, for
example鈥攁nd the effects change over time.

To mimic these effects, the wires between nodes in neural computers carry a
variable weighting: each one may increase or attenuate the signal it carries.
This is the key to how neural computers 鈥渓earn鈥�. A network is 鈥渢rained鈥� to
recognise different patterns of input signals by changing the weights and firing
thresholds of the nodes until it produces the required output.

However, computer scientists have largely ignored synaptic chemistry. As a
result, neural computers miss out many subtle effects that take place in the
brain. Sometimes, for example, a neurotransmitter released at one synapse can
change the way the receiving neuron responds to signals arriving at its other
synapses鈥攅ither boosting or blunting them.

Nor are all these 鈥渘euromodulatory鈥� effects confined to interconnected
neurons. A decade ago, brain researchers were surprised to find that a
neurotransmitter could spread its modulatory message to distant neurons.
Stranger still is that in large quantities this simple chemical is toxic and has
a bad reputation as a constituent of photochemical smog. That chemical is nitric
oxide (NO).

Because NO is so much smaller than other neurotransmitters, it can pass
unhindered through cell membranes. And when the gas meets a neuron with a NO
receptor, it can raise the amount of neurotransmitter released by that neuron in
response to an electrical impulse. In effect it amplifies the neuron鈥檚 influence
on the cells it feeds into. The discovery of NO鈥檚 long-range abilities
demolished the notion that neurons communicate only via synapses and only with
their neighbours. It also showed that an artificial neural network with nodes
connected by wires alone was not just an imperfect model of the brain, but a
pale shadow of it.

Whiff of gas

The work at Sussex brings neural computing closer to current thinking in
neuroscience, by adding a virtual equivalent of NO. It is taking place at the
Centre for Computational Neuroscience and Robotics, a unit set up in 1996 to
encourage neuroscience and computing researchers to talk to one another. It was
here that neuroscientist Michael O鈥橲hea told Phil Husbands, a specialist in
evolutionary robotics, about having gas on the brain. 鈥淚 didn鈥檛 know about NO at
all until about a year ago,鈥� says Husbands. 鈥淚t struck me immediately that it
was interesting from a control engineering point of view. I saw that gases could
modulate the network without changing the wires.鈥�

Together with his colleague Inman Harvey and Dave Cliff, who is now at the
Massachusetts Institute of Technology, Husbands has developed methods for
creating software simulations of neural networks by harnessing the power of
evolution. His networks act as controllers for robots, allowing them to perform
simple tasks such as distinguishing a white triangle from a white square in a
confused black landscape (鈥淩obots: the next generation鈥�, 快猫短视频,
14 January 1995, p 32). To start with, Husbands used conventional neural
networks. But after talking to O鈥橲hea he decided to add a whiff of gas to see
what would happen.

To create one of his controllers, Husbands uses a genetic algorithm which
treats the features of a network as though they are genes to be passed from one
generation to the next. The number of nodes, the patterns of wiring between
them, the weightings applied to those wires and the firing thresholds of the
nodes are all thrown into the genetic mixer. For robots using a camera to see,
Husbands also allows the algorithm to choose any number of pixels from the
camera image and how they connect to the nodes.

Next, a computer generates 100 different networks, all with randomly chosen
values for the features. Each network is tested to see how well it performs,
using a computer simulation of the triangle-location problem. Poorly
performing networks are thrown out, but the better networks are allowed to
reproduce by swapping a gene鈥攁 feature鈥檚 value鈥攈ere and there. The
values assigned to a feature can also change at random, mimicking the mutations
that happen in nature. The new networks created in this way are then tested once
more and the whole cycle repeated. Successive generations yield networks that do
a better and better job of guiding the robot to its goal, until eventually an
optimal solution emerges.

The random nature of the evolutionary process means that the genetic
algorithm does not converge on the same 鈥渂est鈥� network every time it runs.
Husbands hoped that adding gas would increase the number of ways that networks
could evolve, and perhaps generate simpler solutions. But there was a
problem.

鈥淭here isn鈥檛 any space or time in conventional neural networks,鈥� says
Husbands. For virtual NO to have any effect, the positions of all the nodes
would have to be known, together with some way to describe how the gas diffuses
over time. To keep things simple, Husbands and his colleagues limited the nodes
to a flat surface, rather than three-dimensional space, and used a fairly crude
description of how the gas would diffuse in a growing circle.

So, the genetic algorithm for the gas net has to take into account the
positions of nodes, the 鈥渇iring threshold鈥� at which a node will emit the gas,
the speed of diffusion of the gas and whether the receiving neurons became more
or less sensitive to incoming signals. All this on top of the features of a
regular neural net.

Working with postgraduate student Tom Smith, Husbands decided to repeat a
series of tasks previously tackled by Nick Jakobi, who now works at
Math茅matiques Appliqu茅es in Paris. Using genetic algorithms,
Jakobi had consistently evolved conventional neural networks for the triangle
location task after about 6000 generations. A typical successful network used 46
nodes, well over 100 wires and eight pixels from the camera鈥檚 output.

By comparison, Husbands and Smith found that a gas net capable of guiding a
robot to a triangle rarely needed more than 1000 generations, and in some cases
they emerged after only a couple of hundred. The gas nets were also far simpler
than Jakobi鈥檚. A typical gas net used between 5 and 15 nodes and only two or
three pixels. Even more remarkable, the nodes of the gas nets were connected by
hardly any wires: they influenced one another mostly via the virtual gas.

鈥淭his demonstrates the power provided by having two distinct yet interacting
processes at play. Signals are flowing down the wires connecting the nodes at
the same time as the gas modulates the properties of the nodes,鈥� says Husbands.
鈥淪tructurally simple yet dynamically sophisticated networks could be really
useful in, for example, space missions, where you need minimalist systems.鈥�

Route finder

Husbands has also investigated the gas nets鈥� powers of memory. He placed a
robot on a simulated road, with a light on either side, that ended in a
T-junction. As the robot travelled down the road, he switched on just one of the
lights. The robot had to remember which side the light came on and turn in that
direction when it reached the junction.

Again, Jakobi had previously carried out this test with conventional neural
networks and found they evolved to handle this task in an average of 1000
generations. When Husbands added gas to the networks, it took less than 100
generations to reach the same goal. One curious aspect of this experiment is
that some of the final gas nets operated without any gas at all, although all of
them used gas during their evolution. This suggests the gas played a key part in
the learning process of the networks.

O鈥橲hea finds this particularly interesting. He and other neuroscientists
suspect that NO has an important role in learning and memory in the brain. 鈥淥ne
thing that happens as a result of learning is that the structure of the nervous
system changes,鈥� says O鈥橲hea. If a synapse is used frequently, the amount of
neurotransmitter that crosses it increases, so the transmitting neuron has a
greater effect on the receiving cell. This synaptic 鈥渟trengthening鈥� leaves a
long-term memory of prior brain activity. But how does it happen? As NO can
diffuse backwards across synapses from the receiving to the transmitting neuron,
it looks like a strong candidate.

But testing NO鈥檚 role in living creatures is not simple. One solution is for
scientists to model the diffusion of NO in a simulated section of the brain.
Andrew Philippides, a postgraduate student working with O鈥橲hea, is doing exactly
that. He is using a sophisticated three-dimensional network in which neurons
have realistic shapes. 鈥淭his allows us to get a feel for the geometry of the
system and the shape of the cloud of NO as it diffuses away,鈥� says O鈥橲hea. The
only drawback is that this is a mammoth computational task. Simulation of just
one second鈥檚 activity in a network of 12 neurons takes hours of supercomputing
time.

But what of the strategy of building ever bigger networks with more and more
complicated wiring? This is the approach, for example, of Hugo de Garis at the
Advanced Telecommunications Research Laboratory in Kyoto, Japan, who wants to
build a 10 000-node 鈥渞obot kitten鈥�. Husbands is sceptical of this approach.
鈥淥ften by adding more connections you tend to screw up something that is just
starting to work,鈥� he says. 鈥淕as effects aren鈥檛 permanent. They鈥檙e only active
when necessary. It鈥檚 a gentler kind of effect.鈥�

Gas makes neurons more plastic, says O鈥橲hea. 鈥淔or any connectivity pattern
you can have a number of different behaviours depending on the way it is
modulated by gas,鈥� he says. 鈥淣eurons in the circuit are different at different
迟颈尘别蝉.鈥�

It is too early to tell where NO will eventually lead. 鈥淭his may send neural
network research off in a new direction,鈥� says Husbands. His next goal is to
develop gas-net robots with more complex behaviour than any robot controlled by
a neural net. 鈥淚f you want to build machines with anything like significant
levels of intelligence, we think that wires and nodes will not be enough,鈥� he
says. 鈥淵ou鈥檒l also need an artificial pharmacology.鈥�