¿ìè¶ÌÊÓƵ

Nippon Electric Company’s new research laboratory in Tsukuba science
city, just outside Tokyo, boasts many powerful computers. NEC is, after
all, Japan’s largest electronics company and one of its most well known.
But when researchers at the laboratory want to examine a really sophisticated
data processor, they peer through a microscope into a Petri dish containing
half a million transparent worms. The worm is a nematode called Caenorhabditis
elegans; one-third of its body, which grows up to a millimetre long, consists
of brain cells. These cells instruct the worm to wriggle away from hostile
environments, seek out nutrition and reproduce itself. No manufactured computer
can handle anyhing like this complexity of information.

Such direct comparisons, of course, are unfair. A nematode’s neural
system is probably incapable of adding two and two, let alone carrying out
hundreds of millions of mathematical instructions per second, as digital
computers can. On the other hand, the brute processing speed of digital
electronics can create the illusion that computers behave more intelligently
than worms-at least some of the time.

In Japan, computer-makers and government agencies have begun to turn
their attention to a type of data processor that has more in common with
a nematode’s nervous system-or the human brain-than with digital machines.
This is the neural computer, which is a device that is modelled on the neurons
in the brain and the synapses that connect them. Over the past 50 years
or so, research into neural computing, mainly in the US, has had periods
in and out of fashion. The computing machines that have emerged have few
practical uses, apart from acting as tools for modelling the supposed behaviour
of animal brains.

Thanks to the government of Japan, neural networks are about to enter
the mainstream of computer development. Next year, the Ministry of International
Trade and Industry plans to launch a 10-year international research programme
that aims to tackle some of the problems of building neural computers. The
project will cost around 200 billion yen (about 770 million Pounds) and
involve universities, government laboratories and private companies in both
Japan and the rest of the world.

As a successor to the notorious 10-year project to build a fifth-generation
computer, which is due to end next year, the new project has earned the
unofficial epithet of ‘sixth generation’. The fifth-generation project aimed
to build computers capable of manipulating data as pieces of knowledge rather
than numbers, and leading computer scientists are still debating whether
it will succeed (‘Do computers dream of intelligent humans’, ¿ìè¶ÌÊÓƵ,
26 November 1988). The sixth generation project will take computing technology
into another order of magnitude of complexity. ‘The fifth-generation is
working on computers that have about 100 parallel processors,’ says Hidehiko
Tanaka, a professor at Tokyo university who has worked on both fifth and
sixth generation research. ‘The coming project will work with networks of
more than 10,000 processors.’

Neural computers have a history almost as old as electronic computing
itself. More than 40 years ago, Warren McCullough and Walter Pitts at the
universities of Illinois and Chicago proposed building an artificial neural
network of simple processors. They suggested circuits of transistors that
would be analogous to the neurons in the human brain. Their proposition
was that the connections between processors are as important as the processors
themselves.

The reason for the interest is the special ability of neural networks
to make sense of imprecise data. Conventional digital computers work in
a black and white world of ‘either/or’; neural computers can handle less
precise or ‘fuzzier’ inputs of data by acting on the sum of decisions made
by many different parts of the network. This means, for instance, that one
wrong keystroke does not jeopardise an entire program run by a neural network
as it would a program run by a conventional computer.

The ability to handle imprecise data is especially exciting to developers
of machines that understand human speech or handwriting, or of robots with
vision. Even more intriguing is the ability of neural networks to achieve
something rather like learning. They do this by a process called ‘back propagation’,
in which the signal from the output layer is fed back to the links between
neurons. By ‘weighting’ the connections according to the correctness of
the data passing through – that is, according to whether the information
fits a pre-ordained pattern – the network can ‘learn’ to recognise the pattern
if it appears again. This means that when the network confronts a task it
has met before, it will do the task more quickly.

MITI’s 10-year project will try to tackle four areas: basic studies
of neural computing models; applications of neural computers; super-parallel
computers; and optical processors, which MITI says are the most promising
technology for creating neural networks. Tanaka describes the project as
‘rather pointed at basic research’.

At the other end of the spectrum, private companies, which spend four-fifths
of Japan’s total outlay on research and development, are already working
enthusiastically on the development of products. They have found a market
for conventional computers programmed to handle tasks in a ‘neural’ fashion,
building in networks of association between different points in the memory.
Matsushita Electric, the consumer electronics giant, sells a domestic air
conditioner with a neural network that chooses between 4608 possible settings.
The company says that the machine’s ability to measure an imprecise and
varying input of signals from many different thermometers helps it to choose
exactly the right setting.

For most companies developing neural computers, the main payoff up to
now has been in public relations. In 1988 Fujitsu, Japan’s largest maker
of mainframe computers, mounted a startling public demonstration of wheeled
robots guided by neural networks around a model landscape. The networks,
simulated by a program in simple 8-bit computers, weighed up data from 12
sensors mounted on the robots and decided accordingly on five different
courses of action. The demonstration featured a team of robots dressed as
Sherlock Holmes, the fictional British sleuth of 19th-century novels, and
a squad of policemen hunting down another robot dressed as a villain.

Teaching computers to read

NEC has four research groups working on neural technology. Besides the
laboratory for fundamental studies at Tsukuba, one group is developing optical
processors, another is putting the technology into computers and robots,
while the fourth is finding practical applications.

One of the first aims of Kazumoto Iinuma, head of the work on applications
at NEC, is to improve the accuracy of machines that read text. For the past
10 years, the company’s researchers have been trying to develop a neural
network that will iron out the errors that creep into conventional computing
systems when any ambiguity forces digital circuits to make an either/or
decision. NEC’s experimental device, which can run on a modified personal
computer, can recognise the 26 characters of the Roman alphabet.

The company has also developed a neural technique for analysing the
structure of complicated sentences for translating machines. The technique
can judge which ‘unit’ (a logical group of words containing a verb) modifies
another ‘unit’. NEC says that it has developed a classical neural network
with three layers of processors: there are 505 neurons in the input layer,
10 in the output layer and 30 in the hidden layer in between. The company
claims to have tested the device on 370 sentences, each containing five
verbs.

Another experimental network from NEC may improve the accuracy of industrial
robots. A neural network compares the coordinates of several dozen points
on the robot’s programmed path with the coordinates of points through which
the robot actually passes. It then instructs the robot to compensate for
any deviations. Such robots should be far easier to train than existing
ones, where an inaccuracy of a few millimetres can cause a whole production
line to break down.

All these ‘neural computers: are hybrids, in which conventional circuits
of digital logic are programmed to resemble neural networks. Such programs
work well in simple networks, but make heavy demands on computer power.
Although the network gives the impression of carrying out many tasks at
once, in reality every instruction has to take its place in the queue for
the central processor’s time. The number of computations grows exponentially
with the number of neurons.

Building more powerful neural networks means moving to purpose-designed
circuits. Last November Hitachi, the engineering and electronics company,
said that it had built a computer consisting of eight neural networks, each
consisting of 1152 interconnected neurons, built into large-scale integrated
circuits. The company says the machine will be on sale within two years.
Among the applications it has in mind are systems to match fingerprints
and to analyse the fluctuations of stock-market prices – a matter of some
obsession in Japan, which had the world’s largest stock exchange until it
lost half its value in last year’s crash.

But the real breakthroughs in neural circuits may come from optical
rather than electronic chips. Laying down three-dimensional networks of
wires in a microchip is fiendishly difficult. With more intricate networks
it will probably be better to connect processors with beams of light passing
freely through translucent material. Unlike electronic signals, which would
interfere with each other, optical beams can criss-cross each other through
a solid device.

Several companies have reported progress. Last July, Mitsubishi Electric,
the electronics and computer company within the Mitsubishi group, said that
it had developed an optical chip that could recognise the Roman alphabet.
Unlike the NEC device, in which images are converted into digital code before
they are processed, the Mitsubishi chip can process images of characters
directly. The chip, which has an area of one square centimetre, contains
90 neurons divided into input, hidden and output layers. the actual components
are 66 light-emitting diodes, 3648 light modulation elements to act as synapses
and 110 photo-detectors to read the result and convert it into an electronic
signal. Packing in more neurons is simply a matter of making smaller components,
reported Mitsubishi’s researchers in the American journal Optics Letters.

Japanese manufacturers have shown an astonishing talent for putting
huge numbers of electronic circuits onto silicon chips: companies are now
building production lines for memory chips capable of storing and retrieving
16 million bits of information. Building a chip containing tens of thousands
of neurons connected with synapses may be difficult – it will need new materials
and breakthroughs in ways of laying down three-dimensional structures on
microchips – but this is exactly the sort of skill at which Japanese engineers
excel.

Once engineers have discovered ways of building neural networks, what
can they do with them? At the moment, for all companies talk of rebuilding
human brains, neural computers are simply devices for matching patterns.
‘The networks made so far are only classifiers,’ says Tanaka. ‘We want to
build computational models.’ The real prize would go to a computer that
could be left to learn on the job – a self-programming computer.

Everyone agrees that building a brain-like computer needs a lot more
understanding of how neural processes work in nature. Enter NEC’s nematodes.
The company has about a dozen scientists studying nematodes, chosen from
other organisms because their genetic make-up is relatively well understood.
If this is unusual research for an electronics company, the team’s leader,
Johji Miwa, is an unusual figure. Most corporate researchers in Japan are
people who have worked for the company all their lives: they are ‘company
men’ (literally – opportunities for women are scarce). Miwa is different.
He was an academic who joined the team six years ago after a career that
took him to the US and West Germany. Miwa aims to look at the animal as
an information processing machine. ‘Once we understand it, we should be
able to make a machine like it.’

The focus of Miwa’s research shows how far scientists still are from
understanding even this most simple of animals. The difference between an
animal brain and a computer, Miwa says, is that an animal does not distinguish
between hardware and software. The way we build a computer has no influence
on its operation; in an animal, development and behaviour are inseparable.

Studying worm genetics

Miwa set out to prove this by finding a clear example of a single phenomenon
affecting both development and behaviour in an animal. C elegans, the microscopic
nematode, is an ideal subject because of its suitability for classical genetics.
‘It grows from fertilisation to fertilisation in three days,’ Miwa says.
‘Each then produces between 200 and 300 offspring. You go from one to eight
million worms in one week!’

He found an example in the effect that a well-known tumour-promoting
chemical, a phorbol ester, has on C elegans. First, a nematode dosed with
the chemical stops growing. Secondly, it loses control of body movements.
It is unable, for example, to wriggle towards more desirable foods. ‘So
we see two effects, one of development, one of behaviour.’ By a process
of genetic trial and error, the team found that only one gene was responsible.
The next step is to find out why it affects both structure and behaviour.

‘If you look at biology as an information machine, there are two systems.
One is a developmental information processing system, the other is a behavioural
information processing system. But these are not separated as in a computer.
So the molecule, or rather the gene, is the one that affects the two important
²õ²â²õ³Ù±ð³¾²õ.’

It is not clear when, or if ever, this research will fit into the mainstream
of neural computing. It may even lead to a new model for mimicking biological
intelligence. According to Roy Lang, manager of NEC’s exploratory research
laboratory: ‘Biology is a very important information processing system and
we are aware that conventional electronic systems will be saturated in the
future so we have to learn from this very different type of information
processing. This alone is important enough to justify supporting this kind
of research.’

One leading Japanese brain researcher, Masao Ito, head of the Laboratory
for Neural Networks at RIKEN, the government’s institute for physical and
chemical research near Tokyo, says existing neural networks are not realistic
models of minds, let alone recreations of them. ‘Backward propagation of
the error signal doesn’t seem to be realistic. There is no real counterpart
in our brains. A real cerebral network must work on a different principle.’
The problem is that no one seems to have much idea of what that principle
could be and, for the moment, the planners of the sixth generation project
are bogged down with more prosaic problems.

The fifth-generation project involved the setting up a special research
centre, the Institute for New Generation Technology, or ICOT. The neural
project will be more diverse, supporting research at existing institutes
and universities. This will sidestep one of the criticisms of the fifth-generation
project, which was that little money flowed to academic groups. It will
also avoid the need to persuade private companies to send their best researchers
to an outside institute.

But this route around one set of obstacles runs straight into another
– rivalry between MITI and the Ministry of Education, Science and Culture,
which, as the custodian of Japanese universities, has its own agenda for
basic scientific research. As one academic put it: ‘The Ministry of Education’s
position is that it should take care of fundamental research and MITI take
care of applied. But now MITI is getting into fundamental areas, so there
is some overlap between the two ministries.’

Another complication is MITI’s determination to make the research an
international effort, unlike the fifth-generation project, which started
out as a purely Japanese affair. Its motives are a mixture of pragmatism
and idealism. But, in the international climate of suspicion over any Japanese
technological initiative, MITI will have to face some detailed questioning
on its motives for launching the project.

Taking advantage of research

Explaining motives to foreigners has never been the Japanese government’s
strong point, and the neural computing programme will be no exception. MITI’s
interest in neural networks has as much to do with Japanese politics as
with science. First, the Japanese establishment has believed for the past
three decades that a strong computer industry is vital for national survival.
To this end, it has nurtured domestic manufacturers, the largest of which,
Fujitsu, is now second only to IBM in the computer world. If neural networks
are the future of computing, Japan should be in the forefront.

Secondly, neural networks offer an attractive solution to another of
MITI’s obsessions: Japan’s shortage of computer programmers. In one of its
more alarmist projections, MITI recently estimated that the country will
be short of one million people with skills in computer software by the end
of the century. A computer that would program itself has obvious attractions.

More nebulous is the special attraction that new types of computer have
in Japan. The leaders of the only front-ranking technological nation to
emerge from an Asian culture have long been unhappy with the limits of computers
handling data expressed in the characters of the Roman alphabet. The past
20 years have seen a succession of hunts for ‘Japanese-ness’ in a computer.
One example is the TRON project, which aims to build a microcomputer especially
adapted to the Japanese language, especially the Chinese characters that
slow down conventional machines. Another is the fifth-generation project
itself. Neural computing is especialy interesting because part of the widespread
myth of Japanese-ness is that Japanese culture is more attuned to subtle
shades than to hard-and-fast rules. This cultural quest may explain why
Japanese business leaders are prepared to spend money on neural computing
while the rest of the world regards it with suspicion.

A final, but not trivial, motive for launching the programme is to safeguard
MITI’s own survival. The ministry has lost many of the powers it enjoyed
during Japan’s postwar reconstruction. Now MITI’s bureaucrats, the cream
of Japan’s best universities, are seeking new roles. One is to extend MITI’s
activities into more fundamental research.

DEspite MITI’s enthusiasm, people who have made careers of selling computers
are keeping an open mind on the commercial potential of neural networks.
Fujitsu’s chairman, Takuma Yamamoto says: ‘The neural computer is not a
cure-all, it doesn’t have the ability to do everything. fifth-generation
or neural computers cannot solve the shortage of programmers. Certainly,
they can supplement other efforts, but they are not the solution.’

In this respect, Fujitsu is at one with its rival IBM, the world’s largest
computer maker, which does virtually no research on neural computing. IBM’s
chairman, John Akers, said in 1988: ‘We feel that in the foreseeable future
it will have little impact.’ And little has changed since then, says a spokesman.
He notes, however, that the company still supports a few modest research
projects in the field, although most of them occupy the time of only one
or two scientists.

The difference between the two computer giants is that Fujitsu will
be in a much better position to take advantage of any worthwhile results
of the sixth generation programme. This is an alarming concept. Only two
countries have industries capable of making state-of-the-art mainframe computers,
the US and Japan. If the sixth generation project comes up with a breakthrough
in neural computing, and if the machines turn out to be commercially useful,
Japan will be alone.

* * *

Building computers to work like the brain

Neural networks are not cognitive machines. But they behave in a way
more like biological brains than digital computers. This shows in their
ability to handle imprecise data and to learn from experience.

The capabilities of a neural computer depend entirely on the number
of processors and the complexity of connections between them. In this respect
it differs from a conventional, sequential computer. A desktop PC can do
everything a mainframe can do, the smaller machine would just take longer
to do it. The human brain contains something like 10 billion neurons. So
far, researchers have managed to build neural networks that consist of only
a few dozen processors: building more complex networks poses formidable
practical and theoretical problems.

The neurons in such a computer network are simple processors or switching
devices that receive data from several inputs. When the amount of data reaches
a threshold value, the neurons fire a more powerful output signal.

A classical neural network has three layers of processors: an input
layer, an output layer and a ‘hidden layer’ in between. A network of ‘synapses’
connects each processor to all the others in the network: each neuron in
the input layer is connected to each neuron in the hidden layer, which is
in turn connected to each neuron in the output layer.

The data to be processed go first to the input layer. In a pattern-recognition
machine, for instance, this data might be the image of the handwritten letter
‘A:. Each input neuron that receives enough data to trigger an output signal,
relays that data to the hidden layer, which, in the same way, relays the
data to the output layer.

The overall effect is of a device that can sum up a lot of imprecise
information and decide whether it matches a pattern or not. If the formation
does match the image of the letter ‘A’, for instance, the output layer makes
the computer respond to the signal. A conventional computer, no matter how
powerful, must make the decision from the first input of imprecise data
– if that decision is wrong, so are all later operations.

The principle of neural networks is the same regardless of whether they
work electronically or optically, or exist merely int he software of a conventional
computer. Biological brains, however, are much mroe subtle than these crude
artificial networks. Each set of biological signals, which is carried by
a cascade of chemical reactions between neurons, seems to modify the network
as it passes through. Researchers need to do a lot more work on brains before
they will understand them, let alone build artificial replicas.

Michael Cross is a freelance journalist in Tokyo.