¿ìè¶ÌÊÓƵ

Danny Hillis believes that the world’s computer designers face an upheaval
comparable to an industrial revolution. Until recently, what distinguished
a powerful supercomputer from its rivals was the speed of the processor
chip at its core, where all the computing is done. The faster the chip,
the more powerful the computer. Engineers continue to develop faster and
more powerful versions of these chips. But Hillis says their efforts will
meet the same fate as attempts by skilled craftsmen of past centuries to
produce their wares more and more quickly. They were eventually engulfed
by mass production in factories. Hillis, who helped create Thinking Machines
Corporation in Cambridge, Massachusetts, says a similar transformation awaits
computing.

The latest generation of supercomputers replaces the single central
processor with hundreds or thousands of separate processors, all working
together. This approach, called ‘massively parallel processing’, turns the
computer into a factory, with thousands of parallel assembly lines churning
out calculations and swapping results back and forth. Whether it works well
depends partly on the speed of each assembly line – the capabilities of
the computer’s processing chips and internal communications. Far more important,
however, is software to manage and direct the computing factory. Just as
management is crucial for successful manufacturing, well-designed software
must translate any computing problem into a form that an array of processors
can handle efficiently.

Hillis and his company pioneered the development of massively parallel
computers. But when they began, about a decade ago, factories were far
from their minds. Hillis was a graduate student in the Artificial Intelligence
Laboratory at the Massachusetts Institute of Technology. One of his professors,
Gerald Sussman, began prodding students to come up with an entirely new
form of computer that would be capable of something resembling intelligence.
Sussman proposed designing a machine that would be crammed full of processors,
all operating simultaneously and communicating with each other.

The idea attracted funding from the Pentagon’s Defense Advanced Projects
Research Agency and from a few commercial investors. So in 1983, a group
of researchers, including Hillis, left MIT and transferred the entire project
from the AI laboratory into the new company they decided to call Thinking
Machines. By 1986, they had built their first working model, and named it
the Connection Machine. Though it contained 65 536 processors, this was
a compromise with engineering reality; their original vision called for
a million processors.

The group never thought the Connection Machine would be used for heavy-duty
number crunching, the kinds of tasks carried out by huge supercomputers.
The individual processors were too primitive – much less powerful than the
processors in typical personal computers – and it seemed too daunting a
task to write software that would divide up a single large problem among
the machine’s thousands of processors. Nor did the company’s academically
minded founders have much interest in solving prosaic problems. ‘We were
artificial intelligence people,’ recalls Cliff Lasser, a member of the original
group who is now in charge of several software projects for Thinking Machines.
The Connection Machine could only be programmed in Lisp, a language loved
by artificial intelligence researchers, because it gives them a way to express
logical relationships between different concepts, but hardly ever used in
commerce or engineering. Lasser and Hillis thought their novel machine might
be able to recognise objects or crudely mimic the processes of human reasoning
by taking advantage of interactions among the computer’s thousands of processing
units or ‘nodes’.

RADICAL TURN

Attitudes began to change around 1986. Richard Feynman, the American
physicist, whose work on quantum electrodynamics won him a share of a Nobel
prize in 1965, decided to join his son Carl at Thinking Machines. He soon
began telling everyone who would listen that this machine might be surprisingly
good at solving some standard physics problems. Richard Feynman pointed
out that many of these problems are naturally parallel: they involve carrying
out a few basic calculations on a host of different pieces of data, and
these calculations could all be done simultaneously on the Connection Machine.

Computer simulations of a piece of metal being stretched or bent, for
instance, represent the metal as a grid of data points. To simulate the
effects of physical stress, the computer repeats a set of calculations on
data from every point on the grid in turn. The new values from each point
on the grid then affect those next to it, and with each simulated step forward
in time, the computer must recalculate data at each point.

Feynman realised that instead of a single processor going through calculations
for each point in turn, thousands of processors could work in parallel,
calculating values for the entire grid in a single step. The data from each
point would be kept in memory attached to each processor. As part of every
cycle, the nodes would send data to each other, reflecting the impact of
stresses and strains moving along the piece of metal.

Feynman convinced enough researchers at Thinking Machines for them to
go ahead and write a program to solve such a probllem in Lisp, a curious
match of an AI language with a physics problem. ‘It was actually sort of
a joke,’ says Lasser. The shock came when the computer actually ran the
program. The Connection Machine’s primitive processors, working in concert,
churned out calculations at a rate of 120 million floating point operations
per second, or 120 megaflops. This was equal to speeds turned in by the
reigning supercomputers of the day.

This was the Connection Machine’s first step into the mainstream of
scientific and commercial computing. Today, Thinking Machines has grown
to a company with 500 employees and annual sales of $90 million. It is
not alone. The same year that Thinking Machines emerged from MIT, the California
Institute of Technology spawned a company called Ncube, which also builds
parallel machines. And during the past two years, computing giants such
as Intel, Cray and IBM have announced that they are developing their own
massively parallel computers.

Most of these companies approached the design of parallel computers
from the opposite direction from Thinking Machines. Their first parallel
computers had 16 or 32 relatively powerful processors, and so were essentially
a collection of traditional workstations harnessed in parallel. The Connection
Machine from Thinking Machines started with thousands of primitive processors,
requiring the programmer to approach the problem in a fundamentally different
way from the start. But as the original Connection Machine, called the CM-1,
gave way to successor models, the processors used in each subsequent machine
have increased dramatically in power and sophistication, while their number
has actually gone down.

COSTLY CONNECTION

In the latest version of the Connection Machine, the CM-5, each node
contains a Sparc processor, built by Sun Microsystems for its high-performance
workstations, along with four ‘vector processors’ to carry out high-speed
computations. Attached to each node is 32 megabytes of memory, 60 000 times
as much memory as that attached to each processor in the first Connection
Machine. Each node has a peak speed of 128 megaflops, about 20 times as
great as that of a workstation. All these high powered chips are expensive.
Production versions of the CM-5 contain a maximum of 1024 processors and
can cost up to $10 million. Thinking Machines says it is willing to build
a CM-5 with up to 16 000 of these processors, but no-one has offered to
pay the $200 million that it would cost. As chips become cheaper, such
a machine may eventually become affordable.

Companies like Intel or Ncube, which started with powerful processors,
have gradually increased the number in their machines from 16 to 64 or
even several hundred. But despite this apparent convergence between Thinking
Machines and its competitors, the two sides still represent rival camps
in the world of parallel computing. Their differing approaches to the design
of parallel machines have also led to different ways of writing software
to solve large problems in a parallel way.

Until now, the primary customers for massively parallel computers have
been universities and research centres such as Los Alamos National Laboratories
in the US. In Europe, the CM-5 has been ordered by the French National Centre
for Parallel Processing and by the Nuclear Research Centre at Julich in
Germany. But these machines, which can cost from $1.4 million to $10 million,
depending on their size, have found commercial customers as well. Petroleum
companies are using the Connection Machine to run geological models that
help them search for oil. Dow Jones, the financial information and news
service, is setting up a parallel computer to search huge databases. American
Express has just bought a CM-5 but the financial services company is not
saying exactly how it plans to use it.

UNEXPECTED SUCCESS

The founders of Thinking Machines seem astonished and a little discomfited
by their own success. Lewis Tucker, a senior scientist there, says that
he used to spend much of his time trying to persuade people that massively
parallel computing made sense. These days, no one questions the idea. ‘We’re
part of the establishment now,’ says Tucker. ‘It’s really weird.’

Thinking Machines has been transformed from a small band of artificial
intelligence researchers into a supercomputer company. Not everyone is entirely
happy with the change. ¿ìè¶ÌÊÓƵs who pursue pure research on true ‘thinking
machines’ have been pushed out to the fringes of the organisation. Yet in
contrast to outward signs, such as Pinkerton security guards at the company’s
door, the original corps of researchers at Thinking Machines still cultivate
a decidedly noncorporate style. Lasser confesses that ‘almost no one uses
our machines to make money yet’, because companies are still adapting their
software to run on the Connection Machine. Tucker tries to escape his office’s
smoking ban whenever possible for discussions on a bench along the Charles
River. Hillis, the company’s ‘founding scientist’, scurries about in jeans
and a T-shirt, not long back from six months at an institute in New Mexico
where he was experimenting with small computer programs that ‘evolve’ according
to the rules of natural selection.

Massive parallelism began its march from the fringes of the computer
industry into its mainstream when researchers, urged on by Richard Feynman,
realised that ‘a lot of problems are embarrassingly parallel’, says Tucker.
According to David Forslund, deputy director of the advanced computing laboratory
at Los Alamos, most scientists initially thought that parallel machines
would perform well on a few problems, but poorly on most of them. Gradually,
says Forslund, they realised that it is the other way round: most large
computing jobs can be handled naturally in parallel. This was tremendous
breakthrough.

Models of the Earth’s atmosphere and oceans consist of thousands or
millions of grid points, and changes in them can be recalculated in parallel.
The same is true of models of fluid flows or the programs known as ‘crash
codes’ that simulate the effects of a car accident on the vehicle and its
occupants. Conveniently for the programmer, these problems usually require
each processor to share data only with processors handling grid points next
to it, which keep demands on the computer’s internal communications system
within manageable limits. Some other kinds of problem may be less well suited
to massively parallel computers. Computer-aided design programs, for example,
require data from every point on the design to be transmitted to every other
point, which creates a communications bottleneck. But programmers at Thinking
Machines believe that even this problem can be solved once they seriously
turn their minds to considering it in a parallel way.

Retrieving a specific piece of information from a huge mass of data
is a perfect problem for a massively parallel computer. The set of data
is simply divided up across the memory of each processor, and they all search
for the desired information at the same time. Parallel computers are also
ideal for presenting data visually on a screen, because one processor can
be assigned to each point, or ‘pixel’, that makes up the image. To analyse
the structure of a complex molecule in biology, each processor could be
assigned to each atom, and told only which other atoms it is connected to.

The highest hurdle that massively parallel computers still have to overcome
is the lack of software written for them. A computer without software is
about as useful as a machine that exists only as a pile of parts on the
floor. So most R&D work at Thinking Machines these days is devoted to
programming. ‘We’re a software company now,’ says Tucker.

In the mid-1980s, when the Connection Machine could only be programmed
in Lisp, it was of little use in the engineering or commercial world. For
the machine to be programmed in a standard engineering language like Fortran,
the company had to write a piece of software called a Fortran compiler to
translate the standard commands of Fortran into the binary ‘machine code’
that actually controls the computer’s circuitry. Each new type of computer
has its own unique wiring pattern, so each one requires its own compiler.
Writing such a program is a huge and laborious task; the latest version
of Thinking Machines’s Fortran compiler is made up of about 600 000 lines
of code.

Most of the researchers at Thinking Machines wanted nothing to do with
the compiler project, says Lasser, because most of them thought Fortran
was ‘clunky’ and inferior. But marketing staff pushed for it, and a team
of 10 or 20 programmers laboured four years to produce a compiler for a
new version of the language, called Fortran-90. This version of Fortran
includes new commands designed specifically with massively parallel computers
in mind, allowing programmers to handle data as arrays so that the data
can be spread across an array of processors. There is also a new version
of the language C, called C* (C-star), that is designed for parallel computers.

Most existing software, however, is not written in Fortran-90 or C*,
but in earlier versions of these languages. So Thinking Machines developed
software tools that automatically translate commands from traditional Fortran
into equivalent commands that can be dealt with in parallel. For instance,
in traditional Fortran, programmers use a command called a Do-loop to instruct
the computer to carry out a particular operation on many data points in
turn. This Do-loop can now be translated into a command that does the operation
on an array of data all at once.

SHARED MEMORY

Getting a program to run on a massively parallel machine is not that
hard. In parallel machines with powerful individual nodes, a program can
be transferred quite easily to run on just one of the machine’s nodes. A
company called Kendall Square Research has made this task even easier with
an innovation called ‘shared memory’. In its massively parallel computer,
each proccessor can store and retrieve data from a central store of memory,
rather than just from the memory associated with a particular node. There
is not much point to buying a massively parallel computer, however, if the
software it runs only keeps one processor busy. The challenge is to redesign
software so that it takes full advantage of thousands of processors at
once.

Theoretically, the full-scale CM-5 has a peak processing speed of 130
gigaflops, or 130 billion floating point operations per second. This implies
that the CM-5 could solve a problem eight times as fast as the newest and
fastest Cray supercomputer, the Y-MP C90, a machine announced only last
year that incorporates 16 high-powered processors. But when assigned to
solve problems from the real world, with software that was originally written
for traditional computers, massively parallel machines often achieve only
a fraction of these advertised speeds.

In order for parallel computers to reach their potential, programmers
have to rethink fundamentally how they want to attack a problem, and rewrite
programs in ‘parallelese’. Yet many of the most important computer programs
in use today are enormous creations that stretch over hundreds of thousands
of lines of computer code. They represent years of accumulated programming
labour, and scientists and companies that depend on them will not willingly
abandon them. Few of these huge ‘codes’ run on the Connection Machine because
they have not yet been massaged into a form that is handled well by massively
parallel computers. The task of transforming them could be done in a year,
or it might take longer. ‘The true answer is, we don’t really know,’ says
Lasser with a happy grin.

The software shortage is being attacked from many sides. A group at
Los Alamos is rewriting a large climate model for the Connection Machine,
and similar projects are under way at other government laboratories and
universities. Large private firms, such as the oil companies, have begun
translating their proprietary computer codes. Companies that sell massively
parallel computers are also working on this prolblem, because it is the
key to larger sales. ‘There’s tremendous pressure to put this machine into
the hands of ordinary people,’ says Lasser.

According to Forslund, one of the major problems with these software
projects is that they may produce programs that only work well on one type
of massively parallel machine. This is a major risk, because no one can
be sure which computers will succeed in the marketplace.

Part of the problem is that many massively parallel machines, such as
those built by Intel and Ncube, are designed to be programmed in a fundamentally
different way from the Connection Machine. Thinking Machines pioneered an
approach called ‘data parallelism’, sometimes called ‘single instruction,
multiple data’ or SIMD. In this technique, data is distributed among the
processors, which then march in step through a set of instructions, carrying
out the same operations on different pieces of data.

Intel and Ncube, however, have chosen an approach called ‘multiple instruction,
multiple data’ (MIMD), sometimes called ‘message-passing’. If a factory
built on the SIMD model can be likened to a set of parallel assembly lines,
an MIMD factory is full of many small groups working simultaneously, at
their own pace, on different pieces of a product. As each group finishes
its task, it passes its result on to another group, until the final product
is assembled. An MIMD system does not distribute data among different processors
as an SIMD machine does; instead, it parcels out different pieces of the
computing task, or subroutines. Each processor is programmed independently.

Both designs have their defenders. Some programmers prefer the MIMD
approach because it seems a more natural evolution from traditional computers.
Advocates of MIMD also suspect that many problems cannot be handled well
by SIMD computers. These include chess, and artificial intelligence systems
that require a computer to follow complex rules of behaviour. SIMD machines
are also handicapped when small pieces of data have to be treated differently
from the rest. When running a climate model on an SIMD computer, for instance,
calculations for small areas covered by clouds may have to be carried out
on a few processors while the rest simply mark time. This slows down the
entire program.

Yet MIMD computers hold their own perils. They are full of pitfalls
for the programmer, who must keep track of what each processor is doing
at all times – a huge management task. Robert Hyatt, a computer scientist
at the University of Alabama, says that he would rather write a program
for the Cray Y-MP C90, with its 16 very powerful processors, than for a
massively parallel machine. ‘Do you want to use 16 elephants to trample
down the forest or would you rather use 16 384 small dogs? I would rather
spend my time and effort keeping only 16 elephants in line,’ he says. This
is one reason why MIMD machines have generally linked together a smaller
number of powerful processors, rather than the Connection Machine’s thousands
of less powerful processors. Yet if powerful computers of the future continue
to rely on increasing the number, rather than the speed, of processors,
the task of programming them will be made more and more complicated. The
SIMD method avoids some of these pitfalls. All the processors can be programmed
as one, because they all do the same thing at the same time.

COMPUTING’S NEW ERA

Forslund says that the computer industry is trying to bridge the gap
between the two approaches. The latest version of the CM-5 can be programmed
in either style. All companies that sell parallel computers are now cooperating
on a new language, called High Performance Fortran, that is supposed to
help programmers create software that will run well on both SIMD and MIMD
machines. ‘That’s where we’re trying to glue things together,’ says Forslund.

At the moment, massively parallel computers are stuck somewhere between
nuts-and-bolts work on these software problems and the grand vision that
some of their creators, including Hillis, have set out. The journal Daedalus
recently devoted an entire issue to the notion that massively parallel processing
is opening ‘a new era in computation’. In it, Hillis links parallel processing
to such futuristic ideas as all-purpose home robots and ‘virtual worlds’.
Brosl Hasslacher, a physicist in the theoretical division at Los Alamos,
sees them as the key to models of fluid flows that are not based on systems
of equations, but on ‘monster systems’ in which the behaviour of every molecule
is simulated directly as it is bumped and pushed by those around it. Beyond
models for fluid flows, he says, lie similar models for the economy in which
the basic elements are not molecules, but people and their money. James
Bailey, the marketing director of Thinking Machines, goes further. He says
that parallelism may ‘reshape thinking that has gone unchallenged since
the time of Newton, Descartes and even Aristotle’. Underlying much of this
rhetoric is the dimly perceived knowledge that in some way the human brain
with its billions of neurons must be a kind of enormously parallel computer.

Yet today, massively parallel computers do nothing traditional computers
cannot do. Their most ambitious claim is that they do many things much faster.
A group at Los Alamos reworked a standard ocean model for a CM-5, for example,
and it produced results five to 20 times as fast as a Cray Y-MP, a machine
that was the world’s fastest supercomputer when it was introduced in 1988.
On other problems, a parallel machine may produce results hundreds of times
as fast. Another benefit is that surprising advances have sometimes emerged
when scientists rethink problems, looking for ways that parallel computers
could solve them more easily. Forslund says that, in this way, scientists
came up with an algorithm for predicting the orbits of particles in a fusion
reactor that increased the efficiency of calculations by a factor of 1000.

But Bailey says a simple increase in computing power, while not a qualitative
change in itself, can revolutionise the ways that computers are used. If
a task that used to take several days can be accomplished in an hour, it
may suddenly become worthwhile to try it, he says. So when more computing
power becomes available, researchers do not just do more of the same thing;
they try something different. New models of aerodynamic behaviour will be
created, huge databases will be set up, and scientists will write software
to turn their data into colourful images that allow them to understand and
present the information better. Historically, says Bailey, every tenfold
increase in computing power, or every tenfold decrease in its cost, has
opened the door for a new generation of computer applications.

Some researchers are not even convinced that parallel computers of the
future will be contained in one cabinet. Small ones may exist as a collection
of powerful workstations connected by high-speed communication links. In
this model, called distributed computing, a special network of computers
just large enough for the particular job in hand, can be called into service
at a moment’s notice. All the computers, together with their communication
links and software, form one ‘meta-computer’ for as long as it takes to
finish the task. The advantage of this system, which is being tried out
experimentally at many universities, is its flexibility.

Hyatt says that the possibilities of such meta-computers remind him
of a science fiction story. In the story, people link together larger and
larger computers to analyse the most fundamental questions of life, but
each time the computer responds that it is unable to compute an answer.
Finally, as Hyatt remembers the tale, every computer in the entire world
is integrated into one gigantic super-meta-computer. The scientist types
in his query: ‘Is there a God?’ The computer calculates, blinks, and transmits:
‘Now there is.’