The scrum around the stands at virtual reality exhibitions is regularly
five deep. Even though ways of immersing people in simulated environments
were introduced more than three decades ago, it was only last year that
computer technology became cheap enough to encourage developers of VR systems
to display their wares in public. Wild claims for this new form of mass
entertainment quickly followed. It would, among other things, make sex both
safe and satisfying. Now the researchers are working hard to cool our overheated
expectations, and pondering the technology’s likely social effects.
‘I honestly believe that the media hype has backfired. People now expect
much, much more than VR can currently give them,’ says Bob Stone, technical
director of the National Advanced Robotics Research Centre (NARRC) at the
University of Manchester. After all, VR is still a small branch of research
with only about 20 laboratories worldwide working exclusively on it. Most
of them are on the West Coast of America, though there are also projects
under way in Japan, and in Europe, principally in Germany, France, Italy
and Britain.
Researchers want to convince potential users that VR is a means, not
an end. They see it as an enabling technology, possibly the ultimate interface
between humans and computers, with applications as diverse as robot operation
in hazardous situations, video conferencing for business people, recreating
crime and accident scenes for police, empowering the disabled, and curing
psychological problems. This may seem like a climb-down from the wilder
leisure pursuits touted in the press – holidays abroad without leaving home
and virtual sex – but developers are now keenly aware of the importance
of delivering a working product rather than a big idea. ‘If we do not deliver,
and deliver soon, on all the implied and stated promises, virtual reality
runs the risk of falling into the same credibility, funding and profile
nightmare as artificial intelligence – only much more quickly,’ says Brenda
Laurel, an independent consultant in entertainment software and the human-computer
interface based in Los Gatos, California.
Advertisement
DOWN TO EARTH
But what are those ‘implied and stated promises’, and how close are
VR systems to fulfilling them? The stated ones are few; the implied many,
and demanding. People who have only read about VR expect to don a headset
and be transported to another, convincing world where they can interact
naturally with a host of other people through all their senses. The truth
is more prosaic.
There are three parts to a typical VR system. First there are its visual
and aural systems, through which the user sees and hears the virtual world.
The second part is a manual control for navigating through the virtual world.
This can be a simple joystick, but more sophisticated systems provide a
glove containing position sensors for the hand or even a suit that extends
to include whole limbs or the entire body. Finally, there are the central
coordinating processors and software.
How ‘real’ a virtual world appears – and how well a user can exploit
it – depends largely on the quality of its visual system. This is because,
in humans, the eyes play the dominant role in interpreting the environment.
Each eye has about 15 million light-sensitive receptors connected to about
1 million optic nerve fibres. (In comparison, each ear has about 30 000
aural nerve fibres.) The most popular VR vision systems are headsets that
block out everything except two tiny liquid crystal display (LCD) screens,
one for each eye. By selective displacement and distortion of the images
in the left and right screens (a technique learnt from artificial intelligence
work on vision), an apparently three-dimensional view is generated.
Four principal factors determine the quality of that view. First is
the number of picture elements, or pixels, from which the image is built
up. Even the best VR display now offers only 307 200 colour pixels (640
lines by 480 rows) per eye, or about one-fiftieth of the detail of human
vision. A favourite comment among VR researchers is that even the best helmets
leave you legally blind: if the standard optician’s wallchart was shown
in a headset at the same apparent distance as in real life, only the top
row of letters would be legible.
SLOW MOTION
Stone says the ‘ideal minimum’ would be 1000 display lines, which should
let users read at least the second line of the wallchart. By 1995, says
Charles Grimsdale, managing director of Division, a Bristol-based builder
of VR applications, there will be headsets with up to a million pixels per
eye. He thinks these will cost about as much as they do now, which is around
£8500 for a top-quality model, unless a mass market for the product
can be developed.
The second factor in visual quality derives from delays in sensing and
updating the display, which originate in two separate parts of the system.
The longer these delays are, the more jerky and less convincing animation
in the scene appears.
Movements of the head (to ‘look’ at a different part of the scene) or
hand or other limb are usually detected by three electromagnetic coils in
the headset or glove. The coils are mutually perpendicular so that motion
along three axes (up/down, left/right, forward/back) can be sensed separately.
As the coils move in a magnetic field that bathes the user, tiny currents
are induced in them whose size and direction depends on the motion. This
information is digitised and passed to the computer, which uses it to update
the data it holds about the position of the headset or glove. But the process
takes time, creating a ‘latency delay’. Next, the visual processor must
work out what the updated scene looks like after the movement, and feed
that into the visual system. This causes a ‘perceptual delay’.
A combined delay of more than 300 milliseconds can cause motion sickness,
because the eyes and the balance system in the ears disagree about factors
such as gravity and the body’s motion. Researchers at the NARRC who varied
the delay found that even a lag of 120 milliseconds causes problems for
people using VR systems to operate robot arms. Delays can cause overcompensation
as the user tries to correct for movements that have already been made but
have not shown up in his or her visual field. For example, trying to catch
a moving virtual object becomes difficult because the brain is unused to
perceptible time lag in limb movements. Only combined latency and perceptual
delays below 10 milliseconds fall within the ‘perceptual threshold’ – the
time the brain takes to become aware that an event has occurred. The best
of present systems offer delays of between 60 and 70 milliseconds.
VISUAL TRICKS
The third factor in visual quality is the ‘frame rate’, which describes
how often the display is updated. The brain’s ‘persistence of vision’ retains
the impression of an image for 100 milliseconds after it has disappeared.
In the cinema, films are shown at 24 frames per second (fps), or one frame
every 42 milliseconds, which is fast enough to give the impression of continuous
motion. VR frame rates vary from about 10 fps to 60 fps, depending on the
computer’s power, with most American systems offering 25 fps. Increasing
the frame rate reduces one element of the perceptual delay, because there
are more frames per second for changes to appear in. Stone reckons the minimum
requirement for a convincing scene is about 60 fps, and ideally he would
like frame rates of around 200 fps. But he notes that the individual crystals
of LCD screens can be turned on and off only 25 times a second, which is
far too slow: ‘It needs an almost complete rethink of the display technology.
And that’s not going to come from the market yet.’
The fourth measure of visual quality is the number of ‘polygons per
second’ (pps) produced by the visual processor, indicating how detailed
the virtual world looks. Because screens composed of discrete points cannot
draw true curves or show true shading, graphics algorithms approximate curved
surfaces by breaking them into groups of adjacent polygons, each usually
consisting of three discrete points. Thus a circle could be approximated
by sets of polygons composed of adjoining isosceles triangles, all with
their apexes at the circle’s centre and their bases around its circumference.
Clever colouring of the pixels contained in each polgyon creates the impression
of light and shade – another essential of a convincing display, because
shadows provide cues for distance, depth and size.
At present, most VR equipment offers about 20 000 pps per eye. Stone
and his team are aiming for 60 000 pps per eye, and for this two Intel i860
graphics processors – among the most powerful parallel-processing microchips
on the market – will be required per eye. With polygons, the message is
‘the more the merrier’, says Stone. ‘But you need at least 10 000 pps per
eye for serious applications.’
The number of polygons visible in each frame is found by dividing the
pps figure by the frame rate. For a given number of polygons per second,
low frame rates permit a more detailed picture to be displayed. But where
a trade-off must be made between polygons per second and frame rate, higher
frame rates are generally preferred because they reduce perceptual delays.
Our ability to infer complex shapes from crude outlines – what psychologists
call ‘cognitive plasticity’ – permits even cartoon worlds of 500 polygons
to be experientially satisfying, says William Bricken, principal scientist
at the Human Interface Technology Laboratory at the University of Washington
in Seattle.
ENHANCING THE DETAIL
However, even 60 000 polygons is a long way from real life. ‘Very approximately,
it takes a hundred million polygons to simulate what we see in one scene,’
notes Bricken. If you take that as one of the implied promises of VR, it
will take a very long time to deliver – a matter of decades, in Bricken’s
view.
All in all, the computing needs of VR make it ‘a sort of computational
black hole’, says Jonathan Waldern, founder and managing director of W Industries,
a company based in Leicester that makes low-cost systems for entertainment
and commercial use. Improving the visual system also calls for dramatic
increases in raw computing power. A million-pixel screen offering 60 fps,
in which each pixel’s colour and brightness is determined by a 24-bit value
(8 bits for each of the primary colours, red, green and blue), would need
a visual processor output of 180 megabytes per second per eye – or 360 megabytes
per second for stereoscopic use. (In comparison, the text of an average
book can easily be stored in about 2 megabytes.) For a 32-bit processor,
which deals with 4 bytes of data every time it computes, this means 45 million
outputs every second for each eye. As every output must have required an
input or a calculation or both (and so at least one or more computation
cycles reading or calculating data), the processor must be capable of doing
at least 90 million instructions per second (mips). In practice, the calculations
are much more complicated. They involve non-integer arithmetic, known as
floating-point computation, and demand the processor to be capable of performing
around 1000 million floating-point operations per second (megaflops). Even
unsophisticated VR systems, which require 22 megabytes per second per eye,
demand processors able to handle about 120 megaflops, and these are expensive.
A sound system increases the demands. While our eyes collect detailed
information about an environment, the ears provide a huge amount of peripheral
data: ‘If a lion roars behind you, you don’t turn to look at it – you run,’
says Peter Williams, a virtual world designer and sound engineer with Virtual
S of London. Sound is perceived in three dimensions, and is often used unconsciously
to gauge the proximity of unseen surfaces: even with your eyes closed in
a noisy room, you can tell when you are near a wall, and whether you are
facing towards or away from it. Some researchers argue therefore that at
least 400 kilobytes of quadraphonic aural data per second – equivalent to
what a compact disc provides – is needed to make a virtual world more real.
And of course, to make it utterly convincing, sensors and feedback systems
would be needed to stimulate the millions of nerves all over the body.
Clearly, in an ideal virtual world, everything would happen in ‘real
time’, without any delays. With present technologies, the computers best
able to provide such high levels of power work in parallel. They handle
large amounts of information by distributing the calculations among many
processors working simultaneously, rather than by using one processor dealing
with each item in turn. Though these machines were formerly more difficult
and expensive to build and program than the more common sequential systems,
they are now evolving rapidly. Parallel machines also have the advantage
that they are ‘scaleable’ – doubling the number of processors doubles the
speed of computation.
Now personal computers too can model and demonstrate convincing ‘realities’.
Their growing power, which has increased performance by a factor of ten
over the past four years without a rise in price, has been enhanced by supplementary
circuit boards that handle graphics data separately. These circuits speed
up graphics displays as well as lightening the load on the central processor.
This has led to so-called ‘desktop VR’, in which the user views the reality
on a high-resolution desktop screen. A tracker ball – a ball in a socket
manipulated by hand – controls movement within the virtual world. High-definition
computer screens and tracker ball controls are together about one-tenth
the price of headsets. When the price of the software is taken into account,
desktop VR costs around £25 000, about half the price of a headset
system.
VIRTUAL CONFLICT
But this has led to arguments within the VR community about whether
the desktop version is ‘true’ VR, for which two conditions usually apply:
VR systems must be interactive, and the experience ‘immersive’. Most computer
programs are interactive, but what constitutes ‘immersion’? Grimsdale suggests
that VR should encompass the visual field, and that ‘desktop VR’ is really
a form of three-dimensional animation. A less strict view is expressed by
Ian Andrew, managing director of Dimension International, which programs
‘desktop VR’ systems. He says the only requirement is that the user should
become engrossed.
The public seems to agree with Grimsdale. A survey among visitors to
the CyberArts ’91 convention in Los Angeles who had used both types of VR
found that 84 per cent preferred the ‘headset’ version. The absence of stereo
vision makes desktop systems less satisfying as VR experiences, and satisfaction
will probably be key to the technology’s use beyond entertainment.
Manipulating dangerous materials, such as radioactive or toxic waste,
is already one application. A suggestion for another is ‘virtual meetings’
for business people, who could meet for talks without wasting time on travelling.
The present travel-free alternative, video conferencing, tends to be too
formal; business negotiators like to be able to confer and strike deals
in quiet corners as well as in front of the whole meeting. VR would allow
that. There are also applications that could help people overcome physical
disabilities; by magnifying the feedback from limb movements, a VR system
can restore movement a disabled person has lost. VPL, a VR systems manufacturer
from Redwood City in California, is studying the possibilities this technique
may have in helping recovery from nerve damage. VR psychotherapy could be
used to treat phobias by varying the intensity of the ‘threat’ while the
sufferer remains in control.
But for the next five years or so, the balance of use will be with industry
and business rather than medicine, which is unlikely to have enough money
to invest to influence the direction taken in the application of VR. This
gives some breathing space for those concerned about the psychological impact
of the technology. Might it, for instance, become the computer cocaine
– addictive, and abused by rich and poor? Bricken found some odd behaviour
among a group of about 20 people after they had used VR systems for 10 hours
or more, not necessarily in a continuous session. They would be ‘navigating
across a room by pointing, bumping into walls because they aren’t just images,
and dreaming in polygons’, he recalls. ‘VR affects dreaming strongly – it
seems to provide tools for control of the dream-life from within the dream.’
But he does not see that as unusually dangerous. ‘All intense work creates
similar effects. Anyone who has programmed all night will know that programming
slips into dreams. The problem is that we don’t have the faintest clue what
is going on.’
This could explain why VR researchers also disagree about how rapidly,
and with what regulation, VR systems should be introduced to the wider public.
Stone, like many of those researching into how people use machines, suggests
that a group of VR experts without industry ties should monitor the technology’s
development and advise how, if at all, it should impinge on people’s lives.
Others argue that research into VR’s effects on its users should happen
in parallel with technology development, but not delay it. Bob Jacobson,
co-founder of the Human Interface Laboratory and leader of WorldDesign,
a developer of VR systems also based in Seattle, says: ‘The Japanese don’t
think about that sort of thing. They just get on with developing the applications.
Then they look at what those do. We only have a finite amount of time to
prove the worth of what we’re doing. I would say let’s go for performance,
not worry about the minutiae of what we’re doing.’
David Gilmore, a lecturer in psychology at the University of Nottingham
who studies the interaction between people and computers, is less sure.
He sees no danger with simulations of real-life situations, such as driving
a car in a VR environment, but warns that when users can assume totally
different attributes, such as if doctors become able to ‘navigate’ their
way through a brain searching for a tumour, the effects are less predictable.
‘It’s not like driving a car. You don’t have a concept of where you are,
and there are potential conflicts between what you feel and see. You’re
dealing with a world without a frame of reference.’
POTENTIAL SIDE EFFECTS
The trouble is that VR technology is so young that studies on its effects
are mostly small-scale and not scientifically rigorous. For the moment,
researchers can predict its long-term effects only by reference to the effects
of other forms of computer use that are both interactive and immersive.
The clear examples are repetitive strain injury (RSI), a wrist and arm
complaint associated with the intensive use of keyboards, and addiction
to computer games. If someone has to use a VR system for their job, might
it have similar adverse effects?
Gilmore quotes studies on the incidence of discomfort experienced by
people who use personal computers. Surprisingly, those who played computer
games intensively suffered less than heavy users of commercial software.
He says that computer games manage to avoid causing RSI not because they
are better designed than business programs, but because people choose to
play them. ‘The people who suffer most are those who have the least control
over their jobs.’ If VR becomes a tool that people must use at work, then
there are potential problems.
But what about those who become too deeply attached to virtual realities?
Might it change their thinking? Between 1989 and 1990, Iain Brown, a senior
lecturer in the psychology department at the University of Glasgow, studied
addiction to computer games among the city’s secondary school children,
and his findings have worrying implications for a world in which VR is widely
available. Among players of arcade games, the study found distortions in
thinking and behaviour typical of drug addicts. ‘It dictates what they will
do, where they go, and what they spend their money on,’ Brown says.
COMPUTER SIBLINGS
But not all researchers in the field offer such dire analyses. In 1989,
Margaret Shotton, a lecturer in psychology at Nottingham University, completed
a four-year study of computer ‘dependency’ (a term she preferred to ‘addiction’,
with its implication of substance intake) of 106 people who spent an average
of 40 hours a week using a computer, with some sittings lasting nearly 12
hours. More than three-quarters of them were over 21 and nearly a half had
degrees. The difference that emerged between them and a control group of
nondependent computer users was a preference for a life centred around objects
rather than people. ‘The effects were not as dire as suggested in the literature
and the popular press,’ she concluded. ‘In all generations, there have no
doubt been object-centred, shy people who have turned away from human relationships
. . . and have resorted to solitary activities to find satisfaction.’ She
found them more intelligent, imaginative and independent than the general
average.
Brown admits that Shotton’s findings contradict his. Even so, he suggests
that television has made children more likely to become addicted to arcade
games and, in the future, VR. ‘Parents use TV as a child minder,’ he says.
‘It becomes a figure of care to the child, a substitute parent – they get
more stimulation and entertainment from it than through their parents. It
softens them up and makes them ready to transfer their affection from the
TV to the computer game. As the attachment deepens, the game becomes more
than just a substitute parent. It’s also a substitute sibling, a substitute
everything. And once they’re properly softened up by the computer at home,
they’re ready to add a thrill to it by putting their money in at an arcade.’
Though Brown’s hypothesis is untested, try reading the sentences above
with ‘virtual reality’ in place of ‘TV’ or ‘game’. Do they sound plausible?
Or worrying? Or both? For those who use VR without understanding it, simply
for pleasure rather than work, Brown predicts they will find ‘life becomes
oversimplified, with massive deskilling and impoverishment of feeling, into
simple emotions – either great success or great failure’.
To those now stumbling around in a headset, legally blind, this is only
the beginning. The relentless fall in computing prices suggests that high-quality
VR systems will be found in the home before the end of the decade. As Gilmore
comments: ‘The striking thing with new technology is that the things people
predict in advance of these changes never happens . . . VR will probably
be most useful for things we haven’t thought about applying it to.’ Perhaps
now is the time to start thinking.
Charles Arthur is a freelance journalist who specialises in writing
about social issues and technology.