快猫短视频

Let there be light

SIGNOR Alfredo Pergolizzi brushed a speck of dust from the lapel of his
tailored Italian suit and clasped his hands in front of him. Leaning across the
desk in his Vatican office, he smiled expansively at the young film maker and
said, 鈥淪ignor Debevec, I am pleased to inform you that we鈥檝e decided to allow
you one hour to take your pictures in St Peter鈥檚 basilica.鈥 Inwardly, Paul
Debevec groaned. He had hoped to spend three days capturing the majestic
structure in digital pictures for his computer graphics research at the
University of California, Berkeley. But after hours of negotiating, this was
Signor Pergolizzi鈥檚 final offer, so Debevec could only smile and accept. And
start planning how best to use the time.

Debevec, along with graduate student Tim Hawkins, had gone to Rome to use the
Basilica as a set for an experimental film called Fiat Lux鈥擫atin
for 鈥渓et there be light鈥. The final product would be a powerful two and a half
minute animated film set in the famous building depicting a symbolic struggle
between science and the church. Debevec planned to show silver balls crashing
chaotically onto the basilica鈥檚 marble floors and onyx dominoes toppling in
thunderous succession.

If he could pull it off, Fiat Lux would be more than just visually
impressive. It would be a showpiece for the revolutionary computer modelling and
lighting techniques that he and his colleagues had developed over the past three
years. The techniques require only a few digital snapshots of a scene to
construct a virtual film set in which a virtual camera can shoot from any angle,
zoom in and out, and pan and track around the scene. The characters, in this
case silver balls and dominoes, are shot or animated separately and added later,
but appear as if lit from within the scene. 鈥淲ith Debevec鈥檚 techniques, virtual
models of real-life places could seem more real than ever,鈥 says Rob
Shakespeare, a professor of theatre lighting design at Indiana University in
Bloomington.

Since almost any site on Earth can be captured quickly and cheaply in three
dimensions this way, the technique has the potential to revolutionise the way
films are made. What鈥檚 more, it can be used to conjure up models for other
uses鈥攖o help preserve historical sites, for example, or to make
architectural models of buildings. And with easy-to-create virtual film sets
comes the potential for a whole new form of three-dimensional storytelling.

Setting a high-quality film in any public space is fraught with difficulties.
Filming can take days or even months, but getting permission to use the site for
even a few minutes can be difficult, as Debevec discovered. The alternative, at
least for well-funded Hollywood directors, is to build a film set in which the
actors and actresses perform. But again, this is a time-consuming and expensive
business. Another option is to build a virtual model of the scene on computer,
film the action separately and add it later. But making a virtual model look
real has proved almost impossible鈥攚hich is why Hollywood has generally
used the technique only for films set in fictional places.

There are two problems that must be overcome to make a virtual film set look
real. First, the computer modeller has to create a three-dimensional model that
is identical to the original in every way. The shape of every cornice, the
texture of every surface and the details in every carving all have to be
captured accurately. If this sounds tough, it is child鈥檚 play compared to the
other problem: reproducing lighting accurately. In an indoor scene, this is made
up of direct light from sources such as the Sun shining through windows, light
bulbs and indirect light reflected from every surface. Lighting is the key to
how we perceive reality, and thus whether something looks real, says Donald
Greenberg, a pioneer in computer lighting simulations at Cornell University.

One way to make a model is to build it from scratch using a computer software
package that provides building blocks in the form of boxes, spheres and other 3D
shapes. The blocks are assembled like virtual Lego into a shape resembling the
structure being copied. This technique, known as 鈥済eometry-based modelling鈥,
requires the painstaking efforts of a skilled graphic artist to transform the
building blocks into a realistic model.

Instead, a growing number of researchers are turning to image-based methods
to create models. Rather than starting with geometric blocks, they start with
digitised photographs. Part of the breakthrough that Debevec and his colleagues
have achieved is to make this process much easier and the results more
lifelike.

Debevec begins with a single photograph of a scene which he feeds into a
computer program called Fa莽ade. With a mouse or electronic pen, he marks
out the boundaries of the walls, roofs and other large-scale structures in the
picture. The result is a crude outline of the buildings in the scene. He then
selects 3D shapes such as cubes, prisms and curved shapes that roughly match the
shape of these structures and asks the computer to orient and size them until
they fit the outlines in the scene. For example, Debevec might choose a prism to
model the structure of a roof. The computer then sizes and orients the prism
until it fits the perspective shown on the screen. At the end of this process,
the computer has a 3D skeleton of the structures in the scene. Already, it is
possible to fly around this virtual world to see how buildings stand in relation
to one another and even to look at the skeletons from behind.

The next stage is to add structures such as doors, windows and even friezes
and cornices. Debevec does this using another program that compares the 3D
skeleton with the original photograph, and picks out extra details.

The final stage is to 鈥減aint鈥 the skeletons to make them appear solid.
Debevec uses a method akin to projecting photographic images with a slide
projector onto the sides of the model. For this he uses several photographs
taken from different angles so that he can project details that were hidden in
the original shot, such as the view of the back of a building. Now a viewer can
fly around the model and see a realistic view from almost any angle.

Debevec and his colleagues demonstrated these techniques a couple of years
ago using the famous clock tower at the University of California, Berkeley, as a
subject. In the film, the camera circles around and over the clock tower as if
flying in a helicopter. In the background are other campus buildings which all
have a 3D structure. Even private houses near the campus appear in the
model.

Debevec鈥檚 technique solves the problem of trying to model a complex geometry
and then make it look real, says Michael Cohen, a computer scientist who
develops image-based modelling techniques at Microsoft Research in Seattle,
Washington. 鈥淒ebevec builds up models with blocks and then paints them with
辫丑辞迟辞驳谤补辫丑蝉.鈥

The technique works well for creating a 3D snapshot of a scene, but a virtual
film set is more complex. A director will want to insert objects into the scene
in a way that makes them look realistic. Light must reflect off them or pass
through them in a realistic way and their shadows must fall as if they were
really there. The director might also want to add special lighting effects to
create a particular atmosphere.

The actual path of light is easy to model鈥攁 simple ray-tracing program
does the trick, showing which objects are directly illuminated and which fall in
shadow. The difficult bit is understanding how the light reflects off objects in
the scene and provides indirect lighting for other objects. Indirect lighting is
subtle but extremely important in making a scene look realistic. Without it,
colours look flat and the scene seems sanitised and, well,
computer-generated.

The first stage in modelling indirect lighting is to measure it. The idea is
to measure the light field at a single point in space鈥攖he light passing
through that point from all directions. This is easier than it sounds and
Debevec uses a well-known method in which he places a perfect mirrored ball the
size of a grapefruit a few centimetres in front of the camera and photographs
it. Since the ball reflects light coming from all directions (except directly
behind it) into the camera, Debevec gets a pretty good idea of the light field
at that point.

Of course, it is impossible to capture the entire range of luminance in a
single photograph, as anyone knows who has ever photographed an unlit person
standing in front of a bright window. In any scene, whether indoors or outdoors,
the amount of light can vary by many orders of magnitude. 鈥淚n St Peter鈥檚, if you
compare the underside of the dark bronze of Bernini鈥檚 baldachino [the canopy
over the altar] to the light coming from the windows and from the spotlights in
the vaulting, the ratio is well over a million to one,鈥 says Debevec. 鈥淭his is
not atypical of rooms with bare light sources and actually less range than one
would get outside, where the ratio between looking directly at the Sun and
looking into a dark crevice in the bark of a shaded tree is close to 100
尘颈濒濒颈辞苍.鈥

To capture this range, Debevec takes a series of digital photographs of the
mirrored sphere and varies the exposure in each, first capturing the shadow
detail, then the midtones and finally the highlights. Unfortunately, digital
cameras do not generate images in which the pixel brightness is proportional to
the light in that part of the scene (this is to make their images look more like
the ones people expect from photographic film). So Debevec has had to develop
software that combines the series of images in a way that takes this into
account. The result is a complete record of the light field at a single point
across the entire range of brightness levels. With this measurement, Debevec can
work out the light field at any other point in the scene, providing it isn鈥檛
hidden from the camera.

This is highly significant. With the ability to work out the light field at
every point, Debevec can insert an object anywhere in the scene and know how the
indirect illumination would make it appear from any angle. This is the key
technique that allows a director to insert props and actors into a scene.

After perfecting these techniques, Debevec鈥檚 only remaining task was to
create a showcase for them. In Fiat Lux, he wanted to make a movie that
was both visually beautiful and displayed the capabilities of these techniques.
He chose St Peter鈥檚 because he knew the complicated interior geometry and
different types of light emanating from stained glass windows and overhead lamps
would test his techniques to the limit. And the film鈥檚 title not only explains
the goal of the film but is also the motto of the University of California.

The morning after meeting Signor Pergolizzi, Debevec and Hawkins lugged their
cameras, tripods and mirrored balls into the basilica. The ancient church was
begun in about AD 326 and rebuilt by Renaissance architects in the 16th century.
To capture its complex geometry, the two researchers would need to take pictures
from several viewpoints so that all angles were covered. Debevec and Hawkins had
laid out their plan of action the previous night, using a map to decide on
camera locations. Now, on this cool February morning, they worked quickly,
taking all the shots they needed within the allotted time. The only other people
in the basilica were three nuns and a janitor. No matter, thought Debevec. These
bystanders could be eliminated later with a graphics program.

Back in Berkeley a few weeks later, Debevec enlisted the help of a team of
undergraduates to work on the project. Using the new techniques, they gradually
pieced together the model of St Peter鈥檚 using just 10 photographs.

Next came the task of adding the synthetic objects鈥攖he mirrored spheres
and solemn black dominoes. To make the action look real, the surface of each
sphere and domino had to display reflections from the salmon-coloured marble
walls and streaming light that filters down from the stained glass windows. As
the objects move, these reflections had to move as well.

Debevec also used techniques to give Fiat Lux a cinematic quality.
He used slow motion, added soft lighting, varied the camera angles, panned and
moved around the room and used a soundtrack of Gregorian chants and crackling
thunder.

Because of this attention to detail, it took a hundred 167-megahertz
computers and 20 Pentium IIs working full-time for three days to render all the
4500 frames in the animation. When complete, Debevec had succeeded in creating
exactly what he set out to make: a concise, authentic advertisement for the
growing power of image-based techniques to build highly realistic models.

These techniques promise a new level of realism for Hollywood film
makers鈥攂ut there are challenges ahead. Debevec can skilfully mimic the
play of light on artificial objects such as spheres and dominoes because it is
easy to measure and model the way light reflects off their surfaces. However, it
is far more difficult to model the way light interacts with human skin or
clothing. 鈥淩ight now there is no interaction in terms of lighting between actors
and the synthetic environments we put them in,鈥 says Shakespeare.

Debevec is working on a way to capture the way light interacts with the human
form. There is no one-size-fits-all approach. The lighting depends on their
shape, how fat or thin they are, how they move, what colours and even what
fabrics they are wearing. Debevec plans to film actors in a studio under a
sequence of different lighting conditions to determine how each reflects light.
With this information it should be easier to add them to the virtual film set
later.

But film directors will not be the only people interested in Debevec鈥檚 new
techniques. Highly realistic virtual sets can also be used to preserve sensitive
environments in virtual archives so that scientists can see how they are
changing and so that people can enjoy them in the future鈥攚hen the real
thing may no longer exist. Two weeks after the Los Angeles premiere of Fiat
Lux in August at the annual computer graphics jamboree known as Siggraph
99, Debevec was on a plane to Egypt as a consultant on a project that will make
computer models of the pyramids and other antiquities.

With their similarity to the real thing, these virtual landscapes may be the
first step towards of a whole new entertainment experience. One could travel the
world without leaving one鈥檚 living room, says Debevec. And while it may never be
possible to reproduce all the delights of foreign travel, there will be some
advantages. After all, if one travels to the Vatican on a virtual trip, it won鈥檛
be necessary to get permission from Signor Pergolizzi to take pictures.

  • Further reading:
    find out more about Paul Debevec鈥檚 film Fiat Lux at http://fiatlux.berkeley.edu

More from 快猫短视频

Explore the latest news, articles and features