COMPUTER graphics artists face a tough job when they have to combine their hobbits, kung-fu fighters and spacecraft with live action footage for movies. For fantasy creations to look realistic, their appearance on screen has to react to changes in the colour, intensity and direction of the light in different parts of the film set.
This can be an expensive and time-consuming task, but now two scientists in New York have devised a way of simplifying it.
The conventional technique is to go back onto the set and take photographs of reflecting balls placed in various parts of the set. Analysing the reflections on the balls in photographs taken from a variety of angles shows what the lighting is like in that part of the set. This information can then be used to modify the computer generated images (CGI).
Advertisement
But now computer scientists Shree Nayar and Ko Nishino of Columbia University have found a better way to sample the light. They have developed software that can extract detailed lighting information from the reflection on an actor鈥檚 eyeball.
Their software, called VisualEyes, scans images of faces for the limbus 鈥� the boundary between the white part of the eye and the cornea. When the actor is looking straight into the camera, the image of the limbus is perfectly circular, but if the eyeball is turned away, the image becomes elliptical (see Graphic). From its size and shape, the software can determine where an actor is standing relative to the camera, and what direction the actor is looking in.
Using reflections from the eyeball, the program can calculate all the necessary information about the lighting in that part of the set. 鈥淎ll you need is one good frame and the information is there,鈥� says Nayar. The technique doesn鈥檛 even require a complete scene to be visible in the actor鈥檚 eye. Even if the actor is far from the camera and only a few pixels cover their eye the software can calculate the overall light intensity in the scene. 鈥淚t doesn鈥檛 have to be precise,鈥� he says.
Dottie Starling, lighting supervisor at Cinesite in London, which recently created digital effects for King Arthur, Harry Potter and the Prisoner of Azkaban and Troy, says Nayar and Nishino鈥檚 technology could be a real asset. 鈥淚f we could capture lighting from something on the film, instead of taking extra time on set, it would save us a lot effort,鈥� she says.
There are some circumstances in which the technology can fall down. In distance shots, for instance, the image of the actor鈥檚 eye may be too small to convey useful information. And the reflections do not always provide the colour information that is needed, though the researchers are working on ways to improve this by correcting for the colour of different people鈥檚 eyes.
For some tasks, corneal images might be the only option. For instance, if film makers want to revamp old movies by adding new CGI objects or backgrounds, as George Lucas did in 1997 with digitally enhanced versions of the original Star Wars trilogy, the only source of information on the original set lighting would be embedded in the characters鈥� eyes.
Nayar says his software could be used to extract information from archive photographs or films, such as the location they were shot in, or what that subject was looking at. 鈥淵ou could see not only their reaction, but what they are reacting to,鈥� he says.
The researchers will be presenting their technology at the Siggraph computer graphics conference in Los Angeles this summer.
