THE signs of the digital photography revolution are hard to miss, from cameras embedded in our cellphones to gigabytes of images stored on hard drives. But if you thought the revolution finished with the death of chemical film, think again. Computational photography promises equally dramatic changes, turning even the most ham-fisted of snappers into veritable Cartier-Bressons.
We are on the cusp of a new era in which every camera comes with a sophisticated built-in computer, says of the Massachusetts Institute of Technology, who delivered a at an in Monterey, California, this week. Low-cost processing and memory combined with new digital sensors will deliver richer images created by fusing elements from multiple shots and even video.
Advertisement
Hints of the changes to come can be found in cameras such as Casio鈥檚 , which launched last year and has been dubbed the . In poor light, photographers face a difficult choice: use a flash, which can produce a harsh illumination, or go for a long exposure, where the risk of image blur increases. The EX-F1 offers a third option. It shoots a burst of images at long exposures and its computer merges the shots into a single image, reducing the blur as it does so. The process may not yet outperform established anti-blur techniques, such as using a tripod, but its existence is a significant advance in itself.
In labs around the world, researchers are developing a slew of other computational tricks for cameras. 鈥淲e鈥檙e creating images that people have never been able to produce,鈥� says at Stanford University in California.
Many of the new techniques tackle the old problem of capturing a fleeting moment. Imagine watching a kingfisher arrowing towards a lake surface. It takes a lot of patience, skill 鈥� and luck 鈥� to capture the precise moment at which the bird breaks the water鈥檚 surface. Using an everyday digital camera, it is possible to switch to video mode to record the action and subsequently extract the right frame. But video鈥檚 resolution pales in comparison to still photography, so the resulting images are low quality.
at Microsoft Research in Redmond, Washington, and colleagues think they have found a way round this problem. Their system captures a video stream and, every second or so, takes a high-resolution still image of the same scene. After the event, users can review the video and select a frame. The software then uses information from the stills taken immediately before and after that point to enhance the resolution of the video frame.
Cohen鈥檚 software is computer-based at the moment, but he says it could be incorporated into a camera. One way that photographers might use it is by continuously of a scene and only hitting the shutter when they see something of particular interest. The camera would then automatically create a series of stills from the frames captured in the half-second or so before the shutter was pressed, hopefully capturing that crucial moment.
Even cheap cellphone cameras could benefit from the advances. High-quality cameras can capture part of a scene in sharp focus while leaving the background blurred, by adjusting the camera鈥檚 aperture. Now have found a way of simulating the attractive background blur effect, known as 鈥渂okeh鈥�, using a cheap lens.
In their , small motors move the lens and camera sensor in the same direction but at different speeds as the image is captured. This blurs the image. But by moving the sensor and lens at just the right relative speed, Raskar is able to select a vertical plane in the image in which focus is preserved. He says he is now talking to camera and cellphone manufacturers about licensing the technology.
Another feature cameras might one day include is panorama software, which could create 360-degree cylindrical panoramas by stitching together a series of shots, adjusting for slight tilts. A technique known as 鈥渁ll-focus imaging鈥� could be used to create high-resolution close-ups in which, unlike current macro photographs, all of the image is in focus. The technique involves taking a series of macro images in which different layers of the scene are in focus, and then merging them to create a single image. , senior research scientist at Adobe Systems in Seattle, has already developed an all-focus algorithm that has been incorporated into Adobe鈥檚 image-editing package Photoshop.
Alternatively, when half of a scene is in shadow, a camera could automatically take two shots at different exposures and merge them to create an image in which all parts of the scene appear well lit.
鈥淭he distinction between a cellphone and a computer has already gone,鈥� says Cohen. 鈥淣ow the distinction between a camera and a computer is going away.鈥�
鈥淎s has happened with cellphones, the distinction between a camera and a computer is going away鈥�
If there is a note of caution, it comes from camera manufacturers. Many digital cameras already come with basic computational features, such as the ability to lighten areas of the shot that are in shade, and it is not yet clear, at least to the manufacturers, what extra functions users want.
James Banfield, a spokesman for Nikon UK, says that his firm will listen to consumers and add more features if the demand is there. But he points out that the features need to be robust and easy to understand. Some aspects of camera interfaces are already complex, Banfield says, so manufacturers have to think hard before adding further features.
That caution has prompted Levoy to develop his own . Compared with commercial devices, Levoy describes his 鈥淔rankencamera鈥� as 鈥渦gly, big and heavy鈥�. But it is portable and battery-powered, so researchers can take it out of the lab and experiment with it. The device includes a Canon lens, a light sensor from a Nokia mobile and runs the Linux operating system. All of its functions, including metering, focusing, demosaicing, de-noising and white balancing, can be controlled using the camera鈥檚 open-source software, which will be released next year.
Levoy is applying for funding so that he can produce a number of the devices and give them to other research groups. Once the device is out there, the list of computational tricks could get even longer. 鈥淧eople will surprise us by coming up with stuff we haven鈥檛 even thought about,鈥� says Levoy.
