Read more: Engines of the future: What鈥檚 next in internet search?
Searching with pictures, not words, turns a cameraphone into a scanner that turns the world around you into information
Want to find a picture online? Photo-sharing website ImageShack has more than 20 billion of them. Facebook is gaining over 2.5 billion new ones every month. How do you sift through them all?
Advertisement
Right now most search engines simply look at text tags or captions associated with each image. That doesn鈥檛 help much if you are looking for images of things whose name you don鈥檛 know, or that are difficult to characterise in just a few words 鈥� a design of wallpaper or jewellery, for example.
A more sophisticated approach might be to pick out key objects in a photo and construct individual digital representations of them. These could then be matched to objects in other images, enabling you to locate what you want not by entering words but by submitting a photo of the object itself or something related. But there is one huge snag. According to Paolo Pirjanian, founder of , based in Pasadena, California, this would require a huge amount of computer power.
is to extract only key features of an object, creating a 鈥渟ignature鈥� for it which can then be matched to signatures in a library of reference images. The signatures are constructed in such a way that objects can be matched even if viewed from different angles or if partly obscured, says Pirjanian. This takes far less processing power 鈥� so little, in fact, that a cellphone can be used to extract a signature. Pirjanian鈥檚 software already powers a visual search engine that is used on Japanese cellphones. Google鈥檚 own image search software, called Goggles, released in December 2009, uses a similar approach. However, it sends entire images to a remote server for matching.
Ultimately the effectiveness of this kind of visual search depends on the size of the reference library it uses. The Israeli start-up Superfish, for instance, is aimed at shoppers so it restricts its search to images of products from a hundred or so online retailers. Evolution Robotics matches images against 100,000 or so images. Meanwhile, Goggles uses a database of more than a billion pictures.
Goggles is based on software originally developed for face recognition, but to assuage privacy concerns Google has disabled the ability to do face-matching, says head of engineering Vic Gundotra. If the feature is ever turned on, it could spawn a raft of social networking applications that will ensure you never again forget a face.
Searching out loud
Why type a search query when you can just speak it? A quarter of web searches made from cellphones equipped with Google鈥檚 Android software already rely on voice input. Convenience aside, there is a potential fringe benefit to spoken web search. Every search helps refine the underlying speech recognition database, improving performance in noisy conditions, such as on the train, and its ability to handle different accents. Since Google鈥檚 Translate service uses the same database, every voice search should ultimately help you speak better French, too.
