快猫短视频

THERE鈥橲 a great tune lodged in your head. It鈥檚 maddeningly familiar but you
can鈥檛 name the song. How are you going to find it? Short of humming it to a
friend who鈥檚 a music guru, you鈥檙e stuck. But don鈥檛 fret. Soon you鈥檒l be able to
retrieve it from the Web simply by whistling it to your computer.

Think for a moment about doing that with today鈥檚 technology. You鈥檇 open your
favourite search engine鈥�and then what? Without the title, the name of the
artist or a snatch of the lyrics, a text search is useless.

But not for long. Internet searching is leaving the dull, flat world of text
behind and entering the realms of the senses. Innovative researchers and
companies are giving search engines the eyes and ears they need to understand
pictures, music and video. When their creations hit the Web, you鈥檒l wonder how
you ever made do with mere words.

Given the breakneck pace of innovation on the Internet, it is hardly
surprising that today鈥檚 text-based search engines are becoming inadequate. Just
over a decade ago, the entire World Wide Web could be indexed on a single PC.
But information quickly ballooned to fill the available cyberspace, and
researchers had to write programs to seek and index Web pages automatically. In
the mid-1990s, the first commercial search engines crawled out of the lab and
onto the Internet. WebCrawler led the way, followed by Lycos, InfoSeek and
AltaVista. The Web rode an upward spiral of its own making鈥攎ore content
required better search engines, and better search engines fuelled content.

The spiral has reached new heights. Thanks to multimedia PCs, blink-of-an-eye
communications and plummeting prices, the Web has become a medium for listening
to music, viewing images and watching video. Much of this multimedia content is
invisible to text-based search engines.

The new search engines are beginning to open up this hidden Web. At one
level, they are doing what search engines have always done: crawl around
cyberspace finding new Web pages, analyse and index them, and log them on a
database. But instead of ploughing through acres of text looking for keywords,
they are ripping apart video and audio files and analysing their visual
structure and musical style.

The first such search engines are already out there. For example,
Comparisonics of Grass Valley, California, has a search engine that trawls the
Web for sound effects. When it finds one, it calculates the sound鈥檚
鈥渟ignature鈥濃�攁 combination of its frequency, amplitude and other attributes
the company won鈥檛 reveal鈥攁nd stores the signature on its database. When
one sound file has to be matched with another, computers compare their
signatures鈥攖he closer they are, the better the match.

This technology is the basis of Comparisonics鈥檚 website, FindSounds.com. Say
you want to spruce up your e-commerce site with the sound of a cash register,
but you don鈥檛 know where or how to record one. Just type in 鈥渃ash register鈥�, and
instead of finding pages of text with the words 鈥渃ash鈥� and/or 鈥渞egister鈥� in
them, the website returns a list of audio files that go 鈥渒a-ching!鈥�. Listen to
them, and if you want more of the same simply click on the 鈥淔ind sounds like
this one鈥� button.

At this point the search technology transcends text and enters the sensory
universe. The software finds audio files according to how they sound, not what
they鈥檙e called. That means it can locate recordings that aren鈥檛 tagged with the
word 鈥渃ash register鈥濃�攔ecordings, in other words, that would be impossible
for a regular search engine to find.

You can even start your search without using text at all. Just download
software from the FindSounds.com website, record the sort of sound you want and
use the recording to kick off a search for similar sounds.

The most popular audio category is, of course, music. While it is usually
possible to find the songs you want using text, there are occasions when you
cannot. Say you鈥檙e into Radiohead and want to find soundalike bands. How would
you go about it? And how do you track down hot new artists? Independent
musicians are increasingly turning to Internet releases to publicise their
music. 鈥淭he amount of non-commercial music is increasing rapidly because digital
recording studios are available at consumer prices. A lot of that music is
really good, but it is totally inaccessible,鈥� says Philip DesAutels, chief
technology officer at Ereo, a Colorado-based multimedia search company.

In both of these cases, what you鈥檙e searching for is a particular style of
music. But what is style? How do you capture the bebop of Charlie Parker or the
melancholy of Mahler? 鈥淪tyle is an ethereal thing, like love and happiness. It鈥檚
one of those things that鈥檚 hard to explain,鈥� says DesAutels.

Bill Birmingham and his colleagues at the University of Michigan, Ann Arbor,
are betting that some nifty mathematics called Markov modelling can do it. The
idea is to figure out the distribution of notes in a piece and their
relationships to each other. When Mahler wrote a symphony, he had twelve notes
to choose from, each of which could be played singly or in combination. That鈥檚
about 5000 possible combinations, or in mathematical terms 鈥渟tates鈥�, that the
music could occupy at any one moment. These states could also endure for various
lengths of time, from a sixty-fourth note to a crotchet or a semibreve, giving
at least seven possibilities for each combination of notes鈥攚hich means a
total of about 35,000 states.

Birmingham鈥檚 software tracks the different states in a piece of music and
analyses the probabilities of going from one state to another. This is the
essence of Markov modelling. For example, if Mahler scored a C, an E flat and a
G playing together for a semiquaver, there would be a certain probability that
the next combination of notes would be a B and a D for a crotchet. A different
composer would have a different, distinct probability of doing the same thing.
Using a technique similar to that used in speech recognition, the software
analyses music and creates a Markov model containing all the states in a piece
and the probabilities of going from one state to another.

鈥淚t turns out that the composers we have looked at鈥擝eethoven, Bach,
Mozart and so forth鈥攐ccupy very different regions of the state space,鈥�
Birmingham says. What he has found is a mathematical signature of style. He says
his software can pick out the work of a given composer from a collection of
about 100 pieces. In theory, this style signature could be used to search for
music on the Web. Locating Mozart鈥檚 music would be a breeze. Find one piece, and
then search for others in the same style. Radiohead copycats or independent gems
would also be easy to find鈥攋ust input some of the stuff you like and
search for similar styles.

But what about that tune lodged in your head? Could you just hum or whistle
part of it into a microphone and let a search engine find the song? Birmingham鈥檚
team reckons their software can do that too, by identifying frequently recurring
melodies in a piece of music. 鈥淎ll composers, at least in Western music, restate
the main themes many, many times. We search and extract the patterns that occur
frequently,鈥� Birmingham says. Their software filters the music to remove
repetitions that are not themes, but just representative of the music鈥檚
genre鈥攁 walking bass line in blues, for example. Since people can usually
remember themes, such technology could work in search engines, he says.

Commercial search companies are also working towards that goal. Fast Search
& Transfer of Oslo, Norway, is developing a tool that finds pieces of music
based on snatches you whistle, hum or tap into your PC鈥檚 microphone
(快猫短视频, 10 February, p 22).
And Audible Magic of Los Gatos, California,
has song-recognition technology that could be adapted for Internet searches.
Their software analyses how the loudness, pitch and harmony of a song vary over
time and calculates a signature for the song. At the moment, Audible Magic uses
it to monitor Internet radio stations. You can install the software and
instantly identify the song that鈥檚 playing. Music publishing companies can use
the technology to keep tabs on broadcasts so they can charge royalties.

While music researchers are grappling with style, those searching for images
face a different set of problems. How can software, when asked to retrieve
pictures of cats, for example, figure out if there鈥檚 a cat in the frame? 鈥淲e can
work out that there are certain furry objects in an image, but working out that
it鈥檚 a cat, as opposed to a dog or a tiger, is quite difficult,鈥� says Ken Wood,
a computer scientist at AT&T Laboratories in Cambridge.

Wood and his colleagues have gone some way to solving the problem with
Shoebox, a software package that searches and organises photo archives. Scaled
up, the technology could search for images on the Web, says Wood. To start a
Shoebox search, you type in a text request such as 鈥渞ed and hair鈥� or 鈥渟and and
sky鈥�. Out pop pictures of your redheaded daughter or a day at the beach.

The clever thing about Shoebox is that it works out for itself what a picture
contains, whether red hair or sand and sky. First, it breaks the picture up into
regions based on edges and colours. Then it analyses these regions for
attributes such as colour, texture and shape. Neural networks trained to
recognise images of everyday features classify the regions as tarmac, sand, sky
and so on.

If Wood wants to find all the pictures of his wife in the archive, he starts
with one image. He selects her hair as a defining feature by dragging his mouse
over that region and asks Shoebox for other photographs with similar features.
By looking for the colour and texture of her hair, the system retrieves other
photos of her. Occasionally it will retrieve unrelated images, such as one with
some wood that looks like hair. But that鈥檚 not a big deal. After all, we rarely
get mad when a text search throws up irrelevant information鈥攚e simply
refine our query.

Search company Ereo has an idea how such technology might be useful. Say you
visit an online furniture store looking for a chair. You type in 鈥渃hair鈥�, and
you get images of all the chairs in stock. You quite like the beige recliner,
but it鈥檚 not exactly what you鈥檙e looking for. So you click on a button marked
鈥淢ore like this鈥� and twenty similar recliners pop up. Just as with
Findsounds.com, this is the point where the search leaves text behind. The
software uses the shape of the chair to search for other images with similar
features. Such technology could soon become standard fare鈥擡xcite has
licensed Ereo鈥檚 鈥淢ore like this鈥� for its search engines.

The final frontier for multimedia searching is video. But it isn鈥檛 likely to
happen just yet. Video quality on the Internet is still pretty miserable, which
makes extracting information for indexing quite difficult. But all that could
change with new encoding and compression standards for digital video鈥攕uch
as DivX, sometimes called the MP3 of the video world.

Engineers think video searching will involve a combination of non-text tools.
AT&T Laboratories has developed a system that quite literally dissects
digital video, looks at its innards and indexes it. The system, called
AT&TV, monitors television broadcasts from Britain鈥檚 five terrestrial
channels, but Wood says it could be adapted for searching on the Web.

The system works by chopping video into segments based on abrupt scene
changes, cuts, fades or transitions in the soundtrack from speech to music or
vice versa. It then identifies key frames within each segment, analysing each
frame as a still image and extracting visual information, just as Shoebox does
with photographs. The software also analyses the soundtrack, subtitles and other
information tagged onto the video. All the information is indexed to create a
searchable archive. 鈥淪imply type the words 鈥楾ony Blair鈥� and that would almost
certainly get you a segment where Tony Blair is speaking,鈥� says Wood. 鈥淭hen you
could say 鈥榝ind me other clips with a similar audio fingerprint鈥�.鈥�

Similar video analysis engines from Excalibur Technologies of Vienna,
Virginia, and Virage of San Mateo, California, are being used by organisations
such as CNN, Reuters and NASA to index and search their large video archives.
And search company Inktomi of Foster City, California, is planning to add
Virage鈥檚 technology to its search engines.

How far can search engines delve into the realms of the senses? Web companies
are developing technology that digitises the essence of a smell and transmits it
over the Internet. Devices equipped with volatile chemicals convert the digital
data back into smells, just as speakers turn audio signals into sound. When such
data is commonplace on the Web, search engines that sniff won鈥檛 be far behind.
So don鈥檛 be surprised if a few years from now your computer turns its nose up at
the smell in your room, searches the Internet and downloads itself some air
freshener.