NAME this song: itâs in A major, with a fast, driving beat. The singing and lyrics give it an anxious edge and acoustic guitar makes the whole thing sound âfolkyâ. Got it yet? Itâs by the Beatles. Still no luck? Thatâs hardly surprising, yet if Iâd hummed a few bars to you, youâd probably have got it easily. Itâs Help!
While our brains can instantly recognise a tune or song when we hear it, trying to identify the same piece from a written description usually leaves us stumped. What if there was a way to create a precise, objective breakdown of any tune â an assay of all the different characteristics you would need to identify a particular song, rather like the way the DNA in each gene defines the exact structure of the corresponding protein?
This problem is not as abstract as it might seem. Right now there are more than 25 million digital music tracks available online, which is about twice as much music as you could listen to in your lifetime. Thousands more tracks are being added every day, and the latest digital music players have space for up to 15,000 songs â about five weeksâ worth of music. It is hardly surprising that finding new music youâll like, or trawling your own collection for just the right track, can be exasperatingly difficult. âThe more music you have, the more time you have to spend choosing what you want, and the less you use it,â says Matthew Dunn, CEO of MusicIP. âYet thereâs no Google for music right now. Searching for text about music isnât the same as searching for the music itself.â
Advertisement
MusicIP, based in Monrovia, California, is one of a growing number of companies and university research teams racing to develop software that can perform a complete acoustic analysis of any tune it hears. The aim is to distil not just key or tempo, but dozens of different characteristics, including the timbre of the sounds, chord progressions, the individual instruments and even details about each singerâs voice.
Attach this information to every digital music file and you suddenly open up the huge online universe of tunes like never before. You could search not just by title or artist but by almost any musical characteristic (although so far the âsearch termsâ you enter will be example tracks rather than actual words). And if software did the searching for you, it could become a personal digital DJ, capable of analysing your taste in music, creating perfect playlists, and even combing the internet to find new stuff that it knows youâll love.
There is already plenty of software that can organise your music by genre or artist, based on the text information attached to a digital music file, and which can make recommendations based on that information. However, those details arenât always there, or arenât always accurate. With a typical digital music collection, deciding what will suit your mood or choosing a playlist can easily become a chore. âIt takes about 1 minute to select a song from an average music collection,â says Fabio Vignoli, an electronics engineer at the Philips Research Laboratory in Eindhoven, the Netherlands. âTo get 2 hours of music you must select 20 songs, and that takes 20 minutes. Eventually you will not bother.â
Machines with ears
So how about taking the human out of the loop and letting the computer do all the hard work, using software that can perform an audio analysis of each track?
Making sense of the complex, overlapping patterns of sound in music is a difficult processing task. A mathematical algorithm called a Fourier transform can help, converting a signal into a mix of sine waves with different amplitudes and frequencies. A wave with a frequency of 440 hertz, for example, is the A above middle C. However, real instruments donât produce pure tones â they create additional frequencies called overtones. While they complicate matters, these overtones can provide an idea of what instrument is playing a note. Algorithms can also look at patterns of notes to try to figure out basic characteristics of melody, harmony and tempo. Combined, this could give the genre of a piece, and allow you to compare different tracks and group them according to their characteristics.
One of the most sophisticated consumer products aiming to do this is MusicIP Mixer, developed by MusicIP. The software can analyse the music in your personal library and then draw up playlists based on any song you care to choose â recommending other tracks that sound like the song youâve selected.
When I download MusicIP Mixer onto my computer, it grinds away, analysing my collection. First it looks for title or artist tags and compares each song with MusicIPâs online database of 17 million tracks. Then it double-checks the identity of each song by creating a quick digital âfingerprintâ and comparing it against the fingerprints on the central server (see ).
âYou would suddenly open up the huge online universe of tunes as never beforeâ
If the software finds a piece of music that is not on its database, the processor-intensive stuff starts. MusicIP wonât reveal details, but the software essentially plays the song to perform a full acoustic analysis of the track and stores the important characteristics in a detailed acoustic profile. Itâs a relatively long process, taking roughly as long as it would to play the track. Then the software sends the profile to the central server.
Once the analysis has finished, you can select a song from your library and the program will use the acoustic profile to construct playlists of similar songs. For instance, when I highlight Squeezeâs Goodbye Girl, the program seems to pick up on the early-80s pop vibe and puts together a mix of 12 songs by Squeeze, Blondie and Prince. A playlist based on a movement of a Beethoven string quartet pulls up mostly other string quartets. It also gives me the Beatlesâ Sheâs Leaving Home, a pop song with rich string accompaniment. Some might consider this a mistake, but I think of it as a pleasant surprise.
The program also proves its chops by managing to group all of my small collection of Indian classical music together. Iâd suspect it of cheating by using genres, but the tags that came with the music files list their genres as either âworldâ or âotherâ, so the program really does seem to have recognised the common characteristics of the tracks.
Dunn says that the program works so well partly because it has a database of 17 million songs to work with, making it easier for the program to tell where a particular song fits into the music universe.
Impressive as this is, these descriptors may still not be enough to let a computer choose music that a listener wants to hear. At least, thatâs what a California-based company called Pandora has decided, and it is aiming to meet that challenge.
Like MusicIP, Pandora is a web-based music recommendation service, but it aims to classify songs in far greater detail. It assesses 400 different parameters â from the kinds of instruments played to melody, harmony and other detailed information about genre. The singerâs voice alone can be scored on up to 30 characteristics, including the level of vibrato, range, pitch and breathiness.
âThe best way to describe it is a musical description, not someoneâs opinion about how good a song is,â says Tim Westergren, Pandoraâs founder. He calls the system the âMusic Genome Projectâ: just as individuals can be identified by their different combinations of genes, so Pandora aims to distinguish any piece of music according to how it scores on this set of musical âgenesâ.
Itâs my song
On screen, Pandora works like a personalised radio station. I pick a song to base my station on, and Pandora then selects and plays similar songs. You can choose extra songs for the station to increase variety, or tell it that you donât like one of the songs it selected, which makes it recalculate its recommendations to reduce the influence of that particular track.
Pandora relies heavily on processing power, since the program has to look at the list of attributes for each song you choose and figure out which other songs are closest to it. However, the most crucial part of the process â assigning the mammoth list of attributes to each song â is left to the human ear, in this case a team of 40 trained musicians.
Currently it takes about 20 minutes to analyse one song. Yet Westergren doesnât think thereâs much hope of automating that task: âThe ear is unbelievably sophisticated. Weâre a long way from a machine being able to do anything like what a human can,â he says.
This sophistication is obvious in the descriptions you see when you click on a song. When I ask Pandora why it played a song by DJ Shadow, for instance, it tells me âit features electronica roots, use of tonal harmonies, use of electric piano, subtle electric piano riffs and prevalent use of grooveâ. Maybe some day a computer program will be able to detect âgrooveâ, but I doubt any can yet.
âMaybe someday a computer will be able to detect âgrooveâ but I doubt any can yetâ
Mark Sandler, an electronics engineer who runs the Centre for Digital Music at Queen Mary, University of London, agrees that computers canât yet analyse music as well as humans. He says the problems are similar to those faced by researchers working on computer vision and voice recognition. The human brain has been shaped by millions of years of evolution to be good at those things. Teaching computers to catch up wonât be easy.
Nevertheless, Sandler has little doubt that computerisation is essential if we are to analyse all the music thatâs out there, simply because of the size of the task. âA well-trained human being will do better than a computer, but that solution doesnât scale.â
Not surprisingly, Westergren disagrees. Right now Pandora has licences to include 400,000 songs in its database, and is adding about 8000 songs every month. It is only a fraction of what is available, but Westergren doesnât think people mind, as long as the collection is big enough to always turn up music they like.
Despite Westergrenâs pessimism about computerised music analysis, most researchers see Pandora as a model worth aiming for. They just want to reproduce the system without human involvement. âItâs a really hot topic. Many companies are looking into it,â says Xavier Serra, director of the Music Technology Group at Pompeu Fabra University (UPF) in Barcelona, Spain.
UPF is part of a European-funded consortium called SIMAC (Semantic Interaction with Music Audio Contents), a basic research project that takes much the same approach to analysis as MusicIP. The hope is to find more effective ways to pick out complex attributes like tempo, rhythm and style and to measure the way combinations of notes or chords produce harmonies, by combining human cognition studies with music analysis and artificial intelligence software. âWhat we can do now is enough to be able to develop music recommendation or navigator systems for large music collections,â Serra says. âThe goal is to be able to have as complete a description of a piece of music as a human being can do.â
Music identification software looks likely to make a big impression on portable devices such as MP3 players on mobile phones. It could also be used by the music industry itself, to spot songs being exchanged on file-sharing networks. If they can monitor and identify what tracks are being swapped, they could in theory find ways to charge for them or at least control who has access. There are also concerns about privacy. Early this year it was revealed that software on Appleâs iTunes music service could send details of a userâs playlist back to Apple. Internet forums were flooded with warnings that this data could be used to help target advertising, but Apple has said that the company does not collect any personal information on iTunes users.
Despite such worries, we could all soon be hooked on this sort of software â and not just for finding new music. If a program can match your taste in tunes, then why not turn it loose to create âplaylistsâ of novels, pictures and videos too. MusicIP has even developed software that uses information on your lifestyle, diet and tastes to come up with meal suggestions, recipes and even a shopping list of ingredients. In fact, everything youâd need for a perfect dinner â to be accompanied, of course, by a selection of your favourite music.
