Google has trained an artificial intelligence, named SingSong, that can generate a musical backing track to accompany people’s recorded singing.
To develop it, Jesse Engel and his colleagues at Google Research used an algorithm to separate the instrumental and vocal parts from 46,000 hours of music and then fine-tuned an existing AI model – also created by Google Research, but for generating – on those pairs of recordings.
After that training, the adapted AI was able to accept new vocal recordings that weren’t in the training data and create a backing track for them.
Advertisement
To judge its abilities, the researchers got people to listen to 800 pairs of song clips, each 10 seconds long. In each pair of clips, one had a vocal that had been given a backing track by SingSong, while the other had the same vocal with either an existing human-generated backing track that matched the beat and key as closely as possible, the original backing track for the vocal or an entirely random backing track.
In 66 per cent of the trials, participants preferred the SingSong version when asked which clip had the backing track that was more “musically compatible with the vocals”. You can .
at Goldsmiths, University of London says the output of the AI is convincing in terms of key, melody and harmony.
“I wouldn’t say you’d release it as a finished track. It would need development, but it seems to be working in terms of coherence, for sure. The grooves it’s suggesting, they work and they’re interesting. And unexpected, some of them, in a good way that feels like working with another musician,” she says. “It’s exciting and I could certainly see it as part of a process to suggest ideas really quickly. It’s not going to take over quite yet, but I think it’s going to be facilitative for a lot of musicians.”
The results of SingSong are impressive and convincing in most cases, says at the University of Oxford, but he cautions that the technology has the potential to disrupt music production.
“I have some misgivings about what this will mean culturally for us generally, as well as the implications for those musicians whose skill and musical ideas are being mimicked through the SingSong’s trawl of the internet,” he says.
The Google team didn’t respond to an interview request.
The AI might not work for everybody yet, though. The team writes that SingSong was less able to generate good backing tracks when the vocals were recorded by amateur singers because the tempo and pitch varied too much. The group suggests that in future versions, people could sing along to a metronome and automatic pitch correction could be applied to their recordings.
Reference:
