AI and music: will we be slaves to the algorithm?
Pioneers of sound (left to right): George Philip Wright, Jon Eades and Siavash Mahdavi at Abbey Road Studios, London. Photograph: Sonja Horsman for the Observer
First of all thanks to Stuart Dredge for this article.
Tech firms have developed AI that can learn how to write music. So will machines soon be composing symphonies, hit singles and bespoke soundtracks?
From Elgar to Adele, and the Beatles or Pink Floyd to Kanye West, London’s Abbey Road Studios has hosted a storied list of musical stars since opening in 1931. But the man playing a melody on the piano in the complex’s Gatehouse studio when the Observer visits isn’t one of them.
The man sitting at the keyboard where John Lennon may have finessed A Day in the Life is Siavash Mahdavi, CEO of AI Music, a British tech startup exploring the intersection of artificial intelligence and music.
His company is one of two AI firms currently taking part in Abbey Road Red, a startup incubator run by the studios that aims to forge links between new tech companies and the music industry. It’s not alone: Los Angeles-based startup accelerator Techstars Music, part-funded by major labels Sony Music and Warner Music Group, included two AI startups in its programme earlier this year: Amper Music and Popgun.
This is definitely a burgeoning sector. Other companies in the field include Jukedeck in London, Melodrive in Berlin, Humtap in San Francisco and Groov.AIin Google’s home town, Mountain View. Meanwhile, Google has its own AI music research project called Magenta, while Sony’s Computer Science Laboratories (CSL) in Paris has a similar project called Flow Machines.
Whether businesses or researchers, these teams are trying to answer the same question: can machines create music, using AI technologies like neural networks to be trained up on a catalogue of human-made music before producing their own? But these companies’ work poses another question too: if machines can create music, what does that mean for professional human musicians?
“I’ve always been fascinated by the concept that we could automate, or intelligently do, what humans think is only theirs to do. We always look at creativity as the last bastion of humanity,” says Mahdavi. However, he quickly decided not to pursue his first idea: “Could you press a button and write a symphony?”
Why not? “It’s very difficult to do, and I don’t know how useful it is. Musicians are queuing up to have their music listened to: to get signed and to get on stage. The last thing they need is for this button to exist,” he says.
The button already exists, in fact. Visit Jukedeck’s website, and you can have a song created for you simply by telling it what genre, mood, tempo, instruments and track length you want. Amper Music offers a similar service. This isn’t about trying to make a chart hit, it’s about providing “production music” to be used as the soundtrack for anything from YouTube videos to games and corporate presentations.
Once you’ve created your (for example) two-minute uplifting folk track using a ukulele at a tempo of 80 beats-per-minute, Jukedeck’s system gives it a name (“Furtive Road” in this case), then will sell you a royalty-free licence to use it for $0.99 if you’re an individual or small business, or $21.99 if you’re a larger company. You can buy the copyright to own the track outright for $199.
“A couple of years ago, AI wasn’t at the stage where it could write a piece of music good enough for anyone. Now it’s good enough for some use cases,” says Ed Newton-Rex, Jukedeck’s CEO.
“It doesn’t need to be better than Adele or Ed Sheeran. There’s no desire for that, and what would that even mean? Music is so subjective. It’s a bit of a false competition: there is no agreed-upon measure of how ‘good’ a piece of music is. The aim [for AI music] is not ‘will this get better than X?’ but ‘will it be useful for people?’. Will it help them?”
The phrase “good enough” crops up regularly during interviews with people in this world: AI music doesn’t have to be better than the best tracks made by humans to suit a particular purpose, especially for people on a tight budget.
“Christopher Nolan isn’t going to stop working with Hans Zimmer any time soon,” says Cliff Fluet, partner at London law firm Lewis Silkin, who works with several AI music startups. “But for people who are making short films or YouTubers who don’t want their video taken down for copyright reasons, you can see how a purely composed bit of AI music could be very useful.”
Striking a more downbeat note, music industry consultant Mark Mulligan suggests that this strand of AI music is about “sonic quality” rather than music quality. “As long as the piece has got the right sort of balance of desired instrumentation, has enough pleasing chord progressions and has an appropriate quantity of builds and breaks then it is good enough,” he says.
“AI music is nowhere near being good enough to be a ‘hit’, but that’s not the point. It is creating 21st-century muzak. In the same way that 95% of people will not complain about the quality of the music in a lift, so most people will find AI music perfectly palatable in the background of a video.”
Not every AI-music startup is targeting production music. AI Music (the company) is working on a tool that will “shape-change” existing songs to match the context they are being listened to in. This can range from a subtle adjustment of its tempo to match someone’s walking pace through to what are essentially automated remixes created on the fly.
“Maybe you listen to a song and in the morning it might be a little bit more of an acoustic version. Maybe that same song, when you play it as you’re about to go to the gym, it’s a deep house or drum’n’bass version. And in the evening it’s a bit more jazzy. The song can actually shift itself,” says Mahdavi.
Australian startup Popgun has a different approach again. Its AI – called “Alice” – is learning to play the piano like a child would, by listening to thousands of songs and watching how more experienced pianists play them. In its current form, you play a few notes to Alice, and it will guess what might come next and play it, resulting in a back-and-forth human/AI duet. The next step will be to get her to accompany a human in real-time.
“It’s a new, fun way to interact with music. My 10 year-old daughter is playing the piano, and it’s the bane of our existence to get her to practise! But with Alice she plays for hours: it’s a game, and you’re playing with somebody else,” says CEO Stephen Phillips.
Vochlea, which is the other AI startup in the Abbey Road Red incubator, is in a similar space to Popgun. Beatbox into its VM Apollo microphone, and its software will turn your vocals into drum samples. Approximate the sound of a guitar or trumpet with your mouth, and it will whip up a riff or brass section using that melody.
“It’s a little bit like speech recognition, but it’s non-verbal,” says CEO George Philip Wright. “I’m focusing on using machine-learning and AI to reward the creative input rather than taking away from it. It came from thinking, if you’ve got all these ideas for music in your head, what if you had a device to help you express and capture those ideas?”
Many of the current debates about AI are framed around its threat to humans, from driverless trucks and taxis putting millions of people out of work, to Tesla boss Elon Musk warning that if not properly regulated, AI could be “a fundamental risk to the existence of civilisation”.
AI music companies are keen to tell a more positive story. AI Music hopes its technology will help fans fall in love with songs because those songs adapt to their context, while Popgun and Vochlea think AI could become a creative foil for musicians.
Jon Eades, who runs the Abbey Road Red incubator, suggests that AI will be a double-edged sword, much like the last technology to shake up the music industry and its creative community.
“I think there will be collateral damage, just like the internet. It created huge opportunity, and completely adjusted the landscape. But depending on where you sat in the pre-internet ecosystem, you either called it an opportunity or a threat,” he says.
“It was the same change, but depending on how much you had to gain or lose, your commentary was different. I think the same thing is occurring here. AI is going to be as much of a fundamental factor in how the businesses around music are going to evolve as the internet was.”
That may include the businesses having the biggest impact on how we listen to music, and how the industry and creators make money from it: streaming services. They already use one subset of AI – machine learning – to provide their music recommendations: for example in personalised playlists like Spotify’s Discover Weekly and Apple’s My New Music Mix.
The songs on those playlists are made by humans, though. Could a Spotify find a use for AI-composed music? Recently, the company poached François Pachetfrom Sony CSL, where he’d been in charge of the Flow Machines project.
It was under Pachet that in September 2016 Sony released two songs created by AI, although with lyrics and production polish from humans. Daddy’s Car was composed in the style of the Beatles, while The Ballad of Mr Shadow took its cues from American songwriters like Irving Berlin, Duke Ellington, George Gershwin and Cole Porter. You wouldn’t mistake either for their influences, but nor would you likely realise they weren’t 100% the work of humans.
Now Pachet is working for Spotify, amid speculation within the industry that he could build a team there to continue his previous line of work. For example, exploring whether AI can create music for Spotify’s mood-based playlists for relaxing, focusing and falling asleep.
For now, Spotify is declining to say what Pachet will be doing. “I have no idea,” admits Jukedeck’s Newton-Rex. “But to the question: ‘One day, will a piece of software that knows you be able to compose music that puts you to sleep?’ Absolutely. That’s exactly the kind of field in which AI can be useful.”
What’s also unclear is the question of authorship. Can an AI legally be the creator of a track? Can it be sued for copyright infringement? Might artists one day have “intelligence rights” written into their contracts to prepare for a time when AIs can be trained on their songwriting and then let loose to compose original material?
AI Music’s plans for automated, personalised remixes may bring their own complications. “If an app allows you to shape-change a song to the extent that you can’t even hear the original, does it break away and become its own instance?” says Mahdavi.
“If you stretch something to a point where you can’t recognise it, does that become yours, because you’ve added enough original content to it? And how do you then measure the point at which it no longer belongs to the original?”
The answers to these questions? Mahdavi pauses to choose his words carefully. “What we’re learning is that a lot of this is really quite grey.”
It’s also really quite philosophical, with all these startups and research teams grappling with fundamental issues of creativity and humanity.
“The most interesting thing about all this is that it might give us an insight into how the human composition process works. We don’t really know how composition works: it’s hard to define it,” says Newton-Rex. “But building these systems starts to ask questions about how [the same] system works in the human brain.”
Will more of those human brains be in danger of being replaced by machines? Even as he boldly predicts that “at some point soon, AI Music will be indistinguishable from human-created music”, Amper Music’s CEO, Drew Silverstein, claims that it’s the process rather than the results that will favour the humans.
“Even when the artistic output of AI and human-created music is indistinguishable, we as humans will always value sitting in a room with another person and making art. It’s part of what we are as humans. That will never go away,” he says.
Mark Mulligan agrees. “AI may never be able to make music good enough to move us in the way human music does. Why not? Because making music that moves people – to jump up and dance, to cry, to smile – requires triggering emotions and it takes an understanding of emotions to trigger them,” he says.
“If AI can learn to at least mimic human emotions then that final frontier may be breached. But that is a long, long way off.”
These startups all hope AI music will inspire human musicians rather than threaten them. “Maybe this won’t make human music. Maybe it’ll make some music we’ve never heard before,” says Phillips. “That doesn’t threaten human music. If anything, it shows there’s new human music yet to be developed.”
Cliff Fluet brings the topic back to the current home for two of these startups, Abbey Road, and the level of musician it has traditionally attracted.
“Every artist I’ve told about this technology sees it as a whole new box of tricks to play with. Would a young Brian Wilson or Paul McCartney be using this technology? Absolutely,” he says.
“I’ll say it now: Bowie would be working with an AI collaborator if he was still alive. I’m 100% sure of that. It’d sound better than Tin Machine, that’s for sure…”
Try it out
You can experiment with AI music and its close cousin generative music already. Here are some examples.
Jukedeck
As mentioned in this feature, you can visit Jukedeck’s website and get its AI to create tracks based on your inputs.
As mentioned in this feature, you can visit Jukedeck’s website and get its AI to create tracks based on your inputs.
AI Duet
Launched by Google this year, this gets you to play some piano notes, then the AI responds to you with its own melody.
Launched by Google this year, this gets you to play some piano notes, then the AI responds to you with its own melody.
Scape
Brian Eno was involved in this app, where you combine shapes to start music that then generates itself as your soundtrack.
Brian Eno was involved in this app, where you combine shapes to start music that then generates itself as your soundtrack.
Humtap Music
A little like Vochlea in this feature, Humtap’s AI analyses your vocals to create an instrumental to accompany you.
A little like Vochlea in this feature, Humtap’s AI analyses your vocals to create an instrumental to accompany you.
Weav Run
This is part running app and part music app, using “adaptive” technology to modify the tempo of the song to match your pace.
This is part running app and part music app, using “adaptive” technology to modify the tempo of the song to match your pace.
