In October 2022, an official statement from the RIAA (Recording Industry Association of America) accused three websites of potential copyright infringement. They were singled out for the use of unlicensed music in marketing materials, but also called out for the nature of their technology itself.
A single developer or team operates these three websites. They issued an official response on October 17th, explaining that they had removed the problematic demo videos from their page. That being said, they denied that the applications themselves violated copyright law.
It came out that these sites run on open source code maintained by a big technology company, closely partnered with several RIAA labels. This detail complicates the issue and reveals a deeper challenge the RIAA may have in regulating emerging AI mastering technology.
In this article, we’ll cover some of the backstory behind this controversy. I'll also give you a quick overview on how to use each of the tools. For music producers looking to make AI mastering part of their workflow, we’ll include resources on smart plugins for your DAW.
Contents
What is an AI Mastering Tool?
Mastering engineers balance the sonic elements within a mix, so that the final audio file sounds good across all devices. Historically, mastering has required expensive equipment, a trained ear, and some working knowledge of equalization, compressors, limiting and stereo enhancement.
Major record labels still hire professional mastering services. That's part of how they achieve the highest possible sound quality.
Smaller independent artists tend to use services like Landr, Emastered, and Bandlab instead. These affordable alternatives get you most of the way there, from the comfort of your bedroom laptop.
To use a concrete example, Landr let you preview playback through a handful of audio mastering options. It's considered an AI mastering tool because they use machine learning to mimic decisions a human engineer would make.
According this article on mastering by Landr, they use “an adaptive engine with micro-genre detection to make subtle frame-by-frame adjustments using tools like multi-band compression, EQ, stereo enhancement, limiting and harmonic saturation, based on the unique properties of your song.”
Online mastering services simplify the mastering process and limit the number of decisions you have to make. You can explore simple attributes like loudness and warmth. The mastering process only takes a few minutes. Then you can download the mastered tracks in wav or mp3 format.
Apps like Landr and Bandlab have not been singled out in the RIAA’s complaints.
Why the RIAA Can’t Shut Down Songmastr
The RIAA’s early-October concerns about sonic plagiarism were predicated on songmastr using machine learning to train on their music. They also complained that some marketing videos on the website featured unlicensed music.
News of the controversy made its rounds on popular technology sites like Vice and TorrentFreak, before drawing an official rebuttal from the developer of the apps on October 17th.
The AI based extractors and mixers targeted by the RIAA were built on Spleeter, an open source library maintained by Deezer. In a stroke of irony, Deezer is a key partner to several RIAA labels. They are one of the largest, legal streaming platforms in the world.
Deezer's Spleeter library powers another popular application, called the Moises app. Also known as The Musicians App, they have close to 12,000 ratings and 4.7 stars on the Apple App store.
We recently published an article on Spotify's audio-to-MIDI service Basic Pitch. This free web application could be used to pull raw musical material from protected tracks and remix them.
So where does the RIAA's animosity toward AI mixers leave us in the near future, with parallel technologies like Spotify's Basic Pitch? Will they go after the machine learning teams at Deezer and Spotify?
SongMastr: “AI-Powered” Music Mixer
Songmastr is an AI mastering tool that attempts to match a reference song of your choice. The free mastering software runs on an open source Python library called Matchering. To try it out yourself, visit songmastr and upload a target track that you want to master. Next, upload a high-quality, mastered song to use as the reference track.
I ran a series of experiments and found that a 1mb mp3 file with a 23 mb reference track in wav format produced a 9mb wav file with a sample rate of 44.1 khz. This file was generated in about 30 seconds and I was able to download it from their website quickly.
SongMastr can be used up to seven times per week, with tracks at a maximum of 10 minutes length and up to 80MB each. These limitations may only be temporary, as the developers say its due to their server and traffic volume.
The Matchering algorithm behind Songmastr studies the RMS, FR, peak amplitude and stereo width of your reference track, before applying it to the target audio file.
Songmastr has published examples of what their online audio mastering services sound like. These demos apparently no longer violate the RIAA’s terms of service.
Acapella Extractor and Vocal Remover
The names of the other two offending applications, Acapella-extractor and Remove-Vocals, pretty much say it all. One removes vocals from an existing track while the other isolates vocals and lets you hear them on their own. To use the app, you just drag a wav or mp3 audio file and the source-separated audio is available for download moments later.
This kind of technology isn’t emerging in a vacuum. The DJ software company Algoriddim, for example, published Neural Mix in 2020. This source-separation tool helps EDM artists isolate drums, instruments and vocals within a mix, in real time.
Spotify has also teased out a karaoke feature, allowing listeners to isolate the instrumentals from a track and sing along with lyrics on the screen. This feature, while not particularly useful to music producers, could impact playlists of every genre, from hip hop to indie pop.
Smart Plugins for AI Mastering and Mixing
We’ve covered a variety of online mastering services that load within a web browser. But if you’re a music creator or songwriter who works in a DAW, you may be looking for something that runs locally on your machine.
There are a variety of AI-enhanced music mastering plugins available today, offering greater control over your settings, with the option to save in aiff and other lossless formats.
MusicRadar published an article titled The Secret of Smart Plugins, covering seven of the best options currently available.
We generally focus on free tools here on the AudioCipher blog. This article is getting a bit long anyway, so we’ll refer you over to them for insights on how the technology works and details about what makes each plugin unique.