Using The Neutone Audio Plugin: AI Tone Transfer in Your DAW
Several new AI music tools have surfaced over the past year. You might be wondering about AI plugins and whether you can access AI music in the DAW.
Currently at least two companies have started bringing free AI-powered music plugins to market.
Google Magenta Studio offers a collection of AI music plugins, like "Continue" that generates MIDI content based on existing MIDI you input. Google also offers a downloadable DDSP plugin for tone transfer.
Magenta Studio and DDSP still have a way to go before they are really usable. That's not a knock against the Magenta team. They just need to work out some of the weird transients. I'm excited to back these projects again in the future if they improve.
Fortunately, an award-winning Japanese organization called Qosmo decided to tackle the AI music plugin challenge. Run by a group of artists, designers, researchers and programmers, they understood the needs of musicians and worked to create something specially for the DAW. Neutone has since split off and become its own, independent company.
Overview
What is the Neutone audio plugin?
Future of AudioCipher: Neutone for Text-to-MIDI generation?
What is the Neutone audio plugin?
Neutone is an AI audio plugin, as well as a community, that makes third party AI music services like DDSP and RAVE available within a DAW. It currently supports timbre transfer and audio effects only, but additional AI music services might become available on the Neutone ecosystem in the future.
How to load and use Neutone in a DAW
The plugin's main menu offers an intuitive list of tags and authors to choose from. File size for the models range anywhere from 1-20 mb. There are also some bigger models that clock in at 60-260 mb.
Select the models you want to download and hit use to boot one up. Once you do, here's the screen you'll be presented with:
Tweak parameters for gain, reverb, and custom settings like harmonics, noise, and pitch shifting. As you make changes, the blurry abstract image on the right moves slightly as well. It's an interesting creative touch that gives the app more color.
Be sure to check out the Neutone project page for a deep dive into how it all works. Here is a 2.5 hour video published early March 2023, packed full of insights about how to use the app as an audio engineer:
Future of AudioCipher: Neutone for Text-to-MIDI?
Our company built a text-to-music MIDI plugin that generates chords and melodies based on words. The VST uses a non-ai algorithm to model the experience of an AI text-to-music service that we imagine in the future.
As these services continue to develop, we hope to see ethically trained MIDI generation APIs available to consume. Like Neutone, AudioCipher could partner with devs and provide an interface for text input, key signature, chord, and rhythm parameters. We'll push up a data blob and retrieve MIDI in return.
Music producers will boot up AudioCipher, type in a thought, and AI will dish out a collection of musical ideas to pull from in MIDI format.
No more Google Colab. No need to host a local neural network. Just hit submit and go back to working in your DAW while the MIDI and audio files render. The workflow will be similar to Dalle-2 and Midjourney.