By now you're probably already familiar with OpenAI's tool ChatGPT. In this article we'll provide a deep review of how anyone can use the service to generate music.
You can use ChatGPT for just about anything, from chord progressions, melodies, lyrics and tablature to machine-level code.
If you've been following the recent developments in text-to-music and lyrics-to-song generation, get ready to shed some of your preconceived notions. You won't be able to generate musical audio files or singing vocals from ChatGPT.
Users can elect to hear the AI bot speak with an AI voice, but the despite its deep knowledge of music theory and scales, ChatGPT will not sing you a melody.
So what can ChatGPT do, musically speaking?
I'll share some ChatGPT music techniques with you in this article, along with its history so you understand why this became popular in this first place.
Table of Contents
A brief history of AI text-to-music
OpenAI released a new GPT Builder on November 6 2023, making it possible for the first time to upload documents and fine tune an AI bot's expertise in any topic. This has brought us one step closer to creating stable personalities. An AI bandmate or collaborator can now be easily conjured to help reason through problems or get inspired.
Back in 2022, generative image services like Dalle-2, MidJourney and Stable Diffusion proved that artificial intelligence could create realistic imagery in a matter of seconds. Then OpenAI released ChatGPT, further challenging our ideas about human intelligence (and how machines can imitate it). As musicians, many of us began tinkering to see what was possible.
OpenAI previously published a MIDI generation tool called MuseNet in 2019 and an audio generator called Jukebox in 2020. However, the MuseNet API was turned off in December 2022, shortly after ChatGPT was released. OpenAI hasn't made any moves to publish a new music generation tool since then. Jukebox lacks an easily accessible user interface, so it was never widely adopted.
In December 2022, AudioCipher published version 3.0. This text-to-MIDI music plugin doesn't run on artificial intelligence, but delivers the experience of turning words into music in a DAW. The benefit here is that you don't need an internet connection and you gain tight control over key signature, rhythm automation, and chord settings. You can toggle between melody and chord generation settings. Then drag to a MIDI channel and make edits in your piano roll until you've got something you like.
At the end of January 2023 Google published a paper describing an upcoming text-to-music generator named MusicLM. They released it as a free beta app in May 2023. Facebook-Meta soon followed suit and published their own text-to-music app in June 2023 called MusicGen. These apps both create audio files based on written descriptions. As a result, they can assist sample-based music producers but are less useful for people who prefer to work with MIDI.
MusicLM and MusicGen are more inherently musical than ChatGPT, because they were trained on labeled audio files. It's important to know that as a large language model, ChatGPT is trained on music theory concepts but lacks a deeper grasp of music composition. This means that it composes music from a purely intellectual understanding of music, rather than an innate understanding of it.
Now that you have a lay of the land, let's have a look at some of the earliest experiments with ChatGPT and see how they've developed since then.
The first wave of ChatGPT music experiments
The first wave of ChatGPT music came from social media influencers. They were releasing meme videos of famous rappers using UberDuck for vocal impersonation. ChatGPT was only being used to generate lyrics at this stage. The technique was mostly for fun and parody - nobody mistook the music for the actual artists. Here's how the method works:
Users come up with a lyric prompt for ChatGPT, like "write a lyrical verse in the style of [artist] about [topic]"
Find a section of the lyric output that you like and plug it into UberDuck
Export the audio from Uberduck and bring it into your DAW
Use an autotune plugin to apply a melody. If you're looking for a free option, try GVST's GSnap VST.
Since then, a great number of AI voice generator services have been released, specifically for musicians to use. Uberduck is still the most popular for celebrity voices, but other apps provide better support for adding melodies to the voices.
As time went on, people began publishing additional experiments with ChatGPT music prompts focused on chords, melodies, and MIDI generation.
WavTool: An AI DAW powered by GPT4
WavTool is a browser-based AI DAW in depth, showcasing how users can summon its AI assistant to generate MIDI melodies. This can be a shortcut for people who don't want to write javascript programs and build MIDI from scratch. All you have to do is type in what you want and let the DAW get to work.
You'll find that vague prompts like "create a catchy melody" tend to result in underwhelming results, like a C major scale that repeats twice. The more specific you get with your prompts, the better your results will be. Here are some prompt components that I recommend using (or not using) to improve your outcomes:
Request "a combination of quarter, eighth, and sixteenth notes" when asking for a melody. It understands these instructions and will usually comply.
Tell it what scale or key signature you want it to use, if that matters to you.
You can ask it for specific chord progressions using roman numerals.
Avoid asking for a "chord progression in <key signature" because it will usually create an ascending line of chords in that key (i.e. I - ii - iii - IV etc).
You can try asking for chord progressions that fit a genre's style, like a jazzy chord progression or a punk rock progression. Expect mixed results.
If you don't want to sign up for WavTool then you can use ChatGPT to run your experiments for free. Let's move on to that next.
ChatGPT Chord Progressions
Chord progressions are one of the easiest places to get started with ChatGPT music prompts. You can request a progression in a particular key and it will perform quite well. As you drill down further, asking for progressions in the style of a particular composer or genre, the cracks begin to show.
GPT 4.0 requires a paid account but it also represents the latest capabilities of their most powerful model, so we used GPT 4.0 for these experiments.
ChatGPT Music Prompt 1: Write a chord progression in [key signature].
Result (✅): We get a simple four chord progression, including the notes in each chord and a short description of why it chose those chords. The roman numeral format (I - vi - ii - V) accurately corresponds to the chords it provided.
ChatGPT Music Prompt 2: Write a complex chord progression in [key signature].
Result (✅): We chose a more obscure key signature (F# minor) and ChatGPT had no problem handling it. All of the notes and roman numeral progressions are accurate within the context of the key.
ChatGPT Music Prompt 3: Write a chord progression in the style of JS Bach.
Result (✅): We started with a classical composer because ChatGPT would have plenty of access to written information about his chord composing style. The progression included two trademark characteristics of Bach's music - use of the secondary dominants and the use of the numeral 6 representing a first inversion. This distinguishes the notation from contemporary use of the number 6, where it would imply a 6th chord.
ChatGPT Music Prompt 4: Write a guitar chord progression for a jazzy lofi hip hop beat.
Result (✅/❌): Here we find some of the first errors in ChatGPT's output for chords.
The first three chords in this progression do have the characteristic lofi hip hop sound. But the roman numeral notation doesn't make sense. It would be more accurate to call it a ii-V-I in D major.
The last two chords create an unresolved feeling that would not loop well. As a loop-based genre, this is a problem that would take some creativity on the musician's behalf to make it work.
ChatGPT incorrectly identifies G# as the third scale degree of E minor. It makes the same problem with the last chord, calling C# minor the sixth scale degree is C natural. As a result, the last two chords don't match the first three.
ChatGPT Music Prompt 5: Write an 8-chord progression in [key signature].
Result (✅/❌): The results here are technically in F minor, but there are a few problems
The final chord resolves to the tonic prematurely. Normally the return to the tonic would happen after the completion of the 8 chord progression. This could easily be solved by extending the length of the 7th chord for two bars.
The progression is basically just going through the circle of fifths (VII - III - VI - ii - v - i). While this is fairly common, it makes for a jumpy progression unless you use chord inversions. Those inversions aren't indicated her.
The output features triads exclusively, so the chord progression lacks color.
ChatGPT Music Prompt 6: Write an 8-chord progression in [key signature] using inversions for closed harmony.
Result (❌): Here we can really see the limitations of ChatGPT music output. On the one hand, it understands the notion of inversions and closed harmony. This is evident by its written response, but if you actually play the chords here, there are a few glaring problems.
The second chord (G-Bb-Db) was written over D natural in the bass. That note does not belong to the key or chord and is about as ugly as you could ever imagine. It's a terrible second chord for an F minor progression.
The fourth, fifth and sixth chords in these progressions contain large leaps, which violates the prompt's instructions to use closed harmony. ChatGPT even stated its intention to use inversions to create smooth stepwise motion, but violated this rule in its output.
Here is a screenshot of the piano roll to highlight the problems with this progression. The chord highlighted in pink shows the D-G-Bb-Db chord that made our skin crawl. You can see the big chord leaps highlighted in green, that missed the mark for smooth voice leading in the bass.
This concludes the section on generating chord progressions with ChatGPT music prompts. As you can see, it's capable of creating something, but the more specific you get with your requests, the more likely it is to make errors due to its lack of training in the mathematics of music composition.
ChatGPT Guitar Tablature
Generating guitar tabs with ChatGPT is fairly simple. You just ask for what you want and it will spit it out in an ASCII format. Unfortunately, it tends to struggle with getting the key signature and chords right for even the most simple prompts. You can find examples of that in the video above, and watch how he resorted to asking for written descriptions of a guitar solo to eventually get his desired results.
ChatGPT Guitar Tablature Prompt 1: Write a chord progression in guitar tablature
Result (❌): ChatGPT returns an arpeggio in E minor and A minor, but then states that it's a chord progression (it's not) with the chords GM, DM, and CM. It's none of those things, so this is a complete miss.
ChatGPT Guitar Tablature Prompt 2: Create guitar tabs for a ii7 V I chord progression in FMajor.
Result (✅): This time we get the correct response, albeit in a clunky vertical format. By instructing it on the progression we wanted, it seems to have an easier time. We get a rootless Gm7 chord, followed by a C7 chord (we didn't ask for the extension but that's okay) and an F major chord.
ChatGPT Guitar Tablature Prompt 3: Write a metalcore guitar solo in tablature.
Result (❌): Here we asked for metalcore, implying that it should be in a minor or diminished scale. Instead it gave us a rudimentary major scale. It understood that we wanted a melody, but that's about it. It also incorrectly said that this was in E minor, but contains an F natural (10th fret G string) so they key signature is wrong.
We've found other problems with ChatGPT guitar tabs. For example, in tabs the '/' character indicates a slide up and '\' for a slide down. ChatGPT tends to use the wrong kind of slash and doesn't seem to understand what the symbol means.
ChatGPT Lyric and Chord Generation
As a text generator, ChatGPT can perform AI lyric generation all day. When you try to combine the lyrics with chord charts, it works, but if you add melodies, you start to run into trouble. Here's a couple of examples to showcase these limitations.
ChatGPT Lyrics + Chords Prompt 1: Write lyrics for a folk song and add chord charts to it.
Result (✅): The chords are added to the beginning of each word and color coded to make it easily readable. We still need to write our own melody with this prompt.
ChatGPT Lyrics + Chords Prompt 2: Using the same chart, include melodic notation for these chords and lyrics
Result (❌): ChatGPT has created an arpeggiated finger picking guitar riff for the melody. There's no connection between the those arpeggios and the lyrics.
ChatGPT Melodies
Next on the list is melody generation. Unlike chord notation, melodies are a bit trickier to express through plaintext. For this reason, I started by asking ChatGPT for a list of the best formats to use when generating melodies. Here's a list of options that it provides. You can reference them to experiment with your own music prompts:
Standard Notation: Naming notes (e.g., "C-E-G") along with their durations (e.g., "quarter note C, half note E, quarter note G").
Scale Degrees: Naming a melody in relation to the key it's in using scale degrees (e.g., "In the key of C Major: 1-3-5").
Guitar Tablature: You'll get melodies as they appear on a guitar neck, but rhythm will be missing. We already highlighted its problems previously.
Piano Roll Notation: Describe the melody using MIDI note numbers, representing the values on Digital Audio Workstation's piano roll. For example, a C Major arpeggio would be represented as 60, 64, 67. We used this format in the next section, using ChatGPT to write javascript code and create MIDI files. This works surprisingly well with the right prompts.
Solfège: Describe simple melodies using the do-re-mi system. This lacks rhythmic input and will keep you fixed to a simple diatonic scale.
Alphanumeric Pitch Notation: This system uses the letters A-G in combination with numbers 0-9 to represent pitches, with C4 often being set as "middle C". For example, a melody might go "C4-E4-G4-C5".
Pitch frequencies: Name each musical note as a frequency (hz) rather than using standard notation concepts like note names.
To get started, we went with the alphanumeric + rhythm notation.
ChatGPT Melody Prompt 1: Write a melody in <key signature> using alphanumeric pitch notation. Include indicators of rhythm.
Result (✅): The melody we retrieved is a simple C major pentatonic melody, with a combination of quarter and half notes. It's not a particularly interesting tune, but there is some variation, with the arpeggio turning around on the sixth scale degree and returning stepwise with the 2nd scale degree passing tone from E-D-C.
ChatGPT Melody Prompt 2: Use this chord progression and give each chord four beats. Write a catchy melody in the same key signature using quarter and eighth notes. Be sure to indicate the duration of each note in the melody. If the melody contains notes outside of the chord, then it should only be used as passing tones to a note that is in the chord. Here is the progression: Dm7 - G7 - CM9 - Em7
Result (❌): As we saw previously, when the musical mathematics become too complex, ChatGPT starts to lose the plot. It understands what we want it to do, but quickly makes mistakes.
It calls F a passing tone, despite being set over the Dm7 chord.
E minor is a passing tone, but it fails to label it as such
It makes the same mistake with the G note, calling it a passing tone while playing over the G7 chord. Then the next note, A, is not labeled a passing tone. This illustrates that it is not properly checking the melody against the chords, because it lacks the training to do so effectively.
Using Javascript to make ChatGPT MIDI music
The note-and-rhythm notation in the examples above are annoying to use and not very practical. It's more effective to use a free code editor like VSCode to generate MIDI melodies. We'll explore how to do that here.
First, you'll need to download Visual Studio Code and install Node.JS. Both of these are free. Once they're installed, you'll create a new folder on your computer and use VSCode's terminal to CD into it. Save a simple file, let's say "testmidi.js" and within that file, you'll be able to use this technique.
ChatGPT Visual Studio MIDI Prompt 1: Please write javascript code to generate a C major scale in MIDI notation, using the MIDIWriter javascript Node library.
Result (✅): ChatGPT needed a few rounds of debugging to get this right, but all I had to do was tell it the error code and eventually it produced a working piece of code. That's the code we shared above.
constMidiWriter = require('midi-writer-js'); const fs = require('fs'); // Create a new tracklet track = newMidiWriter.Track(); // Define the C Major scale (middle C to the next C) in MIDI numbersletCMajorScale = [60, 62, 64, 65, 67, 69, 71, 72]; // Loop through the notes in the scale and add them to the trackfor(let i = 0; i < CMajorScale.length; i++) { // Adding a note event for each note in the scale, with a duration of '4' (quarter note) track.addEvent(newMidiWriter.NoteEvent({pitch: [CMajorScale[i]], duration: '4'})); } // Create a writerlet writer = newMidiWriter.Writer(track); // Save the MIDI output to a filelet base64String = writer.dataUri().split(',')[1]; let midiData = Buffer.from(base64String, 'base64'); fs.writeFileSync('C_Major_Scale.mid', midiData);We're using a package called MidiWriter.js. Just paste the code above into your testmidi.js file and when you run "node testmidi.js" in your VSCode terminal, it will automatically save the MIDI file to that folder. You can use ChatGPT to elaborate and create melodies with actual rhythm and character. We'll get to that next.
ChatGPT Chord + Melody MIDI Prompt 2: Using the javascript code I've provided, create a ii7 - V7 - IM7 chord progression and then write a melody on top that uses a combination of quarter and eighth notes. The melody should match the chords being played so that they don't clash. Try to use a fluid, natural sounding melody, and give the chord progression some rhythm too.
Result (✅): Wow! ChatGPT created a melody that goes perfectly with the chord progression we provided. All we had to do was switch over to a programming language and the results improve dramatically.
Here's the code so you can test it yourself:
const MidiWriter = require('midi-writer-js');
const fs = require('fs');
// Create two new trackslet chordTrack = new MidiWriter.Track();
let melodyTrack = new MidiWriter.Track();
// Define the chords - Dm7 (D-F-A-C), G7 (G-B-D-F), and Cmaj7 (C-E-G-B)let chords = [
{ pitch: [62, 65, 69, 72], duration: '4' }, // Dm7 quarter note
{ pitch: [62, 65, 69, 72], duration: '4' }, // Dm7 quarter note
{ pitch: [67, 71, 74, 77], duration: '4' }, // G7 quarter note
{ pitch: [67, 71, 74, 77], duration: '4' }, // G7 quarter note
{ pitch: [60, 64, 67, 71], duration: '2' }, // Cmaj7 half note
{ pitch: [60, 64, 67, 71], duration: '2' } // Cmaj7 half note
];
// Add chords to the chordTrack
chords.forEach(chord => {
chordTrack.addEvent(new MidiWriter.NoteEvent(chord));
});
// Define the melody - a simple melody that fits over the chordslet melody = [
{ pitch: [74], duration: '8' }, // D eighth note
{ pitch: [72], duration: '8' }, // C eighth note
{ pitch: [74], duration: '4' }, // D quarter note
{ pitch: [76], duration: '8' }, // E eighth note
{ pitch: [77], duration: '8' }, // F eighth note
{ pitch: [76], duration: '4' }, // E quarter note
{ pitch: [72], duration: '2' }, // C half note
{ pitch: [71], duration: '2' } // B half note
];
// Add melody to the melodyTrack
melody.forEach(note => {
melodyTrack.addEvent(new MidiWriter.NoteEvent(note));
});
// Create a writer with both trackslet writer = new MidiWriter.Writer([chordTrack, melodyTrack]);
// Save the MIDI output to a filelet base64String = writer.dataUri().split(',')[1];
let midiData = Buffer.from(base64String, 'base64');
fs.writeFileSync('ii7_V7_IM7_Progression_With_Melody.mid', midiData);
Here's a visual of what that MIDI file looks like when imported into a DAW like Logic Pro X. As you can see, the melody has a nice shape and it worked perfectly with the chord progression.
ChatGPT Chord + Melody MIDI Prompt 3:
Result (✅/❌): Here we begin to run up against the limitations of ChatGPT's capabilities as a general purpose AI tool. Four-part Bach chorales require melodies to move in some combination of parallel and contrary motion.
After iterating through another five rounds of instructions we eventually got ChatGPT output that's constrained to a single key signature and exports a MIDI file with four voices. There's a good amount of rhythmic variation, but it sounds more like Debussy than Bach. Here's a screenshot of what that final MIDI looks like:
Here's the code so you can try it out yourself:
const MidiWriter = require('midi-writer-js');
const fs = require('fs');
// Create four new tracks
let sopranoTrack = new MidiWriter.Track();
let altoTrack = new MidiWriter.Track();
let tenorTrack = new MidiWriter.Track();
let bassTrack = new MidiWriter.Track();
// Define the A minor scale in three octaves
let aMinorScale = [57, 59, 60, 62, 64, 65, 67, 69, 71, 72, 74, 76, 77, 79, 81, 83, 84, 86, 88, 89, 91];
// Define four melodies using the indexes of the scale
let sopranoMelody = [
{ index: 20, duration: '4' },
{ index: 18, duration: '8' },
{ index: 16, duration: '8' },
{ index: 14, duration: '16' },
// Contrary motion starts here
{ index: 16, duration: '16' },
{ index: 18, duration: '4' },
{ index: 20, duration: '8' },
{ index: 18, duration: '8' },
// Back to parallel motion
{ index: 16, duration: '4' },
{ index: 14, duration: '8' },
{ index: 12, duration: '8' },
{ index: 10, duration: '4' }
];
let altoMelody = [
{ index: 15, duration: '4' },
{ index: 13, duration: '8' },
{ index: 11, duration: '8' },
{ index: 9, duration: '16' },
// Contrary motion starts here
{ index: 11, duration: '16' },
{ index: 13, duration: '4' },
{ index: 15, duration: '8' },
{ index: 13, duration: '8' },
// Back to parallel motion
{ index: 11, duration: '4' },
{ index: 9, duration: '8' },
{ index: 7, duration: '8' },
{ index: 5, duration: '4' }
];
let tenorMelody = [
{ index: 10, duration: '8' },
{ index: 12, duration: '8' },
{ index: 14, duration: '4' },
{ index: 12, duration: '16' },
// Contrary motion starts here
{ index: 10, duration: '16' },
{ index: 8, duration: '4' },
{ index: 6, duration: '8' },
{ index: 4, duration: '8' },
// Back to parallel motion
{ index: 2, duration: '4' },
{ index: 0, duration: '8' },
{ index: 2, duration: '8' },
{ index: 4, duration: '4' }
];
let bassMelody = [
{ index: 5, duration: '8' },
{ index: 7, duration: '8' },
{ index: 9, duration: '4' },
{ index: 7, duration: '16' },
// Contrary motion starts here
{ index: 5, duration: '16' },
{ index: 3, duration: '4' },
{ index: 1, duration: '8' },
{ index: 3, duration: '8' },
// Back to parallel motion
{ index: 1, duration: '4' },
{ index: 3, duration: '8' },
{ index: 5, duration: '8' },
{ index: 7, duration: '4' }
];
// Add the melodies to the tracks
sopranoTrack.addEvent(sopranoMelody.map(note => new MidiWriter.NoteEvent({ pitch: [aMinorScale[note.index]], duration: note.duration })));
altoTrack.addEvent(altoMelody.map(note => new MidiWriter.NoteEvent({ pitch: [aMinorScale[note.index]], duration: note.duration })));
tenorTrack.addEvent(tenorMelody.map(note => new MidiWriter.NoteEvent({ pitch: [aMinorScale[note.index]], duration: note.duration })));
bassTrack.addEvent(bassMelody.map(note => new MidiWriter.NoteEvent({ pitch: [aMinorScale[note.index]], duration: note.duration })));
// Combine all tracks
let writer = new MidiWriter.Writer([sopranoTrack, altoTrack, tenorTrack, bassTrack]);
// Save the MIDI output to a file
let base64String = writer.dataUri().split(',')[1];
let midiData = Buffer.from(base64String, 'base64');
fs.writeFileSync('Four_Part_Fugue4.mid', midiData);
That's as far as we're going to take the MIDI javascript code generation prompts. You can plug any of those snippets into GPT4 as a starting point and ask for additional variations.
For a technical deep dive further into the musical potential and limitations of GPT-4, check out this video from creative coder and composer Marc Evanstein:
If you want a simpler text-to-MIDI experience for your DAW workflow, be sure to check out the AudioCipher VST.