Who knew that Image-to-MIDI converters even existed? Turns out they've been around for more than a decade, built almost exclusively by indie software teams. The quest to turn picture into MIDI may seem a bit odd. But once you convert that first image into sound, you'll feel the power!
The truth is, when you’re making electronic music in a DAW, inspiration can come from anywhere. You don’t have to keep every note that shows up when you use these tools. The point is to reimagine your relationship to the piano roll as a digital art canvas. It might change how you feel about the tool when you go back to making regular music.
Glasys: MIDI Art Performer & Teacher
In 2017, MIDI art went viral and a number of content creators took their shot on YouTube. Entertainer and music educator Andrew Huang created MIDI unicorns and whales, while others designed scrolling panoramas of UFOs in outer space.
When it comes to live MIDI performance, there's nobody better than Glasys. His live performance of complex images not only looks great but also sounds amazing.
Piano savant Jacob Collier has also dabbled briefly in MIDI art, performing MIDI art on Valentines Day and other special occasions. But Glasys alone has dedicated himself to perfecting the craft and teaches others how to do it as well.
In 2017, around the time creators were going viral with their MIDI art videos, electronic music artist Tom Cosm released his own Ableton Max plugin and after gating access for some time, eventually shared it for others to enjoy. That's the piece of software that we'll be focusing on in this article.
How to install and use the Image-to-MIDI Effect
You can pick up a copy of Tom Cosm's Image to Midi app for free or donate an amount of your choosing. Indie developer Michael Ganss published an update in 2019 with improvements to Tom’s code. It has a couple thousand downloads and a five star rating on the Max For Live device library. We'll be focusing on the original app for this article.
As the video tutorial shows, it’s not difficult to use Tom Cosm's plugin. To get started, all you have to do is download the .amxd file and drop it into your Max MIDI Effects folder.
From there, you’ll open Ableton and create two MIDI tracks. The first track acts as a sender and the second as a receiver. Load the Max MIDI effect onto the first track and use the plugin’s open button to select an image file from your computer.
Next, arm the second track and have it receive MIDI from your sender (track one). Open a new MIDI clip on the receiving track (track two) and start recording. It may take anywhere from 30-60 minutes to create the final MIDI file, depending on the complexity of the original image file.
The plugin algorithm makes use of both the black and white keys. It doesn't seem possible to constrain the audio files to one key signature. There might be a workaround with Ableton's fold feature, but it only occurred to me as I was writing this article. Let me know in the comments if you try fold and it works.
This render of the AudioCipher logo took about 45 minutes from start to finish. I tried speeding up the process by cranking up the tempo. 999 BPM broke the app and I discovered that you can't change tempo while it's recording. Eventually I settled on 120 BPM and ran Ableton in the background while I worked on other things. This worked every time, as long as the source file was good.
After exporting the final MIDI file, I brought it into Logic Pro X to explore different color pallets. What's the first image you would turn into MIDI, if you had this tool?
Additional Image-to-Music converters
There’s a surprising number of applications designed to turn images into sound or MIDI. Each one takes its own unique approach. The only app in this space with any obvious utility is Scan Score - it helps jazz and classical transposers convert screenshots of sheet music into MIDI files.
The rest of the apps in this article are used for image sonification. For those not familiar, this means that the software takes visual data and converts it into audio.
i2sm is a Windows application from 2011 that transforms digital photos into sound, using pixel data from any jpg, png or gif file. Playback options include a few different musical instruments. To be clear, this app does not attempt to create MIDI artwork that resembles the original image file.
i2sm was popularized in 2017 by the YouTube channel VineSauce, featured above, after he brutally mocked the app on a live stream that's reached more than half a million people.
A few other popular photo-to-sound apps, like Photosounder and Sonic Photo, came out around the same time as i2sm. They visibly scanned and sonified images in real time, presumably for the user's enjoyment. It's difficult to see what the utility of these apps would be, beyond noise music and possibly science.
Astronomers have famously used visual data sonification to study the stars. Open source libraries like Universum-Total turn scientific data into sound for the purposes of scientific research. Data sonification is fascinating, but a much bigger topic than we can fit into the scope of this article. So I'll leave you with this classic vice episode on turning the sun into music.
Curious to see what kind of musical pixel art others have created? Check out our previous article on the brief history of MIDI art and get our thoughts on where all of this may be heading.