Image-to-MIDI is a niche internet music subculture where pixel art aesthetics are applied to a DAW piano roll. The music composition conforms to the shape and creates a kind of new object, made up of interlocking visual and musical logic.
The most prominent and accomplished figures in the space are Glasys and Sympawnies. We will share some tutorials with you below, including an image-to-MIDI app by Tom Cosm may help you get started creating your own artwork.
Glasys: MIDI Art Performer & Teacher
In 2017, MIDI art went viral and a number of content creators took their shot on YouTube. Entertainer and music educator Andrew Huang created MIDI unicorns and whales, while others designed scrolling panoramas of UFOs in outer space.
When it comes to live MIDI performance, there's nobody better than Glasys. His live performance of complex images not only looks great but also sounds amazing. He has almost singlehandedly dedicated himself to perfecting the craft and teaches others how to do it as well.
Piano savant Jacob Collier has also dabbled briefly in MIDI art, performing MIDI art on Valentines Day and other special occasions.
Try creating your own Image-to-MIDI Effect
In 2017, around the time creators were going viral with their MIDI art videos, electronic music artist Tom Cosm released his own Ableton Max plugin and after gating access for some time, eventually shared it for others to enjoy. That's the piece of software that we'll be focusing on in this article.
You can pick up a copy of Tom Cosm's Image to Midi app for free or donate an amount of your choosing. Indie developer Michael Ganss published an update in 2019 with improvements to Tom’s code. It has a couple thousand downloads and a five star rating on the Max For Live device library. We'll be focusing on the original app for this article.
As the video tutorial shows, it’s not difficult to use Tom Cosm's plugin. To get started, all you have to do is download the .amxd file and drop it into your Max MIDI Effects folder.
From there, you’ll open Ableton and create two MIDI tracks. The first track acts as a sender and the second as a receiver. Load the Max MIDI effect onto the first track and use the plugin’s open button to select an image file from your computer.
Next, arm the second track and have it receive MIDI from your sender (track one). Open a new MIDI clip on the receiving track (track two) and start recording. It may take anywhere from 30-60 minutes to create the final MIDI file, depending on the complexity of the original image file.
The plugin algorithm makes use of both the black and white keys. It doesn't seem possible to constrain the audio files to one key signature. There might be a workaround with Ableton's fold feature, but it only occurred to me as I was writing this article. Let me know in the comments if you try fold and it works.
This render of the AudioCipher logo took about 45 minutes from start to finish. I tried speeding up the process by cranking up the tempo. 999 BPM broke the app and I discovered that you can't change tempo while it's recording. Eventually I settled on 120 BPM and ran Ableton in the background while I worked on other things. This worked every time, as long as the source file was good.
After exporting the final MIDI file, I brought it into Logic Pro X to explore different color pallets. What's the first image you would turn into MIDI, if you had this tool?
Additional Image-to-Music converters
There’s a surprising number of applications designed to turn images into sound or MIDI. Each one takes its own unique approach. The only app in this space with any obvious utility is Scan Score - it helps jazz and classical transposers convert screenshots of sheet music into MIDI files.
The rest of the apps in this article are used for image sonification. For those not familiar, this means that the software takes visual data and converts it into audio.
i2sm is a Windows application from 2011 that transforms digital photos into sound, using pixel data from any jpg, png or gif file. Playback options include a few different musical instruments. To be clear, this app does not attempt to create MIDI artwork that resembles the original image file.
i2sm was popularized in 2017 by the YouTube channel VineSauce, featured above, after he brutally mocked the app on a live stream that's reached more than half a million people.
A few other popular photo-to-sound apps, like Photosounder and Sonic Photo, came out around the same time as i2sm. They visibly scanned and sonified images in real time, presumably for the user's enjoyment. It's difficult to see what the utility of these apps would be, beyond noise music and possibly science.
Curious to see what kind of musical pixel art others have created?
Check out our previous article on the brief history of MIDI art and get our thoughts on where all of this may be heading.