Advances in Musical Robotics: From Animatronics to AI Robots

Remember the opening credits to Westworld, where the AI music robot performs the show's theme song, only to lift its fingers and reveal the piano playing itself?

This article is about the history of music robots and their gradual augmentation by artificial intelligence. We'll share some exciting new developments in this space and plenty of entertaining content to enjoy along the way.

Film critics have pointed out the importance of the player piano in Westworld. For the most part, it's just a prop during character dialogues. But in a few key scenes, the sheet music takes on greater symbolic meaning.

A player piano can only perform the notes that it's been fed. Much like software obeys its code and humans obey their biology and genetics, robots have been designed historically to carry out human programs.

Kurt Vonnegut’s classic novel, Player Piano, used the musical instrument as a metaphor for a "second industrial revolution", where workers are replaced by machines. Why hire a pianist when you could buy a piano that plays itself?

The problem of AI is more important today than ever. Some people feel that humanity's soul is at stake. Technology is accelerating at ever faster rates.

Content creator Mark Rober illustrated this beautifully with a recent video of Chopstix, the fastest player piano ever, powered by artificial intelligence.

Live Animatronic Bands

I grew up in an era where music robots performed on stage at Chucke E Cheese. These animatronic bands were abandoned in 2017. Apparently the live actors in suits were more popular than the robot dance troop. Oh, right, the band looks just as horrifying as I remembered.

Berlin artist Frank Barnes had a better idea. His spin on animatronics was closer to Gwar, playing punk songs with a Terminator frontman. Compressorhead has been a work in progress for over 15 years, adding two members to the band in 2017.

Kuka: An AI Robot that Also Plays Music

Frank Barnes wasn’t the only person making robot music videos in 2017.

That same year, composer Nigel Stanford released the Automatica music video, racking up 25 million views and making a big impression. Stanford was the visionary behind Cymatics, an equally popular video about sound vibration.

His short performance in Automatica centered on Kuka, a programmable AI robot with bulky claws. During a limited set of experiments, Kuka was programmed to play piano, guitar, cello, and other music instruments. The music portrays a positive collaboration between humans and musical robots.

To highlight Kuka's agility and speed, their marketing team also staged a battle against pro athlete Timo Boll. Instead of table tennis, they dueled on musical glass. The composition was recorded ahead of time to give it a polished look and feel. It's an entertaining bit of content that gained attention from the media.

Theatrics aside, Kuka really can perform on music instruments. This video shows it handling a cello, wielding a rotating arm to manipulate the stringed bow.

Swedish composer Fredrik Gran designed the ‘Robot Cellist’ to skillfully maneuver a bow across each cello string and even perform classical music. In the future, you can imagine how robots could be programmed to play entire symphonies.

But the next robot in this article, Shimon, is of an entirely different caliber. It's a musician's robot. This thing can skillfully improvise on an instrument and sing at the same time.

Shimon: The Improvising AI Music Robot

Shimon is an AI music robot designed by professor Gil Weinberg of Georgia Tech Center. Before we jump into his creation, check out this quick explainer video from Weinberg, addressing the question of whether or not robots can be creative:

As the director of Georgia Tech's center for music technology, Weinberg has overseen development of an interactive, marimba-playing music robot named Shimon. A feat of both mechanical engineering and artificial intelligence, Shimon's has been trained with algorithms to improvise and create original music during a live performance.

The video below showcases the first ever live set where an AI marimba player performed amongst humans. Amputee drummer Jason Barnes uses his robotic prosthetic arm to back up the band, further blending the line between man and machine.

Entertaining as the hip hop / jazz hybrid may be, Shimon’s talents is even more apparent during its live solo performances. A full album of material was released on Spotify in 2020, with a follow up EP in 2021.

In this video, Shimon performs marimba and sings at the same time. Staff from the institute of technology share details about how the robot was developed:

To learn more about the technical details behind Shimon, check out this Apple podcast interview with Gil Weinberg.

The bottom line is this. Eventually an AI music robot like Shimon could be mass produced. It would be tooled up with some generative AI music software, like OpenAI's Jukebox and Musenet, but fully integrated within their body mechanics.

It's not difficult to imagine these things syncing up together, like drone swarms, to improvise and coordinate musically with each other in real time.

Animusic: Animated Music Robots

Motion graphics artists have been dreaming up imaginary machine instruments for a long time. Animusic is arguably the most important animation studio to contribute to this vision.

With otherworldly designs and impressive visual mechanics, their official video release dates back to 2001, featuring compilations of short music robot animations.

These kinds of surreal music machines can be built in real life. A marble machine was created by Martin Molin using a purely mechanical system. It's closer to an instrument than a robot, representing yet another side of human inventiveness.

So what do you think about all of this? Are you excited by the prospect of strange new music robots emerging in the world? Does it threaten something important to the human side of creativity and culture -- and will these music robots be part of an advanced toolset that makes songwriting more efficient and enjoyable?

We'll report back in a few months as new technology surfaces in this space.

in