Co-authors Mason Bretan and Gil Weinberg discuss "A Survey of Robotic Musicianship" (cacm.acm.org/magazines/2016/5/201594), their Review Article in the May 2016 CACM.
00:00 It's not about the notes. A record can play the notes.
00:11 It's about giving, and taking, and listening, and watching, and learning. That's music, the most human of expressions.
00:19 But does music truly belong only to human musicians?
00:24 Join us as two researchers describe how mechatronics, artificial intelligence, and human musicians combine for a new kind of creation, in A Survey of Robotic Musicianship.
00:36 [Intro graphics/music]
00:46 The Georgia Institute of Technology is famous for its engineering programs. Less well known is its school of music.
00:56 This is where Ph.D. student Mason Bretan works with Professor Gil Weinberg to build and teach robots about musicianship.
01:06 MASON BRETAN: So we have these two separate things which focus on the intelligence and then the actual physical actuators that generate the sound. And then these two things combined is the study of robotic musicianship.
01:18 The history of musical robots goes back over 2,000 years.
01:23 GIL WEINBERG: If you go back to Archimedes, there are efforts of actually trying to have an automaton that plays music. And they used, back then, they didn't have computers, so they used pneumatic and hydraulic. ... But all of these efforts since Archimedes time -- maybe even before that, we don't know about -- were basically following rules.
01:41 MASON BRETAN: And I think that's really the key difference between some of these previous musical robots and robotic musicians, is that there's this underlying musical intelligence that allows it to make informed musical decisions and actually respond in real time to interacting musicians.
02:02 GIL WEINBERG: A catchy phrase which I try to capture what we are doing is, "Listening like a human, and improvise like a machine."
02:10 To encourage interaction, Bretan and Weinberg's own robots are anthropomorphic, tapping their feet and bobbing their heads, just like their human counterparts.
02:20 GIL WEINBERG: We took one step further in the whole idea of embodiment, and trying to have an automated and animated embodiment that will enhance the music performance from a musical point of view, but also from engaging and enjoyment point of view.
02:37 But while they're anthropomorphic, such robots aren't necessarily humanoid. That fact fundamentally affects their musical decisions.
02:46 MASON BRETAN: So you have this joint optimization between its body and then what it knows about music. And so each musical decision is informed not only by what it knows about the music, but also about just Shimon's design. If we had designed Shimon with a different set of physical constraints -- maybe ten arms instead of the eight that it currently has -- the output would be different. And if it only had one, again it would be different.
03:10 Bretan and Weinberg believe this field of study could advance areas beyond musical creativity.
03:17 MASON BRETAN: Because there's a lot of challenges within robotic musicianship that touch on, like, timing and expression; social interaction; mechanical dexterity. All of these different things apply to robotics in general.
03:32 GIL WEINBERG: People understand improvisation to be something important ... for general understanding of human intelligence.
03:39 Find out more in the review article, "A Survey of Robotic Musicianship", in the May 2016 issue of Communications of the ACM.
03:49 [Outro and credits]