University of Limerick, Ireland, 2003-2004

Collaborators: Niall Griffith, Mikael Fernstrom, Interaction Design Centre

Over the last year (2003-2004) I have been working on a prototype for real-time audiovisual performance and composition called Miró using PD+GEM. In this prototype the gestural input comes through a Wacom tablet that allows the modification of a visual window by playing directly on it. The continuous representation of the gestures is recorded and mapped to some aspects of the user s marks such as the local velocity, pressure, X and Y coordinates. So far the system has three different sound generators (two variations of FM synthesis and Phase Aligned Formant) and three different graphics generators or tools (paintbrush, spray and fountain) that you can assign to any track.

In this way I have dynamic visual feedback in the first stage, i.e., when the user is interacting with the system by manipulating the stylus for the generation of audiovisual sequences. The gestures nuances in terms of trajectory, force and temporal variations (modulations) add expressiveness to the audiovisual material that is fed back in real-time. Then, in a second stage I can playback and organize this dynamic visualization. The outcome is a multiple dynamic visual representation of sounds synthesis generators in the same view.

It is possible to stretch the duration of each track and select sections within them. Also it is possible to playback the sequences in different ways such as: a) synchronizing them to a common metro that triggers each section according to its position on a timeline and b) according to its own period by creating loops. The second method of playback allows the user to improvise by changing on the fly different controls of the track panels such as slot selection, stretch factor, visual depth (zoom in/out)), colour, playback direction and the graphics and sound synthesizers. Obviously, these modifications can also be done off-line. For this purpose separated control panels for each track or sequence and a set of general controls have been implemented.

Miró has been used in concerts such as the multi-media performance Common Ground with the Bawnmore Ensemble of specially physically and mentally abled performers during the Sionna Festival in Limerick-Ireland in 2003, and in the interactive audiovisual piece Coloured Trip in the Music Technology Concert during the Digital Arts Week Now – DAWN04 at University of Limerick in 2004. It has been presented as interactive piece at the Artrónica Festival in Bogotá-Colombia and as a poster presentation at the International Conference on New Interfaces For Musical Expression-NIME04 in Hamamatsu-Japan in 2004.


Franco, E., 2004. Miró: a flexible expressive audiovisual system for real-time performance & composition. Master of Science by Research Thesis. Department of Computer Science and Information Systems, University of Limerick, Ireland.

Franco, E., Griffith, N.,Fernstrom, M., 2004. Issues for Designing a Flexible Expressive Audiovisual System for Real-time Performance and Composition. In Proceedings of the 2004 International Conference on New Interfaces for Musical Expression -NIME04, Hamamatsu, Japan, June 3-5, 165 -168.

Loading more stuff…

Hmm…it looks like things are taking a while to load. Try again?

Loading videos…