Recently I enjoyed seeing Pip Chodorov present one of his films at the Star & Shadow cinema:

It demonstrates tight synchronisation of film and sound, and is really impressive. The construction of the film was several weeks worth of manual work careful synchronising and printing 16mm frame transitions in response to recorded audio.

It reminded me that an exactly equivalent digital workflow can be carried out much less painstakingly with MIDIO, as I demonstrated previously in this video for example:

In response to some questions from the audience, Pip stated that he thought that the ease with which digital effects can be applied makes video less interesting than film. He certainly has a point, it's trivial to quickly apply effects to footage with software editing suites. It's tempting to throw all of these tricks at a piece of footage and end up with a kind of mind-melting frantic collage, with no structure. In my defense, MIDIO videos are carefully built up pixel by pixel, frame by frame, note by note, sample by sample, but this is done programatically and automatically, scripting in Python (or earlier in C++), rather than using GUI video editors. MIDIO involves taking source video footage as input, feeding MIDI file instructions through a set of editing rules to produce an intricate series of edits on the source footage, to give a new video as output. I use the same MIDI file to produce a (digital) audio track. Combining the edited video and the generated audio results in a video where editing and audio are both generated from the same set of instructions, and are therefore intrinsically synchronised, which I find quite satisfying.

MIDI files represent the timing of beginning, end and volume of notes played in a piece of music. They are the digital equivalent of sheet music. MIDI files are also the audio equivalent of vector graphics. I would say a MIDI file is to a .mp3 file as an .eps or .pdf file is to a .jpeg. MIDI is a set of instructions for constructing a piece of sound (music), rather than a sound recording. Vector graphics are a set of instructions for drawing a picture rather than a pixellated raster image. MIDO abuses MIDI and uses it to create sound AND to edit video, which then happen to be in synch.

The MIDI file used to edit this video (and generate the music) is John Roache's arrangement of Scott Joplin's Maple Leaf Rag:

This video uses only the right-hand notes from the Maple Leaf Rag to generate transitions.

Loading more stuff…

Hmm…it looks like things are taking a while to load. Try again?

Loading videos…