January 2006. frames2sound process. It took 14 hours of processing to generate. I used a trailer for a silly movie (I don't remember why this particular one -- I never even saw it).
The brute-force process involved:
1. Using QuickTime or ffmpeg, exported video frames (images) at 15 frames per second (= 1950 images).
2. Using ImageMagick, reduced size of each frame image to 40x15 ( = 600 pixels), and exported to .RGB format, which is each pixel's RGB color value, one per line.
3. For every frame in .RGB format, processed each pixel: mapped color value to a frequency number (black = 60Hz, white is 6000Hz), synthesized a square wave 1/15 seconds long, accumulated (mixed) them all together.
Thus for every frame, it produces a sound formed of 600 frequencies, lasting 0.0666 seconds.
These are appended to a sound file, which when complete, is synchronized with the video to produce what you see and hear here.
* Mapping the color value to a frequency was done with a simple C program I wrote.
* Generating the audio was done with sox, a cross-platform command line utility for audio ( sox.sourceforge.net/ ).