Music composed by Extraboy.
Macroblock studies: research in the digital material of film.
The lossy compressed video image is framed fundamentally different from analog or RAW video footage. First of all, the frames no longer rely upon raw pixels. Instead, macroblocks have become one of the elementary components of the lossy compressed moving image (at least under current standard codecs of, amongst others, the Moving Pictures Experts Group ‘MPEG’). Lossy compressed video often depends on luminance (brightness) and chrominance (coloring) thresholds arranged within 16x16 pixel (more or less) macroblocks within the keyframes (the I-frames) of an image sequence. The thresholds (or frequencies) of chrominance and luminance depend on an oscillating cosine function (following Fourier Transform).
Moreover, the material of the digital film is no longer based on a linear series of discrete images (a sequence); instead the video consists of different kinds of frames (I-frames or reference/key frames, P-frames or forward-predicted frames and B-frames or bi-directional frames), of which only the keyframe possesses a complete matrix of macroblocks. The frames between the keyframes (the P- and B-frames) consist of motion vectors that index only the difference in position (the offset) of the macroblocks between the original and the next frame. The handling of space and time within the video technologies is thus significantly different between the linear analog or RAW footage and lossy compressed footage.
Just like any other technology does, these new digital technologies posses their own specific artifacts. A couple of these artifacts are for instance “macroblocking”, “mosaicking”, “pixelating”, “quilting” or “checkerboarding”. And as is normal with any new technology, there is a growing group of (video) artist exploring these different artifacts of this new and constantly changing digital canvas.
A recently popularized wave of video artworks was based on the deletion of keyframes and the exploitation of the vector motion of P-frames. I described this current, which is dubbed datamoshing, pixel bleeding or just compression art, in some earlier posts.
A bit more obscure (partially because maybe the results are a bit less magical) is a related strategy that doesn’t rely on the deletion of keyframes and the exploitation of motion vectors, but on the exploitation/bending of chrominance and luminance algorithms of the Fourier Transform thresholds of the macroblocks. This method visualizes the otherwise mostly obscured macroblock structures of digital video.
The non-technical term macroblocking thus refers to compression/noise artifacts in lossy compressions, that shows up when for instance some data of chrominance/luminance is broken, the FTT cosine function itself is bended or some video image data is for instance edited in a hexeditor.
In "Demolish the eerie ▼oid" I exploit macroblocking, by opening a DV file in texteditor and playing around with the code. What became clear to me is that multiple things can happen – and I am not sure (yet) what pieces of code trigger what results. In one of the first tries I encountered some quite mystifying, but nice new organizations of color (chrominance) structures. The matrix of macroblocks is still completely intact but the chrominance values are ... off. The structure of the video is however still intact - you can still recognize me when you look very close/pay attention. The extreme color shift make it hard though. The matrix of macroblocks holds some puzzling but pleasantly surprising opportunities.
Extraboy used a rain vst, PIPPO vst and C64 noise to compose a complimentary soundscape.