This paper will present the sound sculpture “Visible Canon: String Quartet without human beings” (2012) by Yasuhiro Sakamoto and Iñigo Giner Miranda. Within the bildwissenschaftliche and neuro-aesthetical framework of the work it explores aspects of uncertainty and embodiment in the process of perception. In addition, it touches upon the uncertainty of audio-visual perception as well as the workings of psychological time.

It is well known that a cross-modal perception of auditory and visual stimuli does not equal a mere summation of individual auditory and visual perceptions. As with the Gesamtgestalt of the perceptual mechanism, sounds are not apprehended individually, but shaped as a Tongestalt, in which auditory and visual stimuli are combined audio-visually, similarly to the perception of auditory apparent motion.

In recent scientific research, audio-visual perception has been usually conceptualised in relation to synaesthesia. However, the notion of synaesthesia refers to individual nerve structures, which do not encompass the implications of Gesamtgestalt. In fact, the latter describes a selective sensory property within the perceptual process, which can be observed in every human being. Recent neurobiological studies seem to underline the universality or non-individuality of cross modal perception. Therefore, we are looking for an alternative hypothesis that explains what happens, when auditory and visual stimuli are perceived at once.

Naturally, it is easy to conduct a perceptual study that records what is shown [input] on the one hand and what is perceived [result] on the other. If the results are clear, it may be explained how they are perceived [process] as well. However, it is not as easy to explain why they are perceived in a specific manner. There is certainly a “rule” of perception such as Gesamtgestalt, yet, it remains unfathomable as long as we observe human perception in relation to the time of physical world alone. At this point, the sciences encounter a certain limit to its applicability to the investigation of the psychophysical sensory properties within audio-visual perception.

In the case of motion, perception follows a horizontal timeline according to the physical continuity of motion. However, sound features a physical continuity as well: It moves both on the horizontal and the vertical timeline. While listening to music, the audience follows automatically, but the brain perceives the structure of music congruently in temporal and constitutive in non-temporal. Also, each sounds, articulation, melody and rhythm are is perceived in relation to a prior and consequent element of the same structure. This, in turn, indicates that during the experience of music our senses enter another world, where present, past and future coexist simultaneously, which further complicates our understanding of audio-visual perception and the ruling mechanism behind it.

All of this leads to the question of how to figure the Gesamtgestalt of perception. In our approach we focus on the cutting point of audio-visual perception and try to embody a cross-modal Gesamtgestalt that links sound (=tone) and vision (=motion) in the perceptual process. In this context, the art-machine “Visible Canon” is conceived of as an experimental device for the exploration of audio-visual perception.

The machine looks like a rolling ball structure that plays music and works like a three-dimensional chart. It plays three tracks of various Canons, including Johan Pachelbel’s Canon, and displays the kinetic movement of an iron ball at the same time. Through the combination of the auditory and the visual structure of the canon the uncertainty and complementary relation of the stimuli calls up the Gesamtgestalt of audio-visual perception.

In the case of Pachelbel’s Canon, it might be less important to envision its structure visually and kinetically at the same time, since it will be perceived automatically as rhythm and melody. However, if a Canon has a more complicated composition, rhythm and melody cannot be followed as easily; in the process, the visual information provides the viewing audience with a powerful key to the understanding the underlying structure of the Canon on the basis of the cross-modal Gesamtgestalt.

With the above method we endeavour to embody the Gesamtgestalt of audio-visual perception. At the same time we have explored the new role of art in modern sciences that provides an experimental device able to overcome the shortcomings of the classical methods.

Loading more stuff…

Hmm…it looks like things are taking a while to load. Try again?

Loading videos…