esthesis explores the confluence of physical gesture, sound and visuals. The user generates a click, which is then captured by esthesis. Through physical gesture the initial impulse is sonically manipulated and visualised, allowing the user to extrapolate the sound and explore the synergetic relationship between their movements, the sounds heard and the visuals seen.
esthesis was funded by the Arts and Humanities Research Council as part of the Transforming Transformation project by Birmingham Conservatoire’s Integra Lab. For more information about the project see: integra.io/transforming-transformation/
This is the fourth experiment in a 1-year AHRC Digital Transformations project run jointly between Birmingham Conservatoire’s Integra Lab and Glasgow School of Art’s Digital Design Studios. The project explores new ways of positioning sound in space using touch-free interaction.
In this experiment we use the Microsoft Kinect as a motion capture device, enabling sounds can be dragged from a palette into a virtual 3D environment and moved around using direct physical manipulation.
Additionally, the system provides the capability to draw spatial trajectories with a simple finger movement and to automate the movement of sounds along the trajectories.
This initial experiment was developed in Unity 3D using the 3Dception plugin by TwoBigEars for binaural positioning.