The Robotic Voice Activated Word Kicking Machine is a surreal exploration of language and our strange relationship with talking to machines, from customer service bots to "intelligent assistants". It combines projection and robotics to explore the crossover between the virtual and the physical. Viewers’ spoken words are converted into text and launched into the virtual world. They accumulate there, sometimes kicked by a robotic foot and sometimes sucked back out into the world as sound.
The physical part of the piece consists of four bells of baritone horns, tubing, microphone, speakers, a mechanical foot and a projector. The software is written in openFrameworks.
When the software starts, it launches a web browser and creates a WebSocket connection to it. When someone starts speaking, the software sends a message to the browser to tell it to start speech recognition. Once the person finishes speaking, the browser sends words back as text back over the WebSocket connection. Each word is then added to a Box2D physics simulation as a series of rectangles linked together with springs. There’s a mesh for each letter and shadows are created in GLSL with a shadow map.
When a word is near the foot, the computer sends a message to an Arduino telling it to kick. The foot is actuated with a linear actuator driven by a Pololu Dual VNH5019 Motor Driver Shield. There is also a foot polygon in the physics simulation. Every time the mechanical foot starts to kick, the virtual foot in the physics engine is animated with a timeline that has been matched to the actual movement of the mechanical foot.