This reactive Koi Pond proof of concept was initially developed using Processing, SimpleOpenNI, & the Box2D library. The fish in the pond are "boids" which can be configured to interact with the digital silhouette of captured users by the depth sensor using blob detection (i.e. computer vision). The experience can be wall projected allowing the depth sensor (i.e. "Microsoft Kinect") to capture users passing by, allowing them to interact with the fish to disturb, or attract them in a feeding frenzy. This was one of several effects made for interactive projected wall experiences for demonstration purposes.
NOTE: The performance of the physics engine is noticeable in this video. However, the native version will be more responsive, and will be able to accommodate dozens of fish interacting with users.
For more information, please refer to my project page: