I came back to this project:
vimeo.com/1600741

Back then the lighting was pretty much fake since the normals where incorrect. I loved the idea and fun factor of the project though, so I could not resist tweaking it a little bit more.

To get correct lighting when using displacement maps there is no way to get around computing a normal map from it. I converted the shader that can be found here:
catalinzima.spaces.live.com/blog/cns!3D9ECAE1F2DC56EF!223.entry
to GLSL to compute a normal map from the webcam image on the fly. I am really loving the results and I think people would have massive fun interacting with it.