It's a track of the feature points in the footage so that I get not a moving camera as with normal camera-tracking, but a moving object in 3d space. Once I have that I can attach a face mesh to the track and project the footage on to that, so that I have something for the virtual 3d glasses to show behind them. The glasses are then rendered separately including the refraction, and put on top of the original footage. Plus a little contact shadow with an extra ambient-occlusion pass. That's pretty much it.

indeed, same here!