[Edit, June 2016: To the best of my knowledge, this work with Arturo Castro is the first example of realistic "face swapping" in realtime. If you're interested in experimenting with the technique, I recommend Audun's implementation here auduno.github.io/clmtrackr/examples/facesubstitution.html ]
Working on an idea with Arturo Castro vimeo.com/29279198 I feel like "good" blending looks almost too natural to be surprising. It doesn't leave any interpolation up to your imagination. It's possible to push this style further, so it's less of a blend and more of a replacement, but then you get unnatural colors and shadows.
FaceTracker library from Jason Saragih web.mac.com/jsaragih/FaceTracker/FaceTracker.html
ofxFaceTracker addon github.com/kylemcdonald/ofxFaceTracker
Download the app here github.com/arturoc/FaceSubstitution