[Edit, June 2016: To the best of my knowledge, this work with Arturo Castro is the first example of realistic "face swapping" in realtime. If you're interested in experimenting with the technique, I recommend Audun's implementation here auduno.github.io/clmtrackr/examples/facesubstitution.html ]
Working on an idea with Arturo Castro vimeo.com/29279198 I feel like "good" blending looks almost too natural to be surprising. It doesn't leave any interpolation up to your imagination. It's possible to push this style further, so it's less of a blend and more of a replacement, but then you get unnatural colors and shadows.