You're right -- I started with 2d tracking on the rotation, but it didn't really work well because the divers came in and out of the frame and I couldn't get the settings for the tracking just right. So, in the end I did it manually. Also, I realize now: you need to track two separate motions, the motion of the camera and the motion of the diver. So first, you want to do stabilization based on the camera movement, and then do stabilization based on the diver movement. Look up "after effects stabilization" tutorials, and I'm sure you'll find what you're looking for if you haven't done that before.

Huge compliment Memo, thanks :)