After a few insane results, I finally managed to get this CAT rig into MotionBuilder, apply a test animation, and get it back into 3ds Max. CAT, despite being a tool that's been around for years has the worst documentation and the least amount of useful tutorials or walk-throughs ever. It doesn't help that it's been intensely buggy since moving to Max from SoftImage, and that most of the features I'm interested in using have been revised for the latest version of Max. Surely somebody has used this pipeline before. I chose CAT because I thought the powerful nature of the preset rigs would help save time and that the animation layers would be a useful way to work, but after all this hassle, I'm not so sure. Maybe I should have stuck with the custom PEN-style rig.
I didn't spend any time tweaking the animation, just slapped it onto the skeleton and let it go. There's obviously some clipping, and the guy ought to ball up his fist if he intends to do any damage there.
I've discovered there is something drastically off about how I rigged his toes -- on solid ground, his toes shouldn't bend downward like that. I'll obviously have to adjust them.
The animation was originally rendered out at 24fps, but I cut it to 12fps in AE to closer approximate traditional cel drawn animation. What's missing in this approach is deformation to simulate motion blur, leaving the image too pristine.
Now that I've seen this workflow SORT of work, I can move forward with the two next challenges:
1) Rigging his elbow pads and knee pads. I'll have to create some sort of helper that floats between the upper and lower legs, and constrain the knee pad to that.
2) Skinning his face. The basic points are already there, but I'll have to figure out exactly how I SHOULD skin and weight the face (using standard max bones between CAT bones as nodes, or just creating a lot of floating CAT bones and then working on constraining those). Then I'll have to figure out how to use the actor face and character face features in MoBu.