People playing darts at the Lark Tavern in Albany, NY.
Thank you for your patience, random serious people :)

This is my second video test with a Canon 5D Mark II, at a higher shutter speed this time, with the same Canon 50mm f/1.4. Once again, nothing interesting going on here, but an opportunity for me to describe my workflow in more details. Or lack thereof. Here are my first experiments taming the beast.

Update 12/11/09: Great news, Adobe's new playback engine, Mercury, looks fantastic and may just save us from those pesky proxies. Check this preview here: tv.adobe.com/watch/davtechtable/sneak-peek-at-the-new-adobe-mercury-playback-engine-technology/
Update 01/01/09: Augment the "Create Proxy Files" section with a comparison between several codecs and several frame sizes.
Update 30/01/08: Add a "About previews and pre-rendering" section.
Update 29/01/08: I found out how to auto-scale low-res proxy files to the project's frame size, and updated this document accordingly. This fixes one major issue.

The issue:
--------------------
As I described in my first video test (vimeo.com/2624143), the amount of data captured by the 5D is hammering both my computers at home (see hardware specs at the end of this document). At 1920x1080p 30fps, the H.264 codec used by Canon seems to push about 39 Mbps of video down the pipe. Ugh. While I can *replay* the clips nicely from Quicktime or Windows Media Player, I'm not able to perform any native editing from Adobe Premiere CS4; the first few seconds play fine, but it's a complete halt soon after that, even in Draft mode. Basic tasks like trimming assets, tweaking ends points, adding and previewing transitions are out of my reach.

While this could be blamed on the H.264 MainConcept codec used in Premiere (ffmpeg's x264 in WMP and Quicktime's H264 codec replay fine), and/or my specs, I'm doubtful I will be able to do any serious work on native 5D's video files at full-res anyway, short of buying a very expensive rig and/or a compatible H.264 hardware decoder card.

One solution:
--------------------
This is, however, not a new problem; people have been dealing with high resolution files in the past through clever use of proxy files and offline editing:
en.wikipedia.org/wiki/Offline_editing
From what I understand, proxy files are alternate versions of your original clips, either re-encoded for faster access and/or resampled to a smaller, lower resolution frame size (preferably with the same aspect ratio). Editing, pre-viewing and draft rendering are performed using those proxies; once you are happy with the results, the final high-resolution movie is created by first replacing the proxies with the original full-res clips. Easy enough.

H.264 is not a slow codec per se, but the bitrate in the full-res 5D files is really high, at about 38.6 Mbps, about the same as a Blu-Ray DVD, more than a HD DVD. I'm not going to re-encode 1920x1080p clips using a different codec at full-res; a faster decoder would only mean more (i.e. less compressed) data. Not only would I end up with a huge amount of video but I'm actually not even sure either my hard-disks or Premiere could sustain that much. Storage is cheap, but at 1920x1080 30fps, this is still a lot of image processing going on. Resizing the clips to a lower 640x360 frame size seemed the way to go in my situation, and I'll describe shortly how I created my proxies and switched between full-res and low-res.

Now if that whole proxy switcheroo seems to make reasonable sense to me, why is it so tedious to set up in Adobe Premiere CS4? Granted, I'm new to Premiere, but from what I gather on Adobe's "About online and offline editing" help page, I'm not the only one to struggle:
tinyurl.com/9jzdhk

My workflow:
--------------------
Here is quick overview of my workflow:
0) Create proxy files for your original 5D clips (I'll go over this in a separate section, let's just stick to the workflow).
1) Create a new Premiere project. The "New Sequence" video and audio settings should match the settings of your original assets, i.e. your native 5DMII files, 1920x1080p @ 30fps. Check my detailed "New Sequence: Canon 5DMII native settings" preset at the end of this document.
2) Drop a few native 5D clips in the project. Important: do *not* put them in the timeline.
3) Switch to offline editing.
a) In the left panel, select all clips, right-click and select "Make Offline...". No deleting the originals! The full-res clips are now disconnected from your project.
b) In the same panel, select all clips, right-click and select "Link Media...". This is our opportunity to replace the now-disconnected full-res clips with any low-res proxies created earlier. Note that if you put your proxies in a separate folder but keep the same file names as the original assets, Premiere will only ask you the location of the first proxy then find the others in the same folder automatically.
4) Adjust the clips settings. If your proxies are not at the same resolution (frame size) as your original assets, select all clips then select "Clip -> Video Options -> Scale to Frame Size". Remember that your project's frame size is locked and needs to match the native full-res format, i.e. 1920x1080p. Your low-res proxy files (say, 640x360) will show up fine in the source window, but they will be displayed centered within a huge (and mostly empty) 1920x1080 frame in the composite area. Selecting "Scale to Frame Size" will make sure your low-res assets are "upscaled" automatically to match the project frame size. If you don't do so, expect translating, scaling or any size-dependent manipulation not to work; a 10 pixels translation would not move the same relative area once you switch back to the 1920x1080p clips. Important: this option can *only* be set on clips that are not in the timeline yet; see step 2).
5) Edit the movie. At this point, this should be smooth-sailing. Add transitions, move clips around, trim, preview, etc., until your are ready to create the full-res movie.
6) Switch to online editing. Almost the same as 3a) and 3b), select all clips, make them offline, then link them back to the *full-res* native clips. The previewing area should look the same, Premiere should not even need to apply "Scale to Frame Size" since your assets now match the exact same settings as your project, 1920x1080p. Again, if your proxies and original assets are in different folders but share the same file names, Premiere should only ask you about the first one and find the others automatically.
7) Render the final movie. Select "File -> Export -> Media...", pick your export settings, start the rendering queue and go grab a coffee. In Columbia. Note that I do not export my movie to full-res, I render to a more friendly 1280x720p frame size that I can safely upload to Vimeo and display straight from my XBox 360 to my HD screen in the living room. Check my detailed "Export Settings: Canon 5DMII Medium Res (H264)" preset at the end of this document. FYI, it took about 12 minutes on my laptop to render this very basic 1 minute clip above.

That's about it so far.

This is still a little tedious for a few reasons:
- Switching between offline and online clips is a pain: if my proxy files do not have the exact same file name and extension as my original files, Premiere will ask me to locate each one of them, one by one, starting from the original file location. This gets old *fast*. Keep that in mind when creating your proxy files (my proxies are .avi when I use the Lagarith codec, native 5D clips are .mov), and try to put your proxies in a separate folder.
- Since there is no guarantee step 7) will be your final step, any last minute tweak will you send back to step 3) once more to work with proxy files, then 4), and 5), etc.

Once you get familiar with this workflow, you will notice that you can skip step 2) and 3) entirely and drop your proxy files directly to a new 1920x1080p project. Proxy files are, after all, what you will be working with most of the time. Only in step 6) do you need to switch your proxies with your original full-res files.

Conclusion:
--------------------
So here you go. Name your proxies carefully and this should work in most situations.

I'm a big fan of Adobe products; from a software engineering perspective, I think they are quite impressive. Premiere is a great product in that respect as well, so that's why I'm surprised offline editing remains a little tricky. What I wanted to see in Premiere:
- drop my native 5D files,
- right click on my clips, let me point to proxy files (and make it easy for me to do so), or flag the original assets as "I need a proxy here" so that Premiere batch-queue the selected clips and render the proxies automatically,
- when it's time to render the final movie, present me with a checkbox that lets me use the proxy files or the native files *for all clips involved in my sequence*; i.e. do not force me to switch back and forth between low-res and full-res, on and on.

Update: Adobe After Effects seems to be doing it exactly right.
tinyurl.com/7lewou

Hopefully I'll be able to refine this process in the future, or not use Premiere, I'm not quite sure yet. Project files are in XML format, I guess one could write a script to replace proxy files with native files automatically. Premiere itself is scriptable directly at run-time, but I've been burned before by scripting on Mac. The Premiere SDK is available for free as well and an export plugin could be created to work directly on a proxies-based project and export a new project using the corresponding native files. I'm not looking forward to any of this :)

Good luck. Feel free to add any comments/updates.
A link to this article was posted to several forums:
cinema5d.com/viewtopic.php?f=29&t=364
dvinfo.net/conf/canon-eos-5d-mk-ii-hd/140600-workflow-premiere-cs4.html
forums.dpreview.com/forums/readflat.asp?forum=1032&thread=30493944
forums.dpreview.com/forums/readflat.asp?forum=1032&message=30493876
adobeforums.com/webx/.59b76728

Here are a few threads to follow as well:
adobeforums.com/webx/.59b755fb
adobeforums.com/webx/.59b75c51
elementsvillage.com/forums/showthread.php?t=43135
forums.dpreview.com/forums/readflat.asp?forum=1032&message=30298160

The rest of this document: creating proxy files, a note about previews and pre-rendering, my hardware specs, detailed Premiere export settings...

Creating Proxy files:
--------------------
Creating proxy files in batch using Adobe Media Encoder is pretty easy. Launch the program, drop your original 5D files, select the whole batch and pick an export preset. If you have not created a preset so far, just select one file, click on "Settings...", adjust your settings, save them as a preset and apply this preset to all remaining files. Start the rendering queue and go get a book.

What settings to choose for your proxy files may vary. Check my detailed "Export Settings: Canon 5DMII Low Res Proxy (Lagarith)" preset at the end of this document. Even if the native 1920x1080p clips were uncompressed, I don't think Premiere could edit them fast enough in real-time, even in Draft mode: I chose to downsize my files to 640x360 (1/9th). Make sure you keep the same aspect-ratio. I also picked a codec that performs well for Non-Linear-Editing: Lagarith.
lags.leetcode.net/codec.html
compression.ru/video/codec_comparison/lossless_codecs_2007_en.html

My nine original 5D clips weighted a total of 366 MB. My proxy files weighted about 273 MB. Not *that much* smaller for a frame size 1/9th the original frame size, but keep in mind that this is a fast lossless codec. I sacrificed some disk space for faster decoding, though I could probably go lossy. I actually would like to use a Quicktime codec to keep the same file extension; feel free to pass any suggestion. Check this thread as well: adobeforums.com/webx/.59b755fb

Update 01/01/09: since I wasn't satisfied with using AVI files for proxies, I converted one original 79 MB (16 s.) clip using several codecs at several frame size, and checked if I could achieve smooth playback from Premiere CS4.

In the tables below, each row is read as:
frame size | container type | codec (bitrate): size MB (size %) | smooth playback? | comments
where QT stands for "QuickTime format/container", AVI for "Microsoft AVI format/container", P2 for "P2 Movie" (aka DV, DVCPRO, etc).

Here is the original 5D Mark II file:

1920x1080 | QT | H.264 (38.6Mbps): 79 MB (100%) | no |

Here are 8 different proxy types at full-resolution, sorted by increasing file size. Premiere CS4 was not able to playback any of them smoothly on my laptop; 1 wouldn't even play (ffmpeg x264 codec); 4 looked pretty bad just by checking still frames.

1920x1080 | AVI | x264 (6Mbps): 12 MB (15%) | no | corrupted, won't play
1920x1080 | QT | H.264 (6Mbps): 12 MB (15%) | no |
1920x1080 | QT | Video: 46 MB (58%) | no | look bad
1920x1080 | QT | Motion JPEG A/B: 55 MB (70%) | no | look no good
1920x1080 | QT | Cinepak: 60 MB (76%) | no | look horrible
1920x1080 | AVI | PicVideo MJPEG3: 198 MB (250%) | no |
1920x1080 | QT | JPEG2000: 230 MB (291%) | no |
1920x1080 | AVI | Lagarith: 379 MB (480%) | no |
1920x1080 | QT | Animation: 535 MB (677%) | no | look horrible
1920x1080 | AVI | ACDV: 678 MB (858%) | no | look corrupted

Here are 5 different proxy types at half-resolution. Good news, 4 of them playback pretty decently and could be reasonably used as proxies.

960x540 | QT | H.264 (6Mbps): 12 MB (15%) | YES |
960x540 | AVI | ACDV: 31 MB (39%) | YES |
960x540 | QT | JPEG2000: 34 MB (43%) | no | degraded and choppy
960x540 | AVI | PicVideo MJPEG3: 68 MB (86%) | YES |
960x540 | AVI | Lagarith: 119 MB (150%) | no | very choppy

Here are 5 different proxy types at one-third-resolution. Not surprisingly, all of them playback smoothly and could be used as proxies, though the JPEG2000 one looked very degraded.

640x360 | QT | JPEG2000: 7 MB (9%) | YES | look very degraded
640x360 | QT | H.264 (6Mbps): 12 MB (15%) | YES |
640x360 | AVI | ACDV: 19 MB (24%) | YES |
640x360 | AVI | PicVideo MJPEG3: 40 MB (50%) | YES |
640x360 | AVI | Lagarith: 63 MB (80%) | YES |

And finally the P2 format (i.e. DV, DVCPRO, DVCPROHD, DVDPro100), which stands in-between as it horizontally compresses recorded images to 960x720 pixels for 720p output (i.e. the pixels are not "square" anymore but this displays just fine). I guess this format is bitrate-oriented as well since both 960x720 and 720x480 output produced the same file size.
en.wikipedia.org/wiki/DV#DVCPRO
Note that converting to this format creates a folder hierarchy containing various files and both audio/video streams separated; just drop the video stream found in CONTENTS/VIDEO to Premiere. Do not expect to be able to read the .MXF video stream directly from Windows Media Player.

960x720 | P2 | DVCPro100 720p 30: 115 MB (145%) | YES | fine but scrubs fast
720x480 | P2 | DVCPro50 NTSC: 115 MB (145%) | YES

What to pick, what to pick? Again, keep in mind that this is for a specific laptop configuration, though the tables above could be used to sort codecs relative to each other. So far I had used the Lagarith codec at 640x360, but if I stick to this resolution I may as well use ACDV or QuickTime H.264 to save some space. More importantly, a QuickTime codec would make it much easier for me to switch between proxies and original assets, since they could share the same file names in separate folders.

If I choose a higher-resolution, say 960x540, I could once again use either ACDV or QuickTime H.264, preferably the later. On my widescreen monitor, the sub-windows used inside Premiere to display either the source assets or the preview area are about 660x370; a 640x360 proxy would display just fine.

For the sake of completion I created a few new 960x540 proxies using QuickTime H.264 at 3 different bitrates: 3, 6 and 12 Mpbs. I could not really notice any difference inside Premiere, they all played back reasonably smoothly.

960x540 | QT | H.264 (3Mbps): 6 MB (8%) | YES |
960x540 | QT | H.264 (6Mbps): 12 MB (15%) | YES |
960x540 | QT | H.264 (12Mbps): 24 MB (30%) | YES |

Note: if you are looking for a free batch converter, I can suggest Virtualdub, which I'm a big fan of, but I spent an unhealthy amount of time trying to get it to process native 5D files, without success. I've read H264 can be opened from Virtualdub, I just doubt 5D files specifically can. I upgraded my Haali Splitter, my ffmpeg distribution, downloaded the Quicktime Virtualdub plugin, no dice. This would either display a green screen and crash, or fail to render the movie properly besides the I-frames.
virtualdub.org/
tateu.net/software/
forum.videohelp.com/topic359864.html

About previews and pre-rendering:
--------------------
Update 30/01/08: Some people suggested I preview (pre-render) the timeline (or pieces of my timeline). I can, but this is certainly not the same. I quote the doc: "Areas that can’t be played at the project’s full frame rate are indicated by a red line in the time ruler. To play these areas, you can set the time ruler’s work area bar over the red preview indicator and render a preview file. This renders the segment as a new file on the hard drive, which Adobe Premiere Pro can play at the project’s full frame rate".

Such previews are pieces of your *timeline* that are pre-rendered *including* all effects. Proxies, on the other hand, are original clips in the assets list. What I mean here is that I can use the same clip several times in my timeline, with different effects applied: if this asset doesn't playback originally, even without any effects, then the corresponding two instances of this clip won't either. I can certainly preview (pre-render) my timeline for both instances of this same clip and achieve smooth playback that way, but that implies I need to pre-render twice (one for each instance), and that I will have to pre-render again and again each time I change the timeline settings, effects and transitions on those two instances.

On the other hand, if I use a good (fast enough) proxy, its contents will be used twice in my timeline and playback should be smooth for all instances of this clip, as long as I don't pile 10 effects on it of course. My point is: given the same amount of effects/transitions, it seems to me that a proxy might allow you to skip the pre-render step much more often. Pre-rendering works fine, I'm just trying to tackle the problem earlier in the workflow.

As far as I'm concerned, the situation is actually much worse than that: I can *not* playback or scrub my 5D clips in a timeline, even without any effects. If I had to use pre-rendering only, I would have to put all my high-res clips in the timeline, pre-render *all of them*, edit my transitions/effects, then pre-render again and again for each change, etc etc.

The problem for me here is that at full-res, the Preview format is just killing my computer. It doesn't provide any flexibility; previewing is essentially creating a less useful proxy file automatically, without the control on frame size and compression. That might be a non-issue on somebody else's machine, but on my PC the preview file format is set to Uncompressed UYVY 422 8bit. None of the few other formats are that much better either, the preview settings offer only a very small subset of the codecs you can use for a proxy.

I just tried again, dropping a single 80 MB file (16 s.) on my timeline and triggering a preview; it took *way* longer to create that preview than just batch-creating proxy files using Premiere's Media Encoder. Actually I had to give up after 10 minutes because Premiere had generated, out of this 80 MB file, something already larger than 1.6 GB. I just don't have the space/horse-power to store/edit files that size, but your mileage may vary. Also, let's face it, I do not need to edit at 1920x1280p; none of the sub-windows in Premiere uses anything close to that effective resolution. Granted, my masks would look a tad more accurate if I created them again a full-res frame than a one-half or one third proxy, but I could use a larger proxy for specific cases...

---------------------------------------------------------------------------
My specs:
Laptop: Dell Latitude D630
- Intel Core 2 Duo T7700 @ 2.4 GHz,
- 4 GB RAM,
- Windows Vista 32 bits
- 64GB Solid State Drive :(

Desktop: Dell Dimension 8400
- Intel Pentium 4 @ 3.2 GHz,
- 2 GB RAM,
- Windows XP 32 bits,
- 150 GB Raid 1 (probably @ 7200 RPM)

---------------------------------------------------------------------------
New Sequence: Canon 5DMII native settings
(matching the Canon 5DMII native video format as close as possible)
General:
* Editing mode: Desktop,
* Timebase: 30 fps
Video Settings:
* Frame size: 1920h x 1080v (1.0000),
* Frame rate: 30 fps,
* Pixel Aspect Ratio: Square (1.0),
* Fields: No Fields (Progressive Scan)
Audio Settings:
* Sample rate: 44100 Hz

---------------------------------------------------------------------------
Export Settings: Canon 5DMII Low Res Proxy (Lagarith)
(matching the Canon 5DMII native video format as close as possible, except for the frame size downsampled to 640x360)
* Format: Microsoft AVI
Filters:
Video:
* Video Codec:
- Video Codec: Lagarith lossless codec,
- Codec Settings: Always Suggest RGB, Mode = RGB, Use Multithreading (that codec performs better in RGB, but YUV would do fine)
* Basic Settings:
- Quality: 100,
- Width: 640,
- Height: 360,
- Frame Rate: 30,
- Field Type: Progressive,
- Aspect: Square Pixels (1.0),
- Render at Maximum Depth: No,
- Depth: 24 bit
* Advanced Settings:
- Expand Stills: No
Audio:
* Audio Codec: Uncompressed,
* Basic Audio Settings:
- Sample Rate: 44100 Hz,
- Channels: Stereo,
- Sample Type: 16 bit,
- Audio Interleave: 0

---------------------------------------------------------------------------
Export Settings: Canon 5DMII Medium Res (H264)
(matching Vimeo and XBox 360 recommended format, 1280x720)
* Format: H.264
Filters:
Multiplexer:
* Basic Settings:
- Multiplexing: MP4,
- Stream Compatibility: iPod (IMPORTANT for Xbox 360 compatibility)
Video:
* Basic Video Settings:
- Codec: MainConcept H.264 Video,
- TV Standard: NTSC (I'm in the US)
- Frame Width [pixels]: 1280,
- Frame Height [pixels]: 720,
- Frame Rate [fps]: 30,
- Field Order: None (Progressive),
- Pixel Aspect Ratio: Square Pixels,
- Profile: High,
- Level: 3.1
* Bitrate Settings:
- Bitrate Encoding: VBR, 2 Pass (slower, but recommended)
- Target Bitrate [Mbps]: 6 (original 5DMII is at 38.6, BluRay 40, HD DVD around 36)
- Maximum Bitrate [Mbps]: 10
* Advanced Settings:
- Set Key Frame Distance: No
Audio:
* Audio Format Settings:
- Audio Format: AAC
* Basic Audio Settings:
- Codec: AAC,
- Output Channels: Stereo,
- Frequency: 44.1 kHz,
- Audio Quality: High
* Bitrate Settings:
- Bitrate [kbps]: 224 (your mileage may vary)
* Advanced Settings:
- Precedence: Frequency (not sure about this one)