The Vimeo Blog

More Posts

Keyframes, inter frame, and live video compression

Michael Cohen
September 26, 2017 by Michael Cohen Staff

So you’ve spent weeks — maybe months — running production, writing scripts, and getting everything set for your live broadcast. You’ve rehearsed, and you’re ready to knock this out of the park. Then, as cameras are rolling, the whole thing looks distorted. All because of the seriously important, seriously complicated process of live stream compression.

Compression is the most important technical element in a live stream, and it can also be one of the biggest pain points, if not executed correctly. Understanding how it works in regards to your live stream is absolutely crucial. I’m going to walk you through some live compression basics,  and also breakdown the troubleshooting process should you ever need to fix a shaky stream while it’s live.

A little background
There’s a lot going on under the hood when you hit that Start (live stream) button. What we experience today is the result of over 20 years of hard work. The first live streamed concert was in 1994. Take a look at what live online video was like before modern compression.

So what is compression? Like the name suggests, compression is all about making your file size smaller, and thus easier to transport and store.. This is critical for live streaming because RAW HD video is massive.

Here’s where compression comes in. Transferring video over the internet from source, to streaming platform, to all the viewers’ machines takes tons and tons of bandwidth. In the past two decades, video compression technology has steadily improved to the point where it is now possible for broadcasters to live stream HD video to millions of viewers, on a variety of devices.

With these advancements came a ton of variables to the process, including many different types of compression itself. The most important by far is...

The mighty MPEG-4
This is the current industry standard for video compression. Everything from Blu-Ray discs, to terrestrial TV, satellite, and, of course, online video platforms like Vimeo use this compression standard to quickly process and transport video. Also called H.264, this technology was created by a group assembled by the ITU (international telecommunications union), and has since been used on nearly all major video platforms. This standard is the one responsible for the rampant growth of the digital video market, as well as the commercial explosion of live streaming.

In order to understand how this compression works, we must explore what exactly is being compressed. If you’re using H.264 for f compression, the video itself doesn’t shrink at all. The way H.264 decreases file size is actually by increasing processing efficiency. How does it work? Well, through algorithms! It all works based on a special type of algorithmic prediction called...

Balancing inter frame prediction
These algorithms use motion prediction to pre-load objects across multiple frames of video. Basically, once an object is completely loaded on one frame, the algorithm identifies the common elements from that frame which appear on the following frames. Using that information, the algorithm then loads the following frames more quickly, and with less processing power than it took to load the initial frame. This creates a system where the only frames which are fully processed are the key frames. The frames which load more quickly and take information from key frames are called delta frames. Having a ratio of nearly six delta frames for every one key frame allows the overall file to be much smaller in size, while still containing a detailed, fluid HD video track.

Encoder settings have an option called “force key frame” and it’s usually measured in frames per second (fps). This setting determines how often the encoder forces a frame of video to be loaded fully without prediction. This interval greatly affects both the quality of the video and the stability of the stream itself.

The balance between forcing (and not forcing) key frames can be a little tricky. On the one hand, forcing key frames too often will result in a stream which takes far too much processing power, thus difficult to transfer, and more likely to crash. On the other, forcing a key frame not often enough causes the prediction algorithm to operate inefficiently; potentially inaccurately predicting elements, which causes the video to appear distorted.

Finding the optimal “force key frame duration” is necessary for producing a seamless live stream. Best results here are to keep your force key frame duration divisible by your fps. For example, a video with a frame rate of either 30fps or 60fps, force a key frame every 2 seconds.  For most even number fps, 2 seconds is your best bet. The only time to use an odd number would be if you have a fps of, say, 25. In this case I’d advise using a force key frame duration of 5 seconds.

How to spot (and troubleshoot) compression issues
While live streaming your event, if you notice portions of the frame becoming pixelated, this is a compression issue called “video artifacting.” You can distinguish artifacting from typical pixelation because only part of the frame will be distorted. This is because during the algorithmic prediction process, elements are being loaded incorrectly and the motion prediction is carrying this mistake across multiple frames. This is the type of live-specific issue that requires a stream to be stopped in order to fix. In order to minimize downtime, having a redundant back-up broadcaster PC is ideal. That way, once the affected stream is stopped, you can switch on the back-up and reconnect for only a quick second of down time. You’ll want to make this switch during a lull in the action, commercial break, intermission. But, if you don’t have a break in the action, make this switch ASAP.

Troubleshooting a compression issue is usually fairly simple. The first step is to see if the machine’s CPU is going crazy. Checking for this is easy: go to your machine’s Task Manager, (or on a Mac, Activity Monitor). Note the percentage of CPU usage. For a healthy machine, anywhere from 15%-75% is fine. Once a machine hits 100%, it is bound to crash. Check in that same tool what else is running on the machine. Usually other programs updating in the background can cause the CPU to spike. Once this is done, and you’ve made sure there is nothing else running on the machine, reboot, and start your live stream up again. At this point if the back-up PC you switched over to is still working fine, I’d recommend staying on that for the duration of the your live stream.

What the future holds
The high demand for digital video is growing rapidly year after year. Live streaming HD video may look great to us now, but, as history tells us, there is always a new-and-improved format just around the corner.

The most likely successor to MPEG-4 is drum roll... MPEG-5. Also called HEVC, this compression standard is based on all the same concepts which made MPEG-4 so successful. It also uses prediction algorithms, but makes some significant efficiency improvements —  handling double the data MPEG-4 can, while processing at the same bitrate, with much higher resolution. With this standard, processing 4K resolution video is not only possible, it’s easy. In fact, even 8K resolutions can be processed using this format.

Only time will tell if MPEG-5 will become as widely adopted as it’s predecessor. But one thing is for certain: compressing video is a need that we will continue to have for a long time.

There you have it, live compression, defined. If you want to get into the specifics of some of the key words (and frames) mentioned above, cruise through our live glossary.

1 Comment

This conversation is missing your voice. Please join Vimeo or log in.