Encoding Ladder: Choosing the Best Codec for Streaming Video

This is the first in a series of posts detailing what you need to know to create the perfect encoding ladder. It’s a high-level introductory series good for streaming novices, but too basic for most experienced streaming producers. This article details how to choose the best codec for streaming video. 

What’s a codec?

A compression technology that shrinks your video to bitrates that you can deliver to your target viewers. All video streamed over the internet is compressed with a video codec (and all audio is compressed as well).

Best Codec for Streaming Video: The Short Answer

Choose H.264. It’s a rough guess, but probably 99% of all websites that directly stream their own videos (rather than using a service) use H.264 only. We love to talk about new codecs, but few websites actually deploy them.

If you’re distributing 4K videos with high dynamic range (HDR), you probably will also encode with HEVC which is the format of choice for HDR videos.

Factors To Consider When Choosing A Codec

Here are four of them.

  1. First, choose a codec that plays on as many of your targets as possible. That’s why H.264 is the simple answer; it plays on virtually all devices that connect to the Internet, whether a computer (Windows/Mac/Linux), mobile (iOS/Android), or living room (dongles, set-top boxes, and smart TVs).
  2. Beyond H.264, you should consider adding a codec when it enables your company to enter a new market. For example, HEVC became popular because of 4K video delivered to the living room, particularly once high dynamic range (HDR) technologies became available.
  3. Very few publishers add codecs strictly for the bandwidth savings that they deliver. Though AV1 is roughly 50 – 60% more efficient than H.264, and 20 – 30% more efficient than HEVC, very few companies use it.
  4. Other factors to consider are encoding cost and playback support.

Why the hesitation in adding new codecs?

At least for the foreseeable future, you must encode to H.264 to reach all target viewers – it’s the only format that all connected formats can play, especially older codecs. So, encoding with a new codec will add several costs, including:

  • The cost of encoding to another codec. Since new codecs are generally more complex than H.264, they cost from 2x to 10x the cost to encode.
  • The cost of testing all players for compatibility. You simply can’t assume that the codecs will play on your target platforms; you have to test to make sure, and then resolve any problems.
  • Reduced caching efficiency. Most large publishers distribute videos through content delivery networks (CDNs). CDNs have servers around the globe to get videos closer to the viewer to improve delivery efficiency. These videos are stored in memory caches on these “edge servers,” and the most requested videos are stored the most often. If you encode in multiple formats, you reduce the likelihood that videos will be stored in the cache, increasing delivery cost and potentially reducing the quality of delivery to your viewers.

Hopefully, these costs will be offset by the savings delivered by implementing a new codec, typically realized through bandwidth savings. Fortunately, bandwidth costs have plummeted over the last few years, which reduces all delivery costs. However, it also reduces the financial benefit of implementing a new codec.

To explain, if you save 50% by implementing AV1, and the cost per GB is $0.50, you save $0.25 for each GB you deliver. If the cost is $0.085, the highest price Amazon charges for its Cloudfront CDN, you save $0.0425. This means you have to stream about 6x the video to achieve the same savings at the $0.50 pricing.

For a complete discussion of these factors, check out the article Computing Breakeven on New Codec Deployments. The article includes a link to a Google sheet with a break-even calculation.

How do I Choose the Codec?

Choosing the codec is a fundamental configuration option in all encoders. With FFmpeg, which we’re using as our example, -c:v libx265 chooses the codec (in this case, the x265 codec, which is HEVC) while -c:a aac chooses the audio codec, in this case, Advanced Audio Coding, which is the default codec for MP4 files. MP4 is a container format which we’ll tackle later.

Virtually every program with a GUI provides the ability to choose your codec. Here’s where you choose a codec for streaming in the the free tool HandBrake,

Choosing a codec for streaming in Handbrake.
Figure 1. Choosing a codec for streaming in Handbrake.

Here’s how you choose a codec for streaming with AWS MediaConvert, Amazon’s cloud encoding tool.

Choosing a codec for streaming in AWS MediaConvert.
Figure 2. Choosing a codec for streaming in AWS MediaConvert.

With this as background, here are a couple of videos that take the conversation about newer codecs much further. This particular discussion covers AV1, HEVC, and H.264, with representatives from Netflix and Warner Media sharing their practices regarding newer codecs, while Bitmovin and AWS discuss their product offerings.

This is a good background on newer codecs like AV1, VVC, LCEVC, and EVC.

Resources:

Choosing a Codec in 2021 and Beyond, YouTube, October 2022.

The Video Codec Landscape 2022, Wowza blog, May 2022.

Computing Breakeven on New Codec Deployments, Streaming Learning Center, February 2021

Real-World Bandwidth Savings from VP9, HEVC, and AV1, Streaming Learning Center, June 2021

About Jan Ozer

Avatar photo
I help companies train new technical hires in streaming media-related positions; I also help companies optimize their codec selections and encoding stacks and evaluate new encoders and codecs. I am a contributing editor to Streaming Media Magazine, writing about codecs and encoding tools. I have written multiple authoritative books on video encoding, including Video Encoding by the Numbers: Eliminate the Guesswork from your Streaming Video (https://amzn.to/3kV6R1j) and Learn to Produce Video with FFmpeg: In Thirty Minutes or Less (https://amzn.to/3ZJih7e). I have multiple courses relating to streaming media production, all available at https://bit.ly/slc_courses. I currently work as www.netint.com as a Senior Director in Marketing.

Check Also

Rating techniques that cut bandwidth costs.

Five Codec-Related Techniques to Cut Bandwidth Costs

The mandate for streaming producers hasn’t changed since we delivered RealVideo streams targeted at 28.8 …

Single-Pass vs Two-Pass VBR: Which is Better?

Let’s start this article with a quiz regarding how the quality and encoding speed of …

My FFmpeg-Related New Year’s Resolution: Document Before I Test

My typical workflow for testing-related articles and reports is to create the command string, run …

Leave a Reply

Your email address will not be published. Required fields are marked *