Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
A recent Netflix post trumpets AV1’s suitability for mass-market distribution, but high encoding costs, the lack of HDR support, and nascent support in the living room make AV1 a non-starter for most producers. Most of these issues should be resolved by mid-2023, when mainstream usage should really start to blossom.
Contents
Netflix: AV1 on (Some) Smart TVs
In their November 2021 blog post, Bringing AV1 Streaming to Netflix Members’ TVs, Netflix announced that “Netflix has started streaming AV1 to TVs.” This post followed the company’s February 2020 announcement that there were “Now Streaming AV1 on Android.” As part of the more recent announcement, Netflix revealed a number of QoE-related improvements including that “AV1 streaming improves quality by up to 10 VMAF without impacting the rebuffer rate,” and “On some TVs, noticeable drops in quality were reduced by as much as 38%.”
In a section entitled Challenge 2: How do we guarantee smooth AV1 playback on TVs?, Netflix detailed their process for certifying AV1-capable TVs, which involved a set of certification streams with “extreme cases to stress test the decoder.” Netflix shared a list of certified TVs with The Verge website, which included the following devices:
- Select Samsung 2020 UHD Smart TVs
- Select Samsung 2020 UHD QLED Smart TVs
- Select Samsung 2020 8K QLED Smart TVs
- Samsung The Frame 2020 Smart TVs
- Samsung The Serif 2020 Smart TVs
- Samsung The Terrace 2020 Smart TVs
- Any TV connected to a PS4 Pro streaming with the Netflix app
- Select Amazon Fire TV devices with Fire OS 7 and above
- Select Android TV devices with Android OS 10 and above
According to the Statistica website, Samsung enjoyed a dominant 32% share of SmartTV sales in 2020, and if you throw in the PS4, Fire, and Android TV devices, plus Android devices from the previous announcement, Netflix wasn’t lying when they claimed that “with this launch, AV1 streaming reaches tens of millions of Netflix members.”
Still, when I posited to a buddy at a high-volume streaming service that the Netflix announcement proved that the Smart TV market was ready for AV1, he disagreed, saying that the list of supported TVs was simply too small. A later peek at The Verge list tends to prove his statement.
Unfortunately, Netflix provided few encoding specifics beyond mentioning that it always encodes at the highest source resolution and frame rate, at 10-bit color depth, and using dynamic optimization. The post further shares that to achieve production level efficiencies, the encoding team evaluated “different tools provided by the encoder,” and “with multiple iterations…arrived at a recipe that significantly speeds up the encoding with negligible compression efficiency changes.” Perhaps at some point, Netflix will share encoding details like those presented in this post comparing the SVT-AV1 and libaom codecs, which may simplify AV1 encoding for other implementors.
What About HDR?
Later in the post, Netflix stated that “another exciting direction we are exploring is AV1 with HDR,” a reminder that 10-bit encoding while making the stream more resistant to issues like banding, doesn’t deliver the complete metadata-driven HDR experience. While AV1 clearly should be able to deliver better quality than HEVC at similar bitrates, you would have to assume that a viewer would prefer HEVC with Dolbyvision or HDR10 to 10-bit AV1. Until AV1 is available with dynamic metadata this seemingly makes AV1 a tough choice for premium content distributors who currently deliver via HEVC-driven HDR.
Note that AV1-based HDR is close; there’s a working draft HDR10+ specification here, and I’ve heard from multiple sources that AV1-based Dolby Vision is in the works if not imminent. Still, draft specifications are a long way from a finished specification with solid support in third-party content-creation products and proven playback support. While AV1-based HDR is definitely coming, it’s clearly not here yet.
AV1 in the Short Term? Got 7-Figure View Counts?
Whatever the encoding efficiency achieved by Netflix, how does the AV1 use case generalize to other producers? As the post points out, “streams on the Netflix service can easily be watched millions of times.” The bandwidth savings on this number of views easily justifies the encoding costs that Netflix invests in its content, whether via the expensive dynamic optimization process, AV1 encoding, or both.
However, this seems to indicate that you can only recoup AV1 encoding expenses when videos are watched a few million times, which limits the benefit of AV1 to a small handful of producers. That appeared to be the case from YouTube’s usage of AV1, which seems to kick in only when videos exceed ~3.5 million views. Of course, that could be a throughput issue rather than a cost issue, as YouTube must keep up with the 500 hours of video uploaded each minute, which is made infinitely tougher when a significant percentage of those hours are encoded with AV1.
AV1 in the Long Term? Could Replace H.264 and HEVC
From a playback perspective, Netflix has proven the Smart TVs and other devices that claim to support AV1 actually work, which makes the waters safe for other producers targeting non-HDR living room viewing of AV1-encoded content. YouTube has done the same for browser-based playback, and also plays AV1 on some Smart TVs, like Samsung’s 4K Samsung Neo OLED TV.
But for producers with lower per-video view counts, AV1 might not become practical until encoding costs drop significantly. At the time of this writing (November 2021), Bitmovin charged a 10x premium over H.264 for AV1 encoding while AWS Elemental charged about 36x. These prices obviously reflect the additional processing time demanded by AV1’s encoding complexity.
Cheaper encoding can come from one of two directions; faster software-based encoding and hardware-accelerated encoding. SVT-AV1 has the backing of Netflix, as well as the Alliance for Open Media, and seemingly Facebook. That said, SVT-AV1 performed poorly in my tests here, and more recently in tests performed by Moscow State University. If SVT-AV1 itself doesn’t pan out, codecs derived from the same codebase, like Tencent’s AV1 codec, which I’ll be testing in December 2021, may provide better performance.
There’s lots of activity on the hardware encoding front from companies like NETINT, Chips&Media, Socionext, Xilinx, and other companies regarding FPGA and ASIC-based AV1 encoding. Though most VOD companies have primarily used software encoders for years, YouTube proved the benefit of hardware-accelerated VP9 encoding for VOD with their ARGOS VCU, which will be updated for AV1 at some point in the future.
I’ve heard from one colleague that the cost-per-minute pricing for the hardware-based cloud encoding will be similar to H.264 (and I asked twice). If this is the case, then at some point in the not-too-distant future:
- AV1 encoding costs will be dramatically lower, perhaps as low as H.264 hardware accelerated cloud pricing
- AV1 will be available in a dynamic metadata solution like Dolby Vision or HDR10+
- AV1-compatible smartTV and OTT penetration will increase to much more relevant ranges
- Browser and mobile support will continue to expand
At that point, AV1 will make sense to a much greater range of premium content and other streaming publishers. I’m calling it mid-2023. At that point, most technology-related hurdles should be behind us.
I should mention one legal-related hurdle for the sake of completeness. Specifically, though the Sisvel AV1 patent pool only claims royalties on “consumer display devices,” and not content, the uncertainty surrounding Sisvel’s claims may delay adaption through the entire streaming media ecosystem, from encoding to playback. Ditto for Sisvel’s VP9-related claims and VP9 usage.
In the interest of full disclosure, note that I perform consulting services for Sisvel, though I received no compensation or direction from Sisvel for the writing of this article.
