Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
At the recent World Wide Developer’s Conference (WWDC), Apple announced that the next versions of Safari, iOS, and tvOS will support HLS with HEVC encoded video. This puts Apple firmly in the HEVC camp, with the Alliance for Open Media camp (Amazon, Cisco, Intel, Google, Microsoft, Mozilla, Netflix, and many others) supporting the AV1 codec (and the VP9 codec before its release).
I discussed the technical implications in this article on StreamingMedia.com. In this article, I’ll predict how Apple’s decision will impact the codec market going forward.
Before exploring the implications of Apple’s move, let’s discuss eight realities. You know most of them, but a quick reminder before the predictions will be useful for everyone.
1. There’s already a ton of support for HEVC.
The figure below shows which platform, browser, and media companies are supporting which codec in the UHD codec war (virtually all encoding companies already support both). Apple is all alone with HEVC only.
Those fully in the Both camp are both specific and representative. That is, it’s not just Intel that supports both; it’s pretty much all CPU, GPU, and SoC vendors. Ditto for Netflix, which represents multiple streaming sites that support both codecs. Ditto for Samsung representing all Smart TV vendors, and Roku representing all OTT vendors except Apple.
Firefox, Chrome, and YouTube are solely in the VP9/AV1 camp, HEVC not included and not invited.
Android, Microsoft Windows 10, and Edge are all interesting cases. Android supports HEVC, but not in Chrome, so it can play downloaded HEVC content, and play HEVC from apps, but not in the browser.
Microsoft 10 has been in and out of HEVC so many times they make Brett Favre look resolute. However, looking at the page entitled H.265 / HEVC Video Decoder in the Windows Dev Center definitely leads this author to believe HEVC is back in to stay. But at least one person in the know has advised that HEVC is only enabled when hardware HEVC playback is enabled on the computer. Obviously, looking at the screenshot below, web infrastructure provider Bitmovin, who has an HEVC/HLS test page here, thinks so. (If anyone in the know from Microsoft wishes to set me straight on the matter, please contact me at [email protected]).
The Bitmovin test page appears to show that HEVC is supported in Windows 10 if the device supports hardware decode.
2. Once HEVC support is in the OS, all browsers can support it for free.
This is how Mozilla supported H.264, which they never directly licensed, on multiple operating systems. With Apple’s move, starting in late 2017, the MacOS will have HEVC playback, and many recent hardware devices (and all new Apple products) will have HEVC hardware decoding. It appears that Windows 10 has HEVC decode, and Edge already supports it, at the very least on computers with hardware decode, possibly all Windows 10 computers.
Just because it’s possible doesn’t mean it’s inevitable. A contact at Google advised that codec implementations are smoother when decode is included with the browser. To optimize the customer experience, Google might choose to include their own decoder, which likely would give rise to a royalty. Beyond that, every major browser vendor save Apple is in the Alliance for Open Media, who are aligned against HEVC. So just because browser vendors can support HEVC for free, doesn’t mean that they will.
Still, browser vendors will have to compete with Edge on the Windows desktop and Safari on the Mac desktop and iOS, all with HEVC decode. Job number 1 for the browser vendors isn’t to support AV1, it’s to gain market share. If Apple does their usual good job pushing the importance of HEVC and HLS, and Safari market share starts to rise on iOS and MacOS devices, browser product managers will be screaming to support HEVC, particularly since it costs nothing.
3. The only companies hurt by the ridiculous HEVC royalty structure are content companies who have to support more than one codec. Oh, and us.
While Apple’s support for HEVC could cost it over $100M annually, you could argue that it costs them nothing, because they ultimately pass the cost along to consumers who buy their Macs and mobile devices. Ditto for Samsung, Roku, Intel, and Netflix. Microsoft can certainly recoup the royalty via sales of Surface and other hardware devices, as well as paid upgrade to Windows 10. Google will do the same for Pixel phones and other hardware devices with HEVC.
Companies that incorporate HEVC into CPUs, SoCs, and GPUs don’t pay the royalty; the seller of the computer or mobile device does. For this reason, hardware HEVC decode is pretty much a standard feature is most medium to high-end chips and chipsets. By the same token, the codec and encoding companies shouting that now is the time to adopt HEVC also aren’t paying the royalty either.
Mozilla has nothing to sell, but they can support HEVC via OS support on Windows, MacOS, iOS, and Android. So really, the only folks who are hurt by the HEVC/VP9 battle are companies who have to encode to multiple formats, which costs under $10/hour. Oh, and since consumers are footing the bill via increased prices, us as well, though it’s peanuts to any single individual.
All that said, if the third HEVC pool comes in with a US $40M cap, HEVC could cost US $100M/year, and with a 20-year patent life, could be a US $2 billion dollar decision. That’s a lot of coin that some low-cost providers serving overseas markets will choose not to spend.
4. Apple chooses codecs to sell hardware, content producers choose codecs to ensure QoE. iOS 11 will include HEVC support for playback on older devices, like my iPhone 6, without HEVC hardware decoding. If playback is slow, or battery life suffers, Apple will gladly sell me a new phone. With a daughter in college and one a year away, thanks but no thanks.
Netflix, on the other hand, is still encoding in VC1 format to support their customers on legacy gaming platforms. They, and all successful premium content distributors, have a fanatical focus on Quality of Experience. If HEVC doesn’t work well on my iPhone 6, they won’t send that iPhone 6 HEVC; end of story.
So, irrespective of what you hear from codec and encoder vendors that Apple’s announcement will open the floodgates of HEVC adoption, expect little movement from actual content producers until performance issues have been sorted out. My guess is that 3-6 months after the Apple releases, we’ll see some premium content distributors migrate to the UHD codec of their choice.
5. Publishers respond slowly to the bandwidth savings argument. YouTube started using VP9 to save bandwidth in early 2014, and playback has been available in Chrome, Firefox, and Opera for at least two years, and more recently Edge. Like HEVC, VP9 has enabled bandwidth drops of between 20 – 50% depending upon resolution. All the same arguments HEVC vendors are now using to convince publishers to use HEVC – lower bandwidth, better quality to mobile users, more video through fixed pipes – have been used since 2014 to convince publishers to deploy VP9.
There’s been some uptake; JW Player uses the codec in the online video platform, and Netflix uses it for mobile downloads. In their 2016 Global Media Format report, encoding.com reported that 11% of the files they produced were in the VP9 format and only 3% in HEVC. But by and large, adoption has been slow, and you would expect the same with HEVC, even if anointed by Apple
6. Apple’s market share for combined tablets and smartphones is about 33%. For smartphones alone, it was around 14.4% according to Gartner. There’s little doubt that we tend to overstate Apple’s trend-setting abilities, particularly in second and third world economies, where low cost rules the day. While Apple can easily absorb and pass along another $2-3 on an $800 iPhone, low-cost Android devices in other markets can’t. So Apple’s decision may strongly influence things in the US and Europe but won’t push global adoption of HEVC.
7. Apple’s choice of HDR technologies is unclear, at least to me. As we all know, HDR makes a greater impact on perceivable quality than 4K at most viewing distances. As far as I know, Apple hasn’t endorsed an HDR technology, though the US SmartTV market seems to have settled on Dolby Vision at the high end, and HDR10 at the low end. Publishers seeking to support SmartTVs and Apple devices may have to encode twice, once for SmartTVs, and once for Apple. Since HDR support is one of the key reasons for deploying HEVC, Apple’s HEVC adoption may actually slow adoption by content publishers until the HDR picture is clear.
8. YouTube will stick with VP9/AV1. This in itself will mandate hardware support for AV1 in all SmartTVs, OTT devices, phones, tablets, and computers.
Implications of the Apple Decision
With this as Prolog, here’s what I see happening.
Prediction 1: HEVC support is incorporated into all browsers on all major platforms by June 2018.
Desktops – On desktops, HEVC is already included in Edge and soon will be in Safari. This will put tremendous pressure on Firefox and Chrome to follow, particularly since adding HEVC doesn’t have to hit their bottom line.
Mobile – HEVC will be in iOS, and will continue to be a marketing focal point for Apple. Android vendors are already paying the HEVC royalty for devices that they sell, including Google for Pixel phones. These vendors will demand HEVC playback in Chrome, probably via HLS, as well as HEVC for internal recording and playback.
This, of course, means that HEVC encoded video can stream to all (then current) platforms, obviating the need for VP9 or another UHD codec. That said…
Prediction 2: AV1 launches with a vengeance in early 2018, with browser support (Chrome, Firefox, Edge) by March 2018, operating system support (Android, Windows) by June 2018, and hardware support by early 2019.
This is just early enough to force consideration by all major content publishers thinking about HEVC. With Netflix, Amazon, and Google (YouTube) in the Alliance for Open Media, you know at least these three will be strongly disposed towards AV1.
Prediction 3: No major publishers implement HEVC/HLS support before 3-6 months after iOS 11/MacOS Sierra ship. This leaves the door open for a full codec analysis between AV1 and HEVC, including encode and decode requirements, hardware support, cost, IP risk, HDR support, software support, the whole nine yards. At least in the US and Europe, one of these codecs will be codec next.
Frankly, I think the FUD (Fear, Uncertainty, Doubt) factor regarding AV1’s IP status make it a long shot, but few publishers should choose their UHD codec without at least a look at AV1. If it delivers a substantial bandwidth reduction as compared to HEVC, it will get a very hard look, particularly when Netflix and Amazon start using it.
Prediction 4: Marketing hype is global, codecs are local. Premium content distributors around the world will choose the best codec for their markets. In second and third world markets, iPhones play a very small role, and there will be plenty of low-cost Android phones, and perhaps even tablets and computers, without HEVC hardware support. In these environments, VP9/AV1 or another codec (PERSEUS?) might be best.
Prediction 5: Ninety-nine percent of streaming producers will stick with H.264 at least through the end of 2019. The HEVC/AV1 battle is for premium content producers and others producing 4K where the bandwidth savings are most significant. Ditto for VR, which again is high bitrate video utilized by an incredibly small share of streaming producers. The average Joe or Joette producer will stick with H.264 for the foreseeable future.
How will we know if I’m right?
Keep your eye on two data points. If lots of premium content producers join Apple on the HEVC/HLS podium, it may put pressure on other publishers to adopt HEVC early. Be sure to scratch beneath the press release, however, to gauge the reality of the support. Apple is a master of making noise. For example, if a content publisher joins Apple, are they converting their entire library to HEVC, or only UHD titles? Or are they only deploying content previously encoded into HEVC?
If browser vendors make HEVC/HLS playback available more quickly than predicted, it could also force a UHD codec selection before AV1 is ready. That said, I really, really, doubt it. Premium content publishers are risk averse and should be. OTT publishers can realize more benefits at much less risk using per-title encoding techniques than they could by implementing a new codec.
For more information and opinions on the impact of the Apple announcement, check out:
- Documents and presentations from WWDC here.
- Dan Rayburn’s commentary, Why Apple’s HEVC Announcement Is A Big Step Forward For The Streaming Media Industry.
- My article on Streaming Media, Apple Embraces HEVC: What Does it Mean for Encoding.
