Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
This is an installment in our ongoing series of “What Is…?” articles, designed to offer definitions, history, and context around significant terms and issues in the online video industry.
Contents
Executive Summary
HTML5 rocketed to the forefront with Apple’s decision to forgo Flash and use HTML5 technology to deliver video to the iPad. Actual HTML5 usage, however, has been slowed by low HTML5-compatible browser penetration, the need to encode video into two or three formats for complete HTML5 compatibility, and the lack of feature parity with Flash, the predominant plug-in based technology for viewing video on the web.
HTML5 needs to be on the radar screen of all streaming media producers, but switching over to HTML5 does not appear to be a short-term priority as of this writing.
HTML5 the Specification
HTML stands for HyperText Markup Language, and HTML is the primary language used to produce most web sites. HTML is a standard maintained by the World Wide Web Consortium (WC3), a group that includes browser developers and other interested technology companies. HTML5 is the latest iteration of HTML to be introduced by the WC3.
In 2004, a group called the HTML Working Group (HTMLWG) was formed by employees of Apple, the Mozilla Foundation, and Opera Software, because these companies “were becoming increasingly concerned about the W3C’s direction with XHTML, lack of interest in HTML, and apparent disregard for the needs of real-world authors.” Later, the WC3 recognized the HTMLWG, and tasked it with providing (among other deliverables):
A language evolved from HTML4 for describing the semantics of documents and applications on the World Wide Web. This will be a complete specification, not a delta specification.
The Group will create a comprehensive test suite for the HTML specification.
To a degree, this dual control has led to two specifications of HTML5 (see, for example, “The two specifications of HTML 5.0: WHAT WG vs W3C,”) or at least the appearance of two specifications. Here’s an explanation from the WHATWG FAQ.
Which group has authority in the event of a dispute?
The editor takes feedback from everyone into account and does not look at the source of those arguments for technical arguments.
The W3C HTML Working Group has an escalation process that in some cases results in a decision being made that differs from the editor’s original decision on a topic. So far, whenever this has happened the WHATWG has gone along with the W3C’s request; nothing of especially big importance has been changed in this manner so far (it’s mostly been editorial issues or mostly minor technical issues). In general the WHATWG will ensure that the normative content of the specifications (the requirements on authors and implementors) remains the same so long as the W3C group doesn’t demonstrate any serious lapses in judgement.
On January 18, 2011, the W3C revealed an HTML5 logo (Figure 1), complete with t-shirts and stickers, and on February 14, 2011, W3C announced that HTML5 would be finalized in mid-2014.
Figure 1. The HTML5 logo.
In contrast, on January 11, 2011, in a blog post entitled “HTML is the new HTML5,” the WHATWG announced that they were moving to a “new development model that eliminated the HTML5 version number and made the existing HTMLWG spec a “living standard.” Ian Hickson, editor of both the WC3 and WHATWG standards, commented:
Since the W3C is still focused on getting HTML5 to last call, I’ll maintain a WHATWG copy of the HTML spec that matches what the W3C HTML WG is working on:
http://www.whatwg.org/specs/web-apps/html5/
I’ve updated the FAQ’s explanation of what specs exist, in case you want to get an overview (it’s getting somewhat complicated!):
http://wiki.whatwg.org/wiki/FAQ#What_are_the_various_versions_of_the_spec.3F
So, HTML5 is dead from the WHATWG perspective, but you can buy an HTML5 t-shirt from the WC3. This complexity notwithstanding, both the WC3 and WHATWG versions of the specification are consistent on how they deal with the video tag, discussed next. For the sake of clarity, we’re going to continue to call the spec HTML5 in this article.
The HTML5 Video Tag
HTML5 is a sweeping specification, with multiple new features and capabilities. From a streaming media perspective, however, what’s particularly interesting about HTML5 is that it contains a video tag that enables browsers to natively play back video within the page, obviating the need for plug-ins like Flash, Silverlight, and QuickTime. This particular aspect came to the forefront in early 2010 when Apple introduced the iPad without support for Flash, with video playback accomplished via HTML5.
To support this feature, however, browser vendors must directly implement codec playback capabilities in their browsers. Originally, there were two video codecs being considered for use in HTML5: Ogg Theora, which is open source, and H.264, which bears royalty obligations. While the original goal of the specification was to specify one video codec that would work across all browsers, this goal proved unreachable. Here’s a quote from a June 29, 2009 Ian Hickson post to the WHATWG message list:
After an inordinate amount of discussions, both in public and privately, on the situation regarding codecs for <video> and <audio> in HTML5, I have reluctantly come to the conclusion that there is no suitable codec that all vendors are willing to implement and ship.
I have therefore removed the two subsections in the HTML5 spec in which codecs would have been required, and have instead left the matter undefined, as has in the past been done with other features like <img> and image formats, <embed> and plugin APIs, or Web fonts and font formats.
The current situation is as follows:
- Apple refuses to implement Ogg Theora in Quicktime by default (as used by Safari), citing lack of hardware support and an uncertain patent landscape.
- Google has implemented H.264 and Ogg Theora in Chrome, but cannot provide the H.264 codec license to third-party distributors of Chromium, and have indicated a belief that Ogg Theora’s quality-per-bit is not yet suitable for the volume handled by YouTube.
- Opera refuses to implement H.264, citing the obscene cost of the relevant patent licenses.
- Mozilla refuses to implement H.264, as they would not be able to obtain a license that covers their downstream distributors.
- Microsoft has not commented on their intent to support <video> at all.
Since then, Google bought On2 and open sourced the VP8 codec as WebM, which has been implemented in Google Chrome ( version 6.0+) and the Opera browser (version 10.6+) and will be supported in Mozilla Firefox 4.0 when it ships (and is currently available in the beta). Google also announced their intention to drop H.264 support from Chrome.
Safari has not implemented any codec other than H.264. Microsoft implemented H.264 in Internet Explorer 9, but later stated that their new browser will support “VP8 video when the user has installed a VP8 codec on Windows.”
While there is no single HTML5 codec that will work in all browsers, by using alternative tags like those shown in Figure 2, websites can fully support all HTML5-compatible browsers.
Figure 2. Jilion uses multiple tags and files to fully support HTML5.
It should be noted that some of these codec limitations can be avoided via downloadable plug-ins. For example, on January 14, 2011, Google promised to release plug-ins that enable WebM support in Safari and Internet Explorer 9 via the HTML standard <video> tag canPlayType interface. In addition, on February 2, 2011, Microsoft released the Windows Media Player HTML5 Extension for Chrome, which will enable the Chrome browser to continue to play H.264 video via HTML5 once Google removes H.264 support from Chrome.
Invasion of the Patent Trolls
Meanwhile, issues of patent infringement reared their ugly heads with both Ogg Theora and WebM. By way of background, MPEG LA manages the patent pool for H.264 and a number of other technologies. In an interview with Streaming Media on March 4, 2010, MPEG LA CEO Larry Horn commented that “no one in the market should be under the misimpression that other codecs such as Theora are patent-free. Virtually all codecs are based on patented technology, and many of the essential patents may be the same as those that are essential to AVC/H.264. Therefore, users should be aware that a license and payment of applicable royalties is likely required to use these technologies developed by others, too.”
In April, 2010, when Steve Jobs was asked why Apple used H.264 instead of Ogg Theora, he reportedly replied “All video codecs are covered by patents. A patent pool is being assembled to go after Theora and other “open source” codecs now. Unfortunately, just because something is open source, it doesn’t mean or guarantee that it doesn’t infringe on others patents.”
MPEG-LA never took any formal action against Ogg Theora, most likely because it was largely supplanted by WebM. Regarding WebM, however, Microsoft attributed its decision not to implement WebM directly in Internet Explorer 9 to Google’s refusal to indemnify, or pay the costs of patent infringement, should Microsoft get sued patent infringement for using WebM:
For example, will Google indemnify Mozilla, a PC OEM, a school, a Web site, a chip manufacturer, a device company, or an individual for using WebM? Will they indemnify Apple? Microsoft? Will they indemnify any or all of these parties worldwide? If Google were truly confident that the technology does not infringe and is not encumbered by patents whatsoever, wouldn’t this indemnification be easy?
On February 10, 2011, MPEG-LA issued a call for patents essential to WebM, which could lead to royalties for anyone using the technology. On March 4, the Wall Street Journal reported that the Justice Department is investigating MPEG-LA to determine whether it’s “unfairly trying to smother a free rival technology for delivering online video that is backed by Google Inc.”
To be fair, licensing H.264 with MPEG-LA doesn’t guarantee that third parties not included in the H.264 patent pool won’t assert intellectual property rights against companies using H.264, with the AVC/H.264 FAQ stating “No assurance is or can be made that the License includes every essential patent.” However, at this point, their appears to be a far greater risk that users of WebM will be subject to third party royalty claims than users of H.264
Using HTML5: Practical Issues
With this as background, let’s examine some practical issues relating to using HTML5.
HTML5-Compatible Browser Penetration
According to NetMarketShare, as of March, 2011, the percentage of HTML5-compatible browsers was well under 46%. This is shown in Figure 3, where three of the top four browsers were older versions of Internet Explorer, comprising 54% of the total, were not HTML5 compatible. Tallying the top 30 browsers in terms of market share revealed that about 40% are currently HTML5 compatible, while 59% were not.
Figure 3. Browser usage stats as of March, 2011.
Now that a release candidate of Internet Explorer 9 is shipping, you would expect these numbers to change dramatically over the next twelve months. On the other hand, NetMarketShare also reports that Windows XP users comprise 55.09% of the installed base of all operating systems, and IE9 won’t run on Windows XP. Certainly these users of legacy versions of Internet Explorer could install other HTML5 compatible browsers, but they could have long before Internet Explorer 9 launched, and most didn’t. One would have to assume that either these desktop users didn’t care about HTML5, were on locked down corporate, government or education desktops, or both.
Note that it’s possible to create HTML5 syntax that displays video first using HTML5 if available, and then falls back to Flash, Silverlight or other plug-in for legacy browsers. This approach should work well for most simple, video in a window, playback applications. However, as a new technology, HTML5 doesn’t offer many of the features available for alternative platforms like Flash or Silverlight, at least not in proven, cross-browser compatible form.
HTML5 Feature Deficits
For example, though YouTube has made an HTML5 player available since January, 2010, YouTube still defaults to the Flash Player. On June 29, 2010, YouTube addressed why in the YouTube API Blog, where the company stated that the video tag doesn’t currently meet all of YouTube’s needs. Areas that needed work included the lack of a standard video format, robust video streaming, content protection, Encapsulation and Embedding, Fullscreen Video ,and Camera and Microphone access.
As the YouTube blog states, progress is being made in many areas. For example, Wowza Media Systems (Figure 4) has previewed a version of Wowza Media Server that streams live HTML5 video with WebM, though it requires a WebM compatible browser for playback. Anevia is also showing a WebM-based live, adaptive streaming solution, though this requires a WebM-compatible browser as well. Flumotion has also implemented live streaming with WebM.
Figure 4. Wowza Media Server streaming live WebM video to WebM compatible browsers.
At the second annual W3C Web and TV Workshop, held in Berlin in February 2011, there were sessions on HTTP Adaptive Streaming and Content Protection, though standards have yet to be proposed or adapted, indicating that a standards-based solution is many months, if not years, away.
In this regard, it’s important to distinguish HTML5-based technologies available for iOS devices from general purpose HTML5 technologies. That is, while Apple has created solutions for live, adaptive streaming and digital rights management for the iOS platform, these won’t work in other browsers. For example, Apple HTTP Live Streaming, which enables live and on demand, single file or adaptive streaming, only runs on “iPhone, iPad, and iPod touch (requires iOS version 3.0 or later) or any device with QuickTime X or later installed.” (). QuickTime X is available only for Mac OS X Snow Leopard, and not for Windows computers.
HTML5 Adoption To Date
The most high profile adaption of HTML5 to date has been by Apple with the iPad, though as mentioned, many components of this solution are Apple specific, and all videos are in H.264 format, so won’t play on the current majority of HTML5 browsers. While many commercial and broadcast sites support the iPad, very, very few (if any) fully support all HTML5 browsers with the required three formats of video.
Multiple UGC sites, including YouTube and Vimeo, have HTML5 compatible players, though both require users to opt in to view the HTML5 pages. At the time of this writing, YouTube (a Google subsidiary) is still producing HTML5 compatible files in both H.264 and WebM formats, with no word if H.264 will be dropped. Neither site encoded video into Ogg format, so the only way for Firefox users, which comprise about 50% of the HTML5 compatible market, to view content from these sites is via the Flash Player.
Otherwise, multiple service providers like Kaltura and Brightcove are making HTML5 compatible solutions available, though the most common applications are for HTML5 video support on iOS devices. Many third parties are releasing HTML5 compatible players and toolkits for game development and other HTML5-based applications.
At WebM’s launch on May 19, 2011, Adobe announced that “we are excited to include the VP8 video codec in Flash Player in an upcoming release, which will help provide users with seamless access to high quality video content on all of their Internet-connected devices.” A WebM compatible version of the Flash Player has neither been shipped, or announced, as of the time of this writing. On August 31, 2010, Adobe shipped the Adobe Dreamweaver CS5 HTML5 Pack which added many new HTML5-related features to Adobe’s popular HTML authoring program.
Why HTML5 Matters to You
HTML5 has become a high profile “got to have it” buzz word that sounds more immediately compelling and accessible in the abstract than it is in reality. Understanding the current status and limitations is critical to all streaming media producers.
For more resources on HTML5 Video, check out Streaming Media’s “25 HTML5 Video Resources You Might Have Missed.”
