Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
[Author’s note: In early May, Apple opened the API for Adobe to access the GPU for H.264 acceleration from within Flash. Click here to view details and some early test results.]In part, Steve Jobs stated that the iPad didn’t support Flash because it was a “CPU Hog,” so Apple used a technology called HTML5 instead. Since the comparative efficiency of Flash vs. HTML5 seemed easy enough to quantify, I endeavored to do so, using YouTube’s new HTML5-based player as the test bed. Specifically, I played a YouTube video in the same browser twice, once via HTML5, once via Flash, and measured CPU utilization during playback.
By way of background, HTML5 is a new markup language that eliminates the need for a plug-in like Flash to play video. Instead, the browser supplies the decode capability directly. There is no standard HTML5 codec, however, with Apple Safari and Google Chrome supporting H.264 playback (which is required to play back video on YouTube’s HTML5 pages), Mozilla Firefox, Opera’s Opera and Chrome supporting Ogg Theora playback, and the 800-pound gorilla supporting neither codec (that would be Internet Explorer). All, of course, still support Flash.
Accordingly, the only browsers that could theoretically play both the HTML5 and Flash pages were Apple Safari and Google Chrome. I say theoretically, because in practice, the Windows version of Safari couldn’t play the HTML5 YouTube page. To complete the picture, I also tested Firefox and Internet Explorer using the Flash plug-in. I tested on the Mac using a MacBook Pro (3.06 GHz Core 2 Duo, 8 GB RAM, OS 10.6.2) while testing on Windows using an Hewlett Packard 8710w mobile workstation (2.2 GHz Core 2 Duo system running 64-bit Windows 7 with 2 GB of RAM).
Contents
Whither Hardware Acceleration?
I spoke with Adobe during my testing to get their thoughts on the procedures outlined below. During our conversations, they let me know that Flash Player 10.1 should deliver substantial performance improvements over 10.0 because it can use the graphic processing unit (GPU) on some computers for decoding video, a capability generically known as hardware acceleration. According to the Adobe Web Site,
Hardware-accelerated H.264 decoding on Windows desktop, notebook, and netbook systems is supported on some video cards and drivers running on Windows XP, Windows Vista, and Windows 7. These include some recent NVIDIA, AMD/ATI, and Intel graphics cards. The player also supports hardware decoding with hardware such as some Broadcom video decoders.
Please read the Flash Player 10.1 public beta release notes (PDF, 105 KB) to learn about supported hardware and find links to download supported drivers. We recommend updating to install the latest drivers available.
In Flash Player 10.1, H.264 hardware acceleration is not supported under either Linux or Mac OS X. Linux currently lacks a developed standard API that supports H.264 hardware video decoding, and Mac OS X does not expose access to the required APIs. The Flash Player team will continue to evaluate adding hardware acceleration to Linux and Mac OS X in future releases.
To be clear, hardware acceleration was available in 10.0, but only when playing in full screen. With 10.1, it’s available for all video playback, and in the tests detailed below, I played the YouTube video within the player interface, not full screen. As the Adobe page says, however, the expanded hardware acceleration will be available on the Windows platform, but not Mac and Linux. Mac users don’t despair, however, since other Mac-related enhancements, like support for Core Animation, should boost Mac performance.
Long story short, Adobe claimed that there would be a substantial reduction in CPU utilization on both the Mac and Windows platforms with 10.1. So I ran two sets of tests; one with Flash Player 10, the other with 10.1.
My Procedures
During testing, I followed this procedure:
– Turned off as many background processes as possible- Updated my graphics card drivers
– Used the same YouTube video for all tests (http://www.youtube.com/watch?v=VJ5KJVCc5C4)- First, I loaded the page
– Then switched the video to 720p, but left the video playing within the player window (not full screen)
– Then waited until the video completely downloaded (watching the little red bar thingie on the bottom of the player).- Then I played until about 14 seconds in, waiting until the opening sequence cleared and the real world video began playing.
– Then, I started monitoring and recording the CPU Usage on Windows Task Manager and % Idle on Activity Monitor on a separate computer for 29 data points for each test. Since Task Manager updates once each second, that meant about 30 seconds of testing on Windows. Because Task Manager updates once every two seconds, that means about 60 seconds of video playback on the Mac.- I used the CPU Usage directly from Windows and subtracted the % idle from Activity Monitor from 100% to derive total CPU utilization on the Mac.- I rebooted between each test.
– I ran each test at least twice to confirm results.
Browsers Tested
I tried to test the most current versions of all programs on both platforms: Here are the versions.
Windows:
– Apple Safari – Apple has a site called Webkit where you can download beta versions of the Safari browser (http://webkit.org/). Since the improvements in Flash Player 10.1 required the latest version of Safari, I downloaded that version – in Windows, it was 4.0.4 (531.21.10).
– Mozilla Firefox – version 3.6
– Google Chrome – 4.0.249.89 (I tried loading the Beta 5 browser, but it failed several times on this computer).
– Microsoft Internet Explorer – 8.0.7600.16385
– Adobe Flash Player – 10.0.45.2 first, then 10.1.51.95
Macintosh:
– Apple Safari – Webkit version was 4.0.4 (6531.21.10,55180).
– Mozilla Firefox – version 3.6
– Google Chrome – 5.0.307.9 beta
– Microsoft Internet Explorer – 8.0.7600.16385
– Adobe Flash Player – 10.0.45.2 first, then 10.1.51.95
Test Results: Macintosh
Table 1 shows the Macintosh results. Some observations.
Table 1. Macintosh results.
- With Safari, native playback in HTML5 (at 12.39) was the most efficient alternative, and consumed significantly less CPU than playback via Flash (at 37.41), presumably because Apple uses H.264 hardware acceleration inside of Safari (see here). Flash Player 10.1 reduced playback CPU by 5 CPU % points (from 37.41 to 32.07), or about 14%. Before assuming that Flash is bad, and HTML5 good, please see the Windows results below.
- With Chrome, playback via Flash and HTML5 were equally inefficient, both much higher than either Safari alternative. This is a head scratcher. I would guess that it means that Google isn’t benefiting from hardware acceleration, though I don’t know if that’s because they “can’t” or simply “haven’t.” Either way, Chrome shouldn’t be your browser of choice on the Mac for spending serious time on YouTube.
- With Firefox, playback via Flash (the only alternative) was slightly less efficient than Safari in Flash, but more efficient than Chrome using Flash or HTML5. Firefox actually slowed down with version 10.1.
I asked Adobe about this and Adobe’s Emmy Huang commented “The 3-5% improvement on Safari is what we would expect with Flash Player 10.1 on Safari using Core Animation. Firefox performance however was slightly degraded because Flash Player switched from using QuickDraw to Quartz 2D.” Now, on to Windows.
Test Results: Windows
Table 2 shows the Windows results. Some more observations.
Table 2. Windows results.
- As mentioned, Safari wouldn’t play the HTML5 videos on Windows, so I have no Flash vs. HTML5 comparisons there (I tested this on three Windows computer and the HTML5 page wouldn’t play on any of them). However, Flash’s ability to access hardware acceleration in 10.1 dramatically reduced Safari’s CPU consumption from 23.22 to 7.43, a drop of 68%, which really makes you wonder how Flash would perform on the Mac if it could access hardware acceleration.
- While Chrome’s numbers were more efficient on Windows, playback with Flash Player 10.0 was about 24% more efficient than HTML5, while Flash Player 10.1 was 58% more efficient.
- Version 10.1 also dropped Firefox’s CPU utilization by a whopping 73% and Internet Explorer’s CPU utilization by 35%.
To these scores, Adobe’s Huang added “These results also indicate the strong benefits in performance hardware decoding brings for video playback in Flash Player and other technologies.”
Overall Conclusions:
When it comes to efficient video playback, the ability to access hardware acceleration is the single most important factor in the overall CPU load. On Windows, where Flash can access hardware acceleration, the CPU requirements drop to negligible levels. It seems reasonable to assume that if the Flash Player could access GPU-based hardware acceleration on the Mac (or iPod/iPhone/iPad), the difference between the CPU required for HTML5 playback and Flash playback would be very much narrowed, if not eliminated.
I don’t follow the politics of the situation, but after noting significant playback efficiencies in Flash Player 10.1 on the Mac, respected technologist and AnandTech founder Anand Lai Shimpi commented “with actual GPU-accelerated H.264 decoding I’m guessing those CPU utilization numbers could drop to a remotely reasonable value. But it’s up to Apple to expose the appropriate hooks to allow Adobe to (eventually) enable that functionality.” So it looks like the ball is in Apple’s court.
Overall, it’s inaccurate to conclude that Flash is inherently inefficient. Rather, Flash is efficient on platforms where it can access hardware acceleration and less efficient where it can’t. With Flash Player 10.1, Flash has the opportunity for a true leap in video playback performance on all platforms that enable hardware acceleration.
Turning full circle, if Anand is right, and I don’t doubt that he is, Apple complaining about Flash being a CPU Hog while not exposing “the appropriate hooks” to enable Adobe to access hardware acceleration seems disingenuous at best. To be fair to Apple, though, the iPad related timing was unfortunate, with the bulk of the development work done under the shadow of Flash Player 10.0, which didn’t offer hardware acceleration other than full screen on any platform and was clearly less efficient than the HTML5-based approach Apple adopted. Now that Adobe has proven the concept on Windows, perhaps Apple will cooperate with Adobe to make hardware acceleration on the Mac, iPad and future devices happen. If they choose not to, however, they should quit pointing fingers at Flash.
What else? We also learned that not all HTML5 browsers/H.264 decoders are created equal. Significantly, with Flash 10.1 deployed, Google’s HTML5 implementation required the most CPU horsepower of all playback scenarios — by far — on the Windows platform. On the Mac, Firefox and Safari with Flash required less CPU horsepower than Chrome’s HTML5 implementation.
At least from a CPU utilization perspective, Flash isn’t BAD and HTML5 isn’t GOOD. It all depends upon the platform and implementation.
Caveats:
Some closing thoughts. I spoke to Adobe about this test methodology because it was mostly a test about Flash. I didn’t contact any other companies because the tests are objective and straightforward. At least they seemed so at the time.
I only tested two computers (one on each platform) because of time constraints. Yes, I know, additional data would have been useful. If you think it would be that useful, don’t complain to me, run the tests yourself and share the results. Just to be sure to run the Flash 10.1 tests on a device with supported graphics hardware.
If you see a flaw in the test procedures, don’t write to complain without:
– documenting new procedures
– testing yourself
– sharing those results.
I’ll be glad to retest if there’s a flaw (actually, not glad, but I will anyway).
Finally, I understand that in addition to calling Flash a CPU Hog, Mr. Jobs also complained about crashing with Flash. Instability is infinitely harder to test for than performance, so I didn’t even try.
Test Notes:
I tried to load Windows on the MacBook Pro so I could produce both OS tests on one hardware platform, but Bootcamp got persnickety and said it couldn’t create a Windows partition. I could have reformatted the MacPro, reinstalled the OS and then tried again, but since I had the 8710w all ready to go, I decided to just use it.
I also tested in YouTube’s Feather mode, which, according to YouTube, “is intended to serve YouTube video watch pages with the lowest latency possible. It achieves this by severely limiting the features available to the viewer and making use of advanced web techniques for reducing the total amount of bytes downloaded by the browser. It is a work in progress and may not work for all videos.” My tests revealed only a very minor difference between Feather mode and normal mode (about 1% either way) so to keep the numbers to a minimum and streamline the results, I showed only those results from Flash mode.
I ran some additional tests on Vimeo, which tracked the YouTube results with Flash Player 10.0. I didn’t run the 10.1 tests because I was concerned that the lack of a mode analgous to YouTube’s Feather mode might prejudice the results.
For those interested in a comprehensive look at Flash 10 vs. 10.1 video playback performance, click here for a comparison in AnandTech.
