Reflections on H.264 and Silverlight from a week at Stanford

All’s been quite on the Streaming Learning Center front as I spent the last week teaching several courses on streaming media production and encoding at the lovely Stanford Campus at Palo Alto. Other than a quick sneak peak at the new Final Cut Studio, which you can read about here, it was pretty much full immersion into the streaming world from a user’s perspective. This led to several conclusions about the future of streaming video and Microsoft Silverlight that I thought I would share.

The first observation came to me when I was explaining low level H.264 encoding parameters to the class – specifically, that many of the students would never really need to know these, or VP6 or WMV encoding parameters, at least not for much longer. This related to multiple trends.

The emergence of User Generated Video (UGC) and Online Video Platform (OVP) vendors as streaming providers for small to medium sized companies.

Five years ago, small to mid-sized companies who wanted to stream video from their web sites needed to encode the video, create a player and hope that whichever site they served it from could handle the hoped for high volume of viewers. They also had to chase eyeballs on their own.

Today, a small company in the same position should just upload the video to YouTube or Vimeo, or, for better service and analytics, OVP vendors like Brightcove, Ooyala and Sorenson. Then they’ll use codes provided by these vendors to embed the videos in their own web site, like I do for the videos on (for background on Online Video Platforms, see  Choosing an Online Video Platform).

If you go this route, you won’t even know what format your vendor is using, and you probably won’t care, so long as the video looks good and plays back in the Flash player virtually all of these technologies use. You’ll get a better player than you could produce yourself, complete with full screen options, embed codes, email links, and a better delivery network than your local ISP. If you opt to upload to multiple UGC sites, like, YouTube, Vimeo and Google Video – and why wouldn’t you? – they can even help get more eyeballs for your videos. Either way, you won’t have to know the optimal setting for key frames, B-frames or reference frames, or how to create a player, you’ll simply entrust your service provider to set them for you.

The emergence of H.264 as the de facto standard

Codec specific knowledge will also become less valuable as the world inevitably turns to H.264. Five years ago, a compressionist’s toolkit needed to include a working knowledge of Windows Media, Real Video, Sorenson Video 3 and MPEG-4, along with an eye out for an emerging standard called H.264. Today, it’s VP6, which essentially has 2 relevant controls (VP6-E/S), Windows Media, and H.264. In two years, unless MPEG-LA totally hoses the royalty structure, most new encoding will be almost exclusively H.264.

Why, you say? Because it’s a joint standard adapted by the International Telecommunications Union and International Standards Organization, which together helps directs the standards adapted by the cellular, video, computer and consumer electronics market places. Obviously, it’s also been adapted by Apple, Adobe and Microsoft.

Looking forward, if video is critical to how you market, sell or support your product, or how you train or communicate with employees and partners, you’re going to need to consider a three or four screen strategy, encompassing cellular, computer, consumer electronics and portable players like iPods. The only compression technology that will work on all of these going forward will be H.264. So, if you’re serious about streaming video, that’s the codec you’ll need to adapt.

Once H.264 becomes the only truly relevant codec, bit-stream standards will coalesce, templates in encoding tools will adhere to them more closely and producing for any of the four screens will be as complicated as picking a target and pressing the encode button. Sure, there will be some variation in encoding parameters, but fewer and fewer producers will care about them.

The emergence of Cloud Encoding as a viable encoding alternative for large companies

On its face, the concept of cloud encoding companies sounds kind of funky – take two hours to upload a one GB raw video file and then start encoding? But then you realize that this is exactly the paradigm used by YouTube, Vimeo and multiple other sites UGC and OVP sites, and it certainly hasn’t held them back. You can also look forward and assume that in two or three years, broadband transmission speeds will approach the speed we loved on our LANs back in the late ‘90s, and that file uploading time will drop precipitously (for more on Cloud Encoding, see Encoding in a Cloud).

As more and more companies store their digital assets in the cloud, at least the re-encoding of these files won’t take a re-upload – just a very high speed transfer from one drive array to another. Since one of the key value propositions of cloud encoding is the transport of encoded files to key distribution partners like Hulu and Brightcove, you would expect more and more of these outlets to be served via template, making the bit-level codec knowledge unimportant for their customers.

Today, to utilize a cloud encoding service, you need to know quit a bit about levels and profiles and entropy encoding. In two or three years, as standard targets coalesce, it will be much less so.

Boosting Data Rates Cure All Ills

Back in the bad old days of streaming video, producing at 40 kbps for display over 56 kbps modems had little, if any, margin for error. You had to shoot, edit and encode to perfection to produce even moderately watchable video. Any error at any phase reduced quality to a blocky, streaky mess.

Today, ESPN streams soccer highlights from their television show at 576x324@700 kbps (video only) and quality looks great. Granted, no one watches soccer highlights, but you get the idea. To produce this video, ESPN uses their high motion TV feed, not a Quaalude-like, motion-limited streaming version. And, they’re using VP6, not H.264, so can boost quality even further simply by moving to H.264 (for more on ESPN, see Details of ESPNs new higher resolution VP6 files).

Don’t get me wrong, there’s still a lot of bad streaming video out there, folks forgetting to deinterlace, producing at incorrect aspect ratios, backgrounds that shimmer and text that’s too small or degraded and unreadable. But if you’re distributing at reasonable data rates, producing for streaming has distilled down from an intensive one month course to a two or three day seminar, shorter if you already know how to white balance a camera and light a scene. In three years, producing for streaming will be no different than producing for television or DVD.

What’s this all mean?

  • If you’re a small to mid-size company, or a consultant to same, you have to strongly consider UGC or OVP service providers over a roll your own solution.
  • If you’re developing streaming production courses, reflect these new realities into your curriculum. Over the next two to three years, the focus has to shift from how to produce streaming media into how to use it most effectively.
  • If you’re planning out your career, either inside an organization or as a consultant, understand that the value of understanding deep level H.264 encoding parameters will peak in 2010 or 2011, then quickly decline. Perhaps another technology will take its place, but perhaps not. Not a lot of folks out there making big dollars telling companies how to produce MPEG-2 for DVDs.

Next up, deep thoughts on Silverlight

Comments (17)

Said this on 7-27-2009 At 05:23 am
Great reflections. And I agree in most of your opinions.
But I think you forget a part of the streaming business that can gain advantages by using the Silverlight technology.
A lot of streaming services, that have been on the market for a long time are using wmp, wmv and wmdrm for selling/providing content with strict content protection demands. With the Silverlight/PlayReady technology these companies has the opportunity to make their content available on mac and other platforms with silverlight. And still use their wmv catalog while changing to H.264 for new content. A switch to flash would require a full catalog re-encoding, new streaming environment, new drm technology (with new acceptance from content owners).
Said this on 7-27-2009 At 09:52 am
Thanks for weighing in - I totally agree - this is one of those captive viewer applications - if folks want it bad enough to pay, they'll download a new plug-in to watch it.
Said this on 7-28-2009 At 11:35 am
Thanks, Jan.

I agree with you that requirements for low-level compression knowledge will change as video becomes more standardized. But I also suspect that low-level knowledge of interactive non-linear and two-way video will need to increase during this time, as we move from the movie/TV model to more of a video-communications model... plausible to you, too?

(Rephrased, we'll always need nerds. ;-)

What you say about erecting barriers before audiences makes sense to me too. The situation with Silverlight seems friendlier than that of the WhatWG's "HTML5" proposals though... it's less of a barrier to make a quick plugin download with invisible UI than it is to update a browser, much less switch browser brands. This year's VIDEO hype seems to be ignoring the same types of questions Silverlight ignored a few years ago...?

Said this on 7-28-2009 At 11:51 am
Thanks for weighing in.

Hmmm. Haven't shaved this AM, so am not sure that I like the thought of video-communications. Like the thought of video nerds always having a job, though, since I am one.

To be honest, I haven't looked at HTML5, though it's on the radar screen. I guess I need to move it up. My experience with Silverlight hasn't been that painless - can't easily get it to work in Firefox, though I'm sure it's my problem, not SL. The more I'm online, however, the less I like to download new plug-ins, and I presume I'm not alone in that.

Until I see some middle of the road companies using SL for outbound marketing and sales videos, it's a tough sell for any class I teach. Cripes, look at how few Flash sites there are using H.264, though penetration of H.264-compatible browsers is 85%+.

Thanks for your comments.
Fallon Massey
Said this on 7-28-2009 At 07:31 pm
I agree with most of what you've said, but it amazes me how ignorant people are about these things.

What you've said could have been said about EVERY product ever placed on the market that wasn't the very frist one ever invented.

You NEVER start out with 100% market share, let's grow up for a minute. Market share has to be built!

I won't use Silverlight on the internet for all the reasons you stated, but I will use it on captive audiences, and the runtime will get there sooner rather than later.

If you ever decide to do the intelligent thing, you might want to compare the penetration of Flash at the same time in it's development. I would also pad their number to make it comprable to Microsoft's considerable marketing muscle.

However, MS is seeding Silverlight the right way, and not using it's dominant OS or Browser. Maybe you should note that as well.
Said this on 7-28-2009 At 08:57 pm
Gee, Fallon, are you the same Fallon Massey who said this here?

"Fallon Massey said:
@christian - You're exactly right. It's silly to take Cynergy as an example because most companies developing RIA solutions would go broke trying to use both platforms unless the skills for each were already in place.

Also, IMO, doing both just defies logic under normal circumstances. The only reason to do that now is the totally pathetic penetration of the Silverlight runtime, due to the bungling of it's rollout by Microsoft.

It may be 2 years or more before you can use SL in a general website, so Flash/Flex is your choice simply because users will be able to see it. Only MS people believe that end users will download a control to buy something from a merchant. In the real world, consumers have a choice, and generally pic the choice that requires less work(read Flash)."

Sounds like we're in complete agreement, and now you're calling me ignorant and questioning my intelligence? Hmm. What happened since April 9 -- big brother get to you?

Thanks for weighing in.

Said this on 7-29-2009 At 01:19 am
"... compare the penetration of Flash at the same time in it's development...."

Earliest info Internet Archive has is from the year 2000, when 96% of consumers tested could view Flash 2 content:

(The adoption curve was gentler then, not as rapid as today, but it was still startling, even then ubiquitous.)

Said this on 7-29-2009 At 02:00 am
Here'z an idea, well Microsoft should make a deal with Google and try and get it to use Silverlight for say HD view. I know it sounds like making a deal with the devil (for either side), but Google is in the business of making money. So if Microsoft can flex its financial advantage to make YouTube use Silverlight, it can get the install base to a self-feeding critical mass level.
Said this on 7-29-2009 At 07:44 am
Sounds like a great idea to me. Microsoft needs persistent, ubiquitous content to drive adaption, and while live events like Wimbledon and the Olympics are high profile, they're not persistent.

Good thoughts, thanks for sharing.

Said this on 7-29-2009 At 07:45 am
Thanks for the facts, JD. Not surprising, since Flash delivered something web site creators and their viewers couldn't get elsewhere.

Said this on 7-29-2009 At 02:32 pm
If penetration were such a deciding factor then no product should ever succeed since they all have to start out w/ 0 penetration in the first place.

I recalled the days when .Net came out in 2002 while Java was at its peak. Everyone outside Redmond laughed it off as a stupid Java wannabe w/o future. What now 7 years later?

SilverLight is a platform play, and the fact that it makes web development easier in terms of both speed and cost of development is music to the ears of developers and their managers. There will be more and more SilverLight web apps coming out, and more and more people installing SilverLight. The penetration rate is a non-factor. Heck it's just one Windows Update away from being 90% after all tho that way feels like cheating.
Roger Larsen
Said this on 7-29-2009 At 02:39 pm
I'm pretty sure that flash's adoption rate had nothing to do with the fact that it was bundled with Windows.

Said this on 7-29-2009 At 03:09 pm
>>If penetration were such a deciding factor then no product should ever succeed since they all have to start out w/ 0 penetration in the first place.

Not true - many products are useful without ubiquity; Unfortunately, a browser plug-in isn't one. This makes the comparison with .Net irrelevant. Certainly, I don't have to download .Net to view someone's external web page.

I'm guessing the reason that Microsoft isn't bundling the plug-in is Anti-trust - remember, they ended up paying hundreds of millions to Real to settle anti-trust issues. It's certainly not because they're doing it the "right way." Otherwise, they would have been silly not to bundle it. So it's probably not "just one Windows update away from being 90%."

If it makes so much sense, show me the money - one three letter network, one UGC site, one Online Video Platform who even offers a Silverlight version. Show me a major corporation who uses it for their external site. Show me any meat and potatoes non-techie site that uses it.

Silverlight may be wonderful for some developers, but viewers don't care. Flash succeeded because it let web sites and their viewers do something they couldn't do before. That's why the adoption rate was so fast. At its most base level, Silverlight is an alternative to Flash, nothing new from the viewer's point of view.

If Silverlight came out with Flash, perhaps it would be a two horse race. But it didn't, but now it's playing catchup with a very similar feature set from the user's point of view.

We'll see where it ends up. Certainly Microsoft has the dough to keep investing, and it's a strategic area. My point was to explain why currently it's not a viable solution for most web site.
Roger Larsen
Said this on 7-29-2009 At 04:03 pm
It seems like the arguments for Flash are getting fewer and fewer, and we will keep hearing this argument until the end of time.

It used to be "if SL had this feature and this feature, THEN we should use it", now the only thing left is adoption.

I think you over emphasize the how reluctant people are against installing plugins in their browsers. When I wanted to watch MLB, it stated that it used flash, yet I had to download another third party plugin to view it. Did I care? Not really.

I think you are absolutely right; “viewers don’t care”. They just want a good experience, and that can be done with both Flash and Silverlight. However, there is a horde of .net developers who finds Silverlight extremely appealing, and Silverlight is not going away anytime soon.

What is going against Adobe these days are their aggressive nature and relentless assaults against anyone writing anything positive about Silverlight. Jd/Adobe in particular likes to question peoples motives, rather than the content of articles. This is a sure path towards alienating developers and the community. Ryan Stewart seems to get this point and have decided to advocate in a more positive manner. Kudos for him, I hope more people from Adobe follow his example.

Again, the adoption rate argument is not even an argument. The first fax machine built cost millions of dollars to produce, yet it had no value. Did that mean that we should never have built it? I guess Facebook should never have built their website, because how in the world could they compete with MySpace? ATI should never had built a CPU, or competed with NVidia. NVidia should never have built a product to compete with 3dfx. I think you get my idea.

You’re argument is that you shouldn’t use something until it is a success. Which again is the hen and the egg problem, you have to USE it for it to become successful. So, I sense a different motive.

Your argument is really "I don't want SL to succeed". Which is fine, but before you do, look at smooth streaming, it's pretty awsome ;)

Said this on 7-29-2009 At 04:13 pm
OK, last response.

>>I think you over emphasize the how reluctant people are against installing plugins in their browsers.

Oh yeah? how many machines do you have Real Player installed on?

>>Your argument is really "I don't want SL to succeed". Which is fine, but before you do, look at smooth streaming, it's pretty awsome ;)

So it is. Still waiting for an objective smooth streaming vs. dynamic streaming review.

My argument isn't that SL won't succeed, it's more that no article I saw about SL3 even mentioned penetration and its impact on how and which web sites could actually use it.

In this regard, the adaption rate argument is completely relevant, and your comparisons non-sensical. I can buy one graphics card for my own use. I can't choose a streaming technology until a relevant subset of my viewers have it installed.

Let's check back in 12 months and see where SL is then.

Thanks for weighing in.
Said this on 8-1-2009 At 10:58 pm
I completely agree with the article's basic premise, that Microsoft needs SL because of how RIAs (not to mention AIR) are rapidly threatening Microsoft's control over the software industry, while users and most developers don't need SL for any reason.

But the truth is Flash didn't really offer anything that couldn't already be done either (via Java applets, DHTML, QT, etc), but it did it far better and was easier to develop than all the alternatives. It was successful because it was better.

To some people, SL is that same game. To them SL does what Flash does but better (largely because they already have Microsoft platform skills, so to them anything Microsoft is better)... but for the end users (and Flash devs), it's just nothing new.

But if you doubt whether runtime penetration matters you should consider why Microsoft chooses to use Flash ads to promote Silverlight. ;)
Said this on 8-2-2009 At 01:22 pm

Thanks for weighing in. Excellent final point, wish I had known.


New comments are currently disabled.