Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
Just a quick note to let you know about the latest book by Jan Ozer, the author of this article. Here’s a brief description from Amazon.
Video Encoding by the Numbers teaches you to optimize the quality and efficiency of your streaming video by objectively detailing the impact of critical configuration options with industry-standard quality metrics like PSNR and SSIMplus. This takes the guesswork out of most encoding decisions and allows readers to achieve the optimal quality/data rate tradeoff.
Since all videos encode differently, the tests detailed in the book involve eight different videos, including movie footage, animations, talking head footage, a music video, and PowerPoint and Camtasia-based videos. The book walks the reader through quality testing, basic encoding configurations, encoding with H.264, HEVC, and VP9, and and encoding for adaptive streaming.
When appropriate, the chapters conclude with a section detailing how to configure the options discussed with FFmpeg, a preferred tool for high-volume video producers, including packaging into HLS and DASH formats (the latter with MP4Box). You’ll also learn how to use key Apple HLS creation and checking tools like Media File Segmenter and Variant Playlist Creator.
You can learn more about the book here.
Here’s the article you came to read. Thanks for coming and we hope you find it useful.
Contents
Streaming 101: The Basics – Codecs, Bandwidth, Data Rate and Resolution
The ability to produce streaming media has suddenly become a critical skill set for many web producers. However, though many of us use streaming-related terms like data rate and bandwidth every day, there may be some residual lack of certainty as to what they precisely mean and why they are important. For this reason, in this introductory article on streaming media concepts, I define file related terms like codecs, bandwidth, frame rate, data rate and resolution, and then delivery options like streaming and progressive download.
Codecs
Codecs are compression technologies with two components; an encoder to compress the file in your studio or office and a decoder to decode the file when played by the remove viewer. As the nifty capitalization in the previous sentence suggests, the term codec is a contraction of the two words encoder and decoder.
There are still image codecs (like JPEG), audio codecs (like MP3) and more video codecs than you could shake a digital stick at. Currently, there’s H.264, VP6, Windows Media and Sorenson Spark in the streaming space, with MPEG-2 dominating the DVD and Blu-ray spaces. H.264 and MPEG-2 are huge in the network and particularly satellite spaces.
It’s instructive to distinguish codecs from development and delivery environments. For example, QuickTime and Video for Windows are two development environments that support a number of codecs, including DV, MPEG-2, AVCHD and DVCPROHD, just to name a few. If you see an AVI file, you know it’s a Video for Windows file; ditto for MOV and QuickTime. But, each format can contain multiple codecs.
Similarly, Windows Media, Flash and QuickTime are the most widely used distribution environments, and each can contain multiple codecs. For example, each delivery environment can deliver video encoded with the H.264 codec.
When producing your streaming files, you typically need to know which delivery environment you’ll be using, since you may prepare the file differently for the different environments. For example, H.264 files encoded for Flash have an .f4v extension, but QuickTime encoded H.264 files with an MOV extension work fine for QuickTime delivery.
Ensuring Playback
To ensure that your files will play on the remote client, you need to make sure that they have the appropriate player installed. For example, with the Flash Player installed, a computer can play H.264 video delivered via the Flash distribution environment, but not Silverlight – for that, you’ll need the Silverlight player.
It’s not as complicated as it sounds; most producers know the distribution environment that they’re targeting and that their potential viewers need the appropriate player. What’s important is that you understand how a codec differs from a development or delivery architecture. Next time someone says “they’re producing in Flash,” you can ask, “Oh, which codec?”
Or, if someone commits the incredible gaffe of saying “I’m producing a QuickTime file,” you can respond haughtily “please don’t be so imprecise. Do you mean that you’re encoding with a QuickTime-based encoder, or producing an MOV file? And, by the way, QuickTime is a developmental and delivery environment, but not a codec, so which codec are you using?”
Bandwidth
Bandwidth is the viewer’s connection speed to the Internet. To a great degree, this connection bandwidth controls your viewer’s ability to retrieve and play video smoothly over the Internet. Higher delivery bandwidths, like those enabled with cable and DSL, allow you to stream higher quality video to your viewer. In contrast, those viewers who still connect via modem won’t be able to view most of the video available on the Internet, at least not in real time. More on that in a moment.
Note that in the early days of streaming video, producers encoded video to meet the bandwidth capabilities of their target viewers. That is, back when most viewers connected via modems, you had to produce postage stamp sized video compressed to somewhere south of 28.8 kilobits per second (kbps) or the viewers couldn’t watch it. Today, with most viewers connecting via broadband capable of 1-2 megabits per second (mbps), most producers encode their video to meet quality and cost concerns.
For example, if you scan the websites of television networks and/or large corporations, most videos max out at 640×480 resolution at around 600-700 kbps, even though many viewers have the capacity to watch higher bit rate streams. That’s because these producers have to pay for their bandwidth and have decided that 640×480 video at 600-700 kbps provides a sufficiently high quality experience to meet their viewer’s needs. In short, back in the day, most producers encoded their files to meet the target bandwidth of their lowest common denominator viewer. Today, as bandwidth to the home has increased, it’s largely a cost/quality issue.
Data Rate
The Data Rate (or bit rate) is the size of the video file per second of data, usually expressed in kilobits or megabits per second. When I say that ESPN distributes their video at 600Kbps, this means that each one-second chunk of audio and video comprises about 600 kilobits of data.
Figure 1. Setting the Data Rate for the file at 468 kbps.
As you can see in the Figure, you set data rate along with other configuration options during the encoding process. Note that to a large degree, data rate is the most important factor in streaming video quality. That’s because all streaming codecs use what’s called “lossy” compression, which means that the more you compress, the more quality you lose. For this reason, all other file characteristics (like resolution, frame rate or codec) being equal, the lower the data rate, the lower the quality of the compressed file.
Frame Rate
Most video starts life at 29.97 or 24 frames per second (fps). Usually, those that shoot at 24 fps deliver at that rate, while many producers that shoot at 29.97 fps deliver at 15 fps to save bandwidth. Though, in concept, it feels like dropping the frame rate by 50% would also drop the data rate by 50% with no loss in quality, it seldom works this way. Rather, according to the research that I’ve performed, the average data rate of video produced at 15 fps is about 20% lower than that produced at 30fps, not 50%. Still a substantial reduction, but often that comes at a subtle quality cost.
For example, when considering 15 fps, note that high motion video will look noticeably choppy to many viewers. In addition, tight facial shots, where lip synch is critical, often a look a bit out of sorts at 15 fps as well. When deciding which frame rate to use for your video, you should produce video at full frame rate and the lower rate, and then compare to see which delivers the best overall presentation.
Resolution
Resolution is the height and width of the video in pixels. Most video is original stored at either 720×480 (standard definition) or 1920×1080 (high definition), but gets sampled down to smaller resolutions for streaming, usually 640×480 resolution or smaller. That’s because as the number of pixels in the file increase, you have to allocate more data rate to maintain the same quality.
Figure 2. A 640×480 video has four times the pixels of a 320×240 file.
For example, a 320×240 video has 76,800 pixels in each frame, while a 640×480 video file has 307,200, or four times more, which is evident in Figure 2. That means you have to apply four times the compression to achieve the same data rate, which usually means noticeably reduced quality. That’s why data rate and resolution are integrally linked in quality related streaming decisions. As you can imagine, a video data rate of 250 kbps should produce excellent quality at 320×240 resolution, but would look disastrous at 640×480 resolution.
When producing streaming video, you have two options. Option 1 is to choose a data rate, then produce at the highest resolution that looks good at the data rate. Option 2 is to choose the desired resolution, then produce at the data rate necessary to produce good quality at that resolution. The key point is that you should always consider one when discussing the other. Just for the record, note that the most common video resolutions for 4:3 video are 640×480, 440×330, 400×300, 320×240, 240×180 and 160×120. The most common resolution for widescreen 16:9 videos are 640×360, 480×270 and 320×180.
Editor’s Note: You can find a more up-to-date article on bandwidth, data rate, and resolution here.
___________________________________________
If you’re finding this article helpful, note that the author of this post now has a book out entitled
Video Encoding by the Numbers. Click here for more information.
___________________________________________
Delivery Modes
It’s important to recognize that when you deliver video over the Internet, you have multiple options, including streaming, progressive download and download and play. Note that the mode you choose will have significant impact on how you produce your files.
Streaming
The concept of streaming means that you click the button on a website, the video starts playing immediately, and it continues to play more or less smoothly to the end. When you stream video, the data rate of the encoded file must be somewhat smaller than the bandwidth capacity of the remote viewer; otherwise, the video will frequently stop playing. For example, if you try to watch ESPN.com on your broadband connection, the videos will probably stream smoothly from start to finish. If you connect via a 28.8Kbps modem, the video will stop and start like pre-strike Major League Baseball salary-cap negotiations.
Video delivered via streaming is delivered via a streaming server. Some streaming servers have specific requirements for files that they deliver; for example, files delivered via the Apple streaming server must be “hinted” for streaming. In addition, streaming performance is usually enhanced if you encode your file using constant bit rate (CBR) encoding, which I’ll define in a subsequent article.
So, when producing files for delivery via streaming, you should:
a. Encode at a data rate that’s comfortably less than the bandwidth of most target viewers
b. Determine if there are any streaming server specific requirements for these files. More on this I future articles.
c. Encode using CBR data rate control.
Progressive Download
Progressive download is a fancy name for video delivered by a regular HTTP web server, and not a streaming server. In most instances, video delivered using this technique is stored to the viewer’s hard drive as it’s received by the server, and then played from the hard drive. In contrast, streaming video is usually not cached locally, so is more secure.
The benefit of progressive download (in addition to saving the costs and administration of a streaming server) is that though the initial playback may not be that smooth, if you wait long enough, the video will play smoothly because you’ve got a local copy stored to your disk. Apple reportedly pioneered this technique for George Lucas, who famously said, referring to one of his Star Wars movies, “there are 24 frames per second in this movie and none of them are optional.”
I still remember my oldest daughter waiting hours for a high-resolution preview of the Disney movie Dinosaur to download over our modem connection. When it finally finished, it looked great. Like George Lucas, when you distribute your videos via progressive download, you can encode at a data rate high enough to ensure compressed quality. If you’re concerned that your viewers won’t wait even a few minutes for high-quality video, you can always post a low-resolution version for quick viewing and a high-resolution version for optimal quality.
However, note that the hard-drive caching involved with progressive download makes your video easier to pirate. Streaming is a much safer alternative because the video you deliver is never resident on the end user’s hard drive.
Again, as with files produced for streaming, note that some progressive download technologies also have specific requirements. For example, files delivered to the QuickTime Player must be encoded with the Fast Start option selected.
Finally, most producers who deliver via progressive download produce their files via variable bit rate (VBR) techniques, which I’ll define in a subsequent article, since it delivers the optimum blend of file size and quality. To summarize, when producing for deliver via progressive download, you should:
a. Encode at a data rate the presents a decent balance between quality and delivery waiting time
b. Make sure that you comprehend any technology specific requirements for progressive delivery
c. Encode your files via VBR data rate control.
Download and Play
In some instances, you may choose to force the viewer to completely download the video file before they can play it. In this case, there are typically no production specific requirements for these files.
That’s it for now. In the next article, I’ll dig a bit deeper, defining terms like codecs, and discussing compression related parameters like constant vs. variable bit rate encoding, and frame types like I, B and P frames. If you’ve ever stared at an encoding configuration screen and wondered what these terms meant and how to choose from the available options, tune in next week.
