Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
Contents
Figure 1. The four critical aspects to consider before video encoding.
Executive Summary: While video quality is always top of mind, consider deliverability, compatibility, and playability before finalizing your video encoding presets.
Too often we look at video encoding with a single point of focus; achieving the best possible quality for the lowest possible data rate. However, as I was reminded when I spoke on a panel at Streaming Media East with several encoding pros from Beamr, Brightcove, Verizon, and Yahoo, achieving the best possible file at the lowest possible data rate is useless if you can’t deliver the file, or play it once delivered. It was a great reminder that whenever you encode a file, you have four major concerns, quality, compatibility, deliverability and playability. In this brief article, I’ll define each term and suggest some tools you can use to help measure performance in each.
Quality
Achieving the best possible quality for each bitrate is certainly the most obvious goal for compressionists, and there are two sides to this equation, quality and file size. Regarding quality, I’m a big user of objective video quality metrics, which attempt to mathematically compute how real world viewers would subjectively rate the videos.
The tool I use the most is the Moscow University Video Quality Measurement Tool (VQMT), which can compute multiple quality metrics like Peak Signal-to-Noise Ratio (PSNR), Structured Similarity Index (SSIM), and Video Quality Metric (VQM). In addition to producing a mathematical score, the program can compare two files and display the results visualization graph shown below, which tracks the comparative score over the duration of the file and lets you compare frames from the original file to the encoded files by clicking Show frame on the bottom right. You can read my review of the tool here.
Figure 2. The Results Visualization screen of the Moscow University VQMT.
Another tool I use frequently for quality comparisons the SSIMPlus Quality of Experience Monitor (SQM). SQM has two unique and useful features. First, it uses the SSIMPlus algorithm to rank videos, which produces a score that purports to directly correlate with viewer’s subjective perception; 80-100 scores predict an excellent rating, 60-80 predict a good rating, and so on. In addition, the tool can predict subjective ratings on a variety of platforms, from smartphone to 4K TV. Here’s my review of SQM.
The other aspect of the quality is file size. Don’t assume that your encoder is delivering the selected data rate unless you’ve actually checked. When I’m working with test files, I calculate the target file size, which is seconds x (video + audio bitrates)/8 (for the bits to byte conversion) and verify accuracy within 5% or so. I also use the tool Bitrate Viewer, shown below, which can analyze H.264 files and report the data rate.
Note that you can’t test a single file and feel comfortable with the result. For example, while a talking head video might compress accurately, an action video might stress the encoder and deliver a much higher bitrate stream. At the other end of the spectrum, synthetic videos like screencams or PowerPoint tutorials are often way too low, which can lead to degraded quality video. So test multiple times, checking all types of videos that you typically encode.
Deliverability
The concept of deliverability relates to the ability to smoothly deliver the file, which is particularly important over constrained connections like cellular. This involves the bitrate control technique used to encode your file (constant vs. variable bitrate encoding), which I detail in a long discussion here, or capped CRF, as shown in Figure 3. Though the average bitrate of the file shown in Figure 3 is 509 kbps, the peak bitrate of 1990 kbps is nearly four times higher, which definitely could cause deliverability issues over constrained connections.
Figure 3. Bitrate Viewer showing an average bitrate of 509 kbps and a peak of 1990 kbps.
No matter which bitrate control technique you deploy, you should visualize the bitstream with a tool like Bitrate Viewer. Obviously, this is particularly useful if you’re experiencing stoppages during playback and want to learn why. You can learn more about where to get Bitrate Viewer and how to use it in a video tutorial located here. Another tool you can use for this is Telestream Switch, which gives a lot more detail about the encoded file, though you’ll need to spend $300 to get the Pro version for this feature.
Compatibility
Compatibility relates to the player’s ability to load and play the file. Typically, the biggest issue here is the H.264 (or soon HEVC) profile used for the video. When encoding for computers and OTT, you can use the High profile, which delivers the best quality. When encoding for mobile, you have to take into account that older, slower devices can only play the Baseline profile. Most iOS producers use the encoding ladder recommended in Apple Tech Note TN2224, shown below, as a starting point for this. 
Table 1. The compatibility matrix presented in Apple’s Technote TN2224.
Note that the Android operating system can only decode Baseline-encoded video. While this is usually supplemented by hardware-assisted playback from the device manufacturer, this isn’t always the case. My Nexus 10 Android tablet, for example, will only play Baseline-encoded video. For this reason, some producers encode all streams targeted towards mobile Android devices using the Baseline profile.
Playability
The fourth pillar, playability, has two related components; the CPU required to playback the file, and the battery life that playback consumes. Playability is obviously critical with mobile devices, but is also an issue with desktops for producers distributing in 4K or with UHD codecs like HEVC and VP9.
When recommending an encoding configuration for mobile, always check the CPU required to playback that stream. On the Android platform, I use an app named CPU Monitor Advanced Pro (Figure 4). If I see a number like 100%, as shown on the top graph in Figure 4, I know that frame rate will likely be less than 100%, and that battery life will suffer. On iOS, I use the System Status app to check CPU utilization. On Windows computers I use Performance Monitor, while on the Mac platform I use Activity Monitor.
Figure 4. Android CPU utilization as shown in CPU Monitor Advanced Pro.
Interestingly, it was the playability aspects that the two gents from Verizon and Yahoo claimed they spent the most time debugging, particularly with mobile. This will become an issue on desktops and notebooks when producers start deploying HEVC, VP9, or the AV1 codec from the Alliance for Open Media.
Video Encoding Recommendations
The bottom line is that before deploying a new encoding configuration, you should:
1. Verify output quality both objectively and subjectively
2. Verify that the encoder is meeting the target bitrate with a variety of files.
3. Check the bitrate profile with a tool like Bitrate Viewer or Switch.
4. Verify playback compatibility by checking the requirements of each target platform.
5. Check playback CPU utilization on the lower end of your target platforms to ensure playback at full frame rate and that your files don’t consume too much power.
Additional Resources
Encoding for Multiple Screen Delivery, Udemy ($35)
Producing Streaming Video for Multiple Screen Delivery, Jan Ozer (on Amazon)
About the Streaming Learning Center: The Streaming Learning Center is a premiere resource for companies seeking advice or training on cloud or on-premise encoder selection, preset creation, video quality optimization, adaptive bitrate distribution, and associated topics. Please contact Jan Ozer at [email protected] for more information about these services.
