Posts about video quality metrics
While standards-based video codecs like H.264 and HEVC tend to dominate, standards-based video quality metrics have never risen to the same usage or attention level. With two innovative and highly accurate metrics now available from the ITU-T, this may change in the near term. Briefly, these models are: ITU-T Rec. P.1203, which calculates the quality…
TL/DR: Netflix updates VMAF to include a new model that reverses out the effects of techniques like sharpening, contrasting, and histogram equalization. As I reported in VMAF is Hackable: What Now?, you can increase VMAF scores via simple image enhancement techniques like sharpening, contrasting, and histogram equalization, which, when applied inappropriately, may not actually improve…
Happy to report a five-star review for my course, Computing and Using Video Quality Metrics: A Course for Encoding Professionals. As far as I know, it’s the only course available that teaches how to compute and use metrics like VMAF, PSNR, SSIMPLUS, and SSIM, including how to create rate-distortion curves and BD-Rate comparisons. The review…
There are many quality metrics that use a variety of scoring mechanisms. One of the most frequently used metrics for subjective testing is called Mean Opinion Score, or MOS. The MOS rating is on a scale from 1-5 as shown below (from Wikipedia). You can use a variety of techniques to compute MOS, but typically…
I just finished one consulting project that involved testing 22 clips with three codecs to produce nine rate-distortion charts per clip (1080p, 720p, 480p for VMAF, SSIM, and PSNR) and am working another with 35 clips with 4 RD charts each with two encoding modes. So efficient chart creation in Excel is now a huge…
If you’re comparing codecs with video quality metrics, you should consider tuning for that metric. However, x264 and x265 don’t have a VMAF tuning option. According to my analysis, it appears that tuning for PSNR is the best option and one you should strongly consider. When working with VP9, there’s an additional complication; tuning for…
Producing the chart above involved encoding 35 files using 5 different presets and then recording start and stop times. Since the files were about 20-seconds long, seconds were important. I’m sure a smarter compressionist than I could write a Python script to harvest the start and stop times but I do it by hand, which…
Just a quick note to let you know about some recent findings relating to the Netflix VMAF metric. By way of background, VMAF launched in June 2016 as the metric that powered Netflix’s per-title encoding engine, replacing PSNR. The fact that Netflix created and used VMAF gave the metric tremendous credibility and because Netflix open-sourced…
This lesson teaches you how to compute VMAF with FFmpeg. It includes a download link to a specially compiled version of FFmpeg that can compute VMAF and to a zipped file that contains the batch files and input/output files shown in the lesson. I’m adding it as a lesson to my course Computing and Using…
Recently, video quality metrics have gotten a lot of hate, primarily from codec vendors. For example, yesterday on LinkedIn, respected Beamr CTO Dror Gill wrote, “everyone agrees that subjective (human) testing is the only accurate way to measure true perceptual quality.” After I pointed out industry support for using video quality metrics, Gill graciously changed…
