Streaming Learning Center Where Streaming Professionals Learn to Excel
Related Articles
Moscow State University’s Video Quality Measurement Tool was already good. Enhancements in the new version, including new metrics and the ability to run multiple analyses simultaneously, make it even better.
Moscow State University’s Video Quality Measurement Tool (VQMT) has always been one of the least expensive and most functional quality measurement tools available. The latest updates (10.0 and 10.1) add new metrics and provide significant user interface, performance, and feature enhancements that make it a must-have upgrade for all serious streaming video producers. Like previous versions, the product costs $999 with upgrade pricing available for some owners of previous versions. Here’s a brief overview of the new features, which we explore in a video at the end of this article.
By way of background, VQMT is a Windows application that also runs via the command line in both Windows and Linux environments. VQMT is a very flexible tool that includes over 15 metrics, both reference and non-reference, many with different variations. The UI version can compare one or two compressed files to a single source, which is great for comparing different encoding techniques or encoding configurations.
In the UI, VQMT graphs the results (see Figure 1) so you can gauge performance over file duration and identify problem areas. For example, Figure 1 compares the same file, an excerpt from Tears of Steel, encoded using constant bitrate encoding (green) and 200% constrained VBR encoding (orange) measured using the new VMAF metric. Though the average scores are quite close, the big green drop off around 30% into the file indicates a potential problem area as does the sharp downward spike on the extreme right.
Figure 1. The results plot comparing CBR and 200% constrained VBR. Note the multiple tabs on the upper left, discussed below.
You can move the slider to any point in the graph, and then click Show Frame on the bottom right to view the source and two encoded frames. It’s a great way to identify any problem areas and to see if the scores actually reflect artifacts that would be noticeable by your viewers.
Key New Features
In terms of new metrics, Netflix’s VMAF is very high on the list. I’ve been working with VMAF since it appeared in the 10.0 beta, and at first the analysis was single-threaded and exceptionally slow. Since then, MSU has done a great job accelerating performance on multiple-core computers like my HP Z840; while VMAF is still slow because it involves multiple metrics, it’s now quite usable. New in 10.1 is the NIQE metric, reportedly one of the best non-reference metrics available.
From a UI perspective, version 10.0 represents a complete rewrite, which initially is jarring but ultimately is vastly superior. There are multiple usability enhancements, including the ability to drag and drop files into the main screen, a nice convenience. More important is the ability to run multiple tests simultaneously, as evidenced by the multiple tabs on the upper left of the figure.
That is, one you start the metric calculation, you can load new compressed and source files into the program, and start another analysis. In previous versions, the program was locked until the initial computation completed. Though you could load multiple instances of the program (and you still can), this was cumbersome and prone to crashing. I’ve run ten simultaneous analyses in version 10.1 without so much as a hiccup, a great way to leverage the 40 cores on my Z840.
The other great usability enhancement in the command line is the ability to analyze files of disparate resolutions without first converting them to Y4M format. For example, to compare a 720p file against the 1080p source in previous versions, you had to scale the 720p file to a 1080p Y4M file. It’s simple and fast, but it’s an extra step, and if you’re analyzing 100 files overnight or over a weekend, and/or working with 4K or larger files, you may not have the disk space available to do so. In version 10, you can now compute any metric without first converting to Y4M.
As an example, I recently completed a 2700p60p VR project that involved 3-minute test files, each of which was 150GB in size in Y4M format. If I had to pre-convert to test, I could test about 5 files without running out of disk space, a severe limitation. With this new capability, disk space is no longer an issue.
Though these features may sound modest, they vastly increase the testing throughput possible with VQMT. If objective quality metrics are an important part of your configuration and testing practice (and they should be), these make the latest upgrade absolutely essential.
