Compressionists: You’re Only as Good as Your Tools

I’m reviewing AV1 encoders for Streaming Media Magazine and was performing some QC on the AV1 files that I produced with the latest version of FFmpeg. Two file details I always check are GOP size and whether the GOPs are closed or not. I check these for multiple reasons.

First, FFmpeg doesn’t have the most robust error messaging so it’s easy to mangle a switch (open-gop vs. opengop, for example), and end up with an open GOP. In addition, the test files I’m using have multiple frame rates, including 24, 23.976, 25, 29.97, 30, and 60fps, so it’s incredibly easy to hose the GOP size when you’re adopting your script to the different files. Third, while these configuration options make incredibly little difference to ultimate quality, you want to get these options right when sharing your command strings or encoded files. So, I check.

I started checking a 24 fps file in Zond 265 from Solveig Multimedia ($1,390 for the version with AV1 support) which is the featured image atop this article. Checking for closed GOPs on the right, I saw there were 6 closed GOPs and no open GOPs. So far, so good. Then I noticed that the GOP length was 71 frames when it should have been 48. Ruh-roh.

I started by checking my command strings which looked correct. Then I thought to load the file into Elecard StreamEye 4.0 which I had recently installed as part of Elecard StreamEye Studio (contact Elecard for pricing). This showed the information below. Specifically, though the placement in the stream was frame 71, the frame was displayed as frame 48. 
Mystery solved, the FFmpeg commands work as advertised, and StreamEye saved me countless hours of research and debugging. To be fair to Solveig Multimedia, the Zond tool may have offered the same information in another screen or window, but I had StreamEye loaded and it was easier to simply check there.

These tools are expensive and in many ways targeted more for codec developers than compressionists. But, if you’re going to be working with advanced formats like HEVC, AV1, and soon VVC, you need an analysis tool that can tell you whether your encoder is producing the properly configured output. Otherwise, you’ll be flying blind which is uncomfortable even if you won’t be publishing your results.

Though I only understand about 5% of what these tools can tell me, it’s an absolutely critical 5%, and I couldn’t do my job effectively without them. I’ll be experimenting more with these tools as I start encoding with different AV1 codecs and may come back with additional tidbits.