One of the gaps in my online course on Video Quality Metrics and my upcoming course on metrics at Streaming Media West was the ability to compute VMAF using FFmpeg on Wind. I know it’s been possible for quite a while now but since I’m not a coder I can’t compile the version of FFmpeg necessary to make it work.
So, I reached out to friend and colleague Abharana (Abi) Bhat, Ph.D, who is the Video Architect, Video Quality at DAZN, the London-based OTT subscription sports streaming service. Abi compiled the necessary code, which you can download here, and documented how to compile the codec yourself should you want to do so in the future. So, a huge thanks to Abi for her hard work and gracious contribution.
What I’ll do in this post is to document how to run VMAF using the compiled version of FFmpeg, which is pretty straightforward.
The shortest command string that worked was this:
ffmpeg.exe -i output.mp4 -i input.mp4 -lavfi libvmaf="model_path=vmaf_v0.6.1.pkl:log_path=VMAF.txt" -report -f null -
Pretty simple stuff;
ffmpeg.exe – call the program
-i output.mp4 -i input.mp4 – list encoded file first, then source
lavfi– call librafilter
libvmaf – Identify the filter
"model_path=vmaf_v0.6.1.pkl – path to VMAF model (this is the default model)
:log_path=VMAF.txt – path and name of the file to contain scores
-report – saves the log file (not essential)
-f null - – Tells FFmpeg to output a null file
You can find all the libvmaf options here. Here are the most relevant ones (IMHO)
model_path – Set the model path which is to be used for SVM. Default value: “vmaf_v0.6.1.pkl”
log_path – Set the file path to be used to store logs.
log_fmt – Set the format of the log file (xml or json).
phone_model – Invokes the phone model which will generate VMAF scores higher than in the regular model, which is more suitable for laptop, TV, etc. viewing conditions.
psnr – Enables computing psnr along with vmaf.
ssim – Enables computing ssim along with vmaf.
ms_ssim – Enables computing ms_ssim along with vmaf.
n_threads – Set number of threads to be used when computing vmaf.
This string incorporates many of these options and worked for me.
ffmpeg.exe -i output.mp4 -i input.mp4 -lavfi libvmaf="model_path=vmaf_v0.6.1.pkl:phone_model=1:log_fmt=xml:psnr=1:ms_ssim=1:ssim=1:log_path=metrics.txt" -report -f null -
So, it computed VMAF using the phone model as well as PSRN, SSIM, and MS_SSIM. Here’s what the log file looked like (click the figure to see it at full resolution). You see the computed metrics and the frame scores for the first two frames. It took 31.68 seconds to compute all metrics on my HP Zbook notebook. By way of comparison, the VMAF-only computation took 12.46 seconds on the same machine.
For those who care about these things, I checked and the VMAF scores were identical to those produced by the Moscow State University Video Quality Measurement Tool, which is my go-to tool for metric computations.
For more on the pre-conference session at Streaming Media West in a few days:
W4. Objective Quality Metrics 2D & 3D: What They Are, How To Compute Them, & How To Use Them
Monday, November 18: 1:30 p.m. – 4:30 p.m.
Every compression practitioner should understand what objective quality metrics do, how to compute them, and how to use them. This workshop guides attendees through the following steps: an introduction to metrics like PSNR, SSIM, SSIMplus, and VMAF, plus several 3D metrics; how to compute metrics with open source tools like FFmpeg and VMAF Master, as well as third-party tools like the Moscow State University Video Quality Measurement tool and SSIMWAVE VOD Inspector; how to interpret the results; how to create rate-distortion curves and BD-Rate computations: and how to use these tools to choose a preset, compare codecs and encoders, build an encoding ladder, and other functions.
For more on the course, click the course image below, or click here.