A previous blog post entitled Compute VMAF Using FFmpeg on Windows covered this same topic but with a version of FFmpeg compiled by my colleague Abharana (Abi) Bhat, Ph.D., who is the Video Architect, Video Quality at DAZN, the London-based OTT subscription sports streaming service. Now the standard Windows FFmpeg download from Zeranoe includes VMAF; all you have to do is download the models and run the proper command strings.
So, in this post, I’ll detail how to download those models and how to compute VMAF and other metrics with FFmpeg on Windows. I’ll assume that you know how to download FFmpeg from the Zeranoe site and insert the FFmpeg path into your Windows environment which is critical to VMAF operation. If you don’t know how to do this, there’s a fabulous YouTube video that covers it here.
Once FFmpeg is Installed and In Your Windows Path
Now you have download the VMAF model that you want to use. Currently, there are three:
- The default model – vmaf_v0.6.1.pkl, which is “trained to predict the quality of videos displayed on a 1080p HDTV in a living-room-like environment.”
- The 4K model – vmaf_4k_v0.6.1.pkl, “which predicts the subjective quality of video displayed on a 4KTV and viewed from the distance of 1.5 times the height of the display device (1.5H).”
- The anti-hacking model – vmaf_v0.6.1neg.pkl, where “neg” stands for “no enhancement gain.” This reverses out techniques like “sharpening, contrasting, histogram equalization, among others, could boost VMAF scores. While such operations could enhance the image quality as perceived by the end viewers if applied properly, it is evident that VMAF tends to overpredict the perceptual quality even when such operations are overused.”
Note that the VMAF phone model isn’t a separate model but a mode you can run with any of the models shown above. You’ll learn how below.
Here’s how you download the models and the associated .pkl files.
1. Go to the Netflix GitHub model page here.
2. Click the desired model to download. In this case, I’m choosing the .pkl file for the default model. This takes you to another GitHub page for that model. On the top right (partially hidden), right-click the Raw button and choose Save link as.
3. This bit is important. Save the file in the same bin as ffmpeg.exe. If you don’t do this, the command strings shown below won’t work. For me, this is always c:\ffmpeg\bin as shown in the figure below. Note that you can save these models anywhere you’d like, but you’ll have to adjust the command strings below for that custom location.
4. Once you get the .pkl file, follow the same procedure for the .model file. Again, if you don’t get this file, and save it in the c:\ffmpeg\bin folder, the command strings shown below won’t work.
5. Download the .pkl file and .model file for each model that you want to run.
OK, now let’s move onto the command strings.
Creating the Command Strings
Here’s a very simple command string:
ffmpeg.exe -i output.mp4 -i input.mp4 -lavfi libvmaf="model_path='C\:\\ffmpeg\\bin\\vmaf_v0.6.1.pkl':log_path=VMAF.txt" -report -f null -
Pretty simple stuff;
ffmpeg.exe – call the program
-i output.mp4 -i input.mp4 – list encoded file first, then source
lavfi– call librafilter
libvmaf – Identify the filter
"model_path='C\:\\ffmpeg\\bin\\vmaf_v0.6.1.pkl' – path to VMAF model (this is the default model). I’m not quite sure why you need the double backslashes, but I got the command string from the engineer who is creating the Zeranoe FFmpeg builds, and it works. You also need the single quote at the start and end of this path.
:log_path=VMAF.txt – path and name of the file to contain scores
-report – saves the log file (not essential)
-f null - – Tells FFmpeg to output a null file
Note the single and double quotation marks, which must be properly placed or the command string won’t work. So:
- First double quote before
- The first single quote before the path to the model
- The close single quote at the end of the path
- The close double quote at the end of the libvmaf filter commands, in this case, the
You can find all the libvmaf options here. Here are the most relevant ones (IMHO)
model_path – Set the model path which is to be used for SVM. Default value: “vmaf_v0.6.1.pkl”
log_path – Set the file path to be used to store logs.
log_fmt – Set the format of the log file (xml or json).
phone_model – Invokes the phone model which will generate VMAF scores higher than in the regular model, which is more suitable for laptop, TV, etc. viewing conditions (phone_model=true to enable; false is the default).
psnr – Enables computing psnr along with vmaf.
ssim – Enables computing ssim along with vmaf.
ms_ssim – Enables computing ms_ssim along with vmaf.
n_threads – Set number of threads to be used when computing vmaf.
pool – Set the pool method to be used for computing vmaf. Options are
This string incorporates many of these options:
ffmpeg.exe -i output.mp4 -i input.mp4 -lavfi libvmaf="model_path='C\:\\ffmpeg\\bin\\vmaf_v0.6.1.pkl':log_fmt=xml:psnr=1:ssim=1:phone_model=true:log_path=harm_mean.txt:pool=harmonic_mean" -f null -
So, it computed VMAF using the phone model, as well as PSRN, SSIM, and MS_SSIM, and computed the scores via the harmonic mean, which does a better job weighing variations in the stream than the simple mean.
Here’s what the log file looked like (click the figure to see it at full resolution). You see the computed metrics and the frame scores for the first two frames. It took 13.5 seconds to compute all metrics on my HP Zbook notebook.
For those who care about these things, I checked and the VMAF scores computed via the arithmetic mean were identical to those produced by the Moscow State University Video Quality Measurement Tool, which is my go-to tool for metric computations.