In metric validation, the root-mean-square error (RMSE) measures accuracy: the typical size of the gap between a metric's predicted score and the actual human MOS, expressed in the MOS scale's own units. It is 0 when the prediction is perfect and grows as errors grow, so a smaller RMSE means the metric's numbers land closer to where people put them. Like the Pearson correlation, RMSE is computed only after a monotonic fitting curve (a five-parameter logistic) maps the metric onto the MOS scale, because the raw scales do not line up. Its blind spot is that, as an average of squared errors, it can hide a rare large miss behind many small ones, and because it depends on the score scale it is not comparable across different metrics. RMSE is one of the four ITU-T P.1401 validation statistics alongside SROCC, PCC, and the outlier ratio; together they grade monotonicity, accuracy, and consistency. As an example, an outside reproduction reported a VMAF-DMOS RMSE of 12.7 VMAF points on a 4K set.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
RMSE tells you how far a quality metric's predictions sit from real viewer scores, which matters before you trust it to make decisions. Across our seven-volume handbook on the video stack, Volume 1 is where to start for RMSE and metric validation: VMAF, SSIM, PSNR, subjective MOS and how objective metrics are checked against human ratings. Written by Nikolay Sapunov, CEO at Fora Soft.
Validating a quality metric and need to read its RMSE against MOS?
We will validate your chosen metric against human scores on your content and report RMSE, correlation and where it fails. 250+ video products shipped since 2005.
