Ground truth in video quality is the human verdict, the subjective Mean Opinion Score produced when a panel of viewers rates clips under controlled conditions. It is the bedrock the field rests on, because video quality is ultimately defined by the people watching. Objective metrics earn their authority entirely by reference to it: SSIM was validated by showing it tracks human ratings better than older pixel-error measures, and VMAF is trained directly on subjective opinion scores using machine learning. The standard way to measure agreement, set out in ITU-T P.1401, uses the Pearson correlation for accuracy, the Spearman rank correlation for ranking, and RMSE for error size. The catch is that ground truth is expensive and slow to gather, needing at least fifteen observers, controlled conditions, and a reported confidence interval, so it cannot run on every encode. The governing rule is simple: when a metric and a properly run subjective test disagree, the ground truth wins and the metric has hit a blind spot on that content.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
Ground truth for video quality is what viewers say in a proper subjective test, and every objective metric is only as good as its fit to it. We wrote seven volumes covering the whole video stack, and for ground truth start with Volume 1: subjective scores, objective metrics and how they are trained and checked, with Volume 6 on model evaluation. Written by Nikolay Sapunov, CEO at Fora Soft.
Not sure whether your quality metric still matches ground truth for your content?
Send us your content and metric scores: we run a small subjective test and tell you how well the metric tracks viewers and where it misleads. 250+ video products shipped since 2005.
