The outlier ratio is a validation statistic that captures consistency, the most practical of the three validation questions: how often a metric is wrong by more than human uncertainty can explain. It is the fraction of clips where the metric's prediction error is larger than the viewers' own disagreement on that clip, specifically larger than the 95 percent confidence interval of the MOS, as defined in ITU-T P.1401. A low outlier ratio means the metric rarely surprises you, because even a model with a fine average error can blow one clip in twenty so badly that the mistake is dangerous; the outlier ratio is what catches that. It runs from 0 to 1. Its blind spot is that it depends on how tight the subjective test's confidence intervals are: looser intervals from a noisy panel make the metric look more consistent than it is. The outlier ratio sits beside SROCC, PCC, and RMSE as the four core statistics a trustworthy validation reports together, each guarding against a different way of flattering a metric.
