Subject screening is the removal or down-weighting of raters whose votes do not reflect quality - viewers who misread the scale, lose attention, or rate at random - so their noise does not pollute the MOS. It happens in two stages. Pre-screening checks eyesight before the test (acuity on a Snellen chart, colour vision on an Ishihara plate). Post-screening, the subject of this term, runs after the data is in, with three families that trade simplicity for accuracy. The ITU-R BT.500-15 kurtosis rule is a hard test rejecting a subject frequently outside a 2-sigma band whose misses are balanced high and low - random voting - applied once, to small non-expert panels. The P.910 correlation method drops subjects whose ratings correlate weakly with the panel MOS, catching the consistently-wrong voter the kurtosis test misses. The newest, soft approach is the subject-behaviour model (standardized in P.913), which estimates each subject's bias and inconsistency and forms a bias-subtracted, consistency-weighted MOS instead of discarding data. Always fix the rule before seeing the data.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
Subject screening keeps a few careless raters from bending the MOS, but the rule has to be fixed before anyone looks at the data. Our seven-volume handbook covers the video stack end to end; for subject screening start with Volume 1, which explains subjective testing, MOS and its confidence, and validating quality metrics against viewer scores. Written by Nikolay Sapunov, CEO at Fora Soft.
Running a subjective test and unsure which raters to screen out?
We will review your test protocol and raw scores and set a subject screening rule that holds up to review. 250+ video products shipped since 2005.
