Subjective testing is the formal process of having human viewers rate video quality under controlled conditions to produce ground-truth quality scores. Where objective metrics (PSNR, SSIM, VMAF) try to predict perceived quality from numerical analysis of the video, subjective testing measures it directly: real people, real eyes, real opinions. It's slower and more expensive than running objective metrics, but it's the only way to be sure that a "quality improvement" looks better to actual viewers.
The methodologies are standardised by ITU-R Recommendation BT.500 (for video) and ITU-T Recommendation P.910 (for multimedia and video conferencing). The most common protocols: ACR (Absolute Category Rating — rate each clip alone on a 5-point scale), DSCQS (Double-Stimulus Continuous Quality Scale — rate source and compressed side-by-side on a slider), DCR (Degradation Category Rating — compare each clip to a hidden reference), SSCQE (Single-Stimulus Continuous Quality Evaluation — continuous rating of long clips). Each has trade-offs in sensitivity, scalability and viewer fatigue. Typical panels are 15–30 viewers; viewing conditions, displays and content selection are all carefully controlled.
For a product team, subjective testing is the final-validation step before any major encoding-pipeline change rolls to production. The pattern: optimise your encoder using objective metrics (VMAF is the everyday choice), then validate that the change actually looks better to humans through a subjective test on a representative sample of your content. Commercial labs (TestDevLab, Subjectify, Eyevinn Technology, Brightcove research division) run subjective panels professionally for around $5K–25K per study. Major streaming services (Netflix, YouTube, Disney+) maintain in-house subjective-testing capabilities and run them whenever they ship a codec change at scale. Don't ship a new codec or ladder to your audience without one.
