Subjective testing is the formal process of having human viewers rate video quality under controlled conditions to produce ground-truth quality scores. Where objective metrics (PSNR, SSIM, VMAF) try to predict perceived quality from numerical analysis of the video, subjective testing measures it directly: real people, real eyes, real opinions. It's slower and more expensive than running objective metrics, but it's the only way to be sure that a "quality improvement" looks better to actual viewers.

The methodologies are standardised by ITU-R Recommendation BT.500 (for video) and ITU-T Recommendation P.910 (for multimedia and video conferencing). The most common protocols: ACR (Absolute Category Rating — rate each clip alone on a 5-point scale), DSCQS (Double-Stimulus Continuous Quality Scale — rate source and compressed side-by-side on a slider), DCR (Degradation Category Rating — compare each clip to a hidden reference), SSCQE (Single-Stimulus Continuous Quality Evaluation — continuous rating of long clips). Each has trade-offs in sensitivity, scalability and viewer fatigue. Typical panels are 15–30 viewers; viewing conditions, displays and content selection are all carefully controlled.

For a product team, subjective testing is the final-validation step before any major encoding-pipeline change rolls to production. The pattern: optimise your encoder using objective metrics (VMAF is the everyday choice), then validate that the change actually looks better to humans through a subjective test on a representative sample of your content. Commercial labs (TestDevLab, Subjectify, Eyevinn Technology, Brightcove research division) run subjective panels professionally for around $5K–25K per study. Major streaming services (Netflix, YouTube, Disney+) maintain in-house subjective-testing capabilities and run them whenever they ship a codec change at scale. Don't ship a new codec or ladder to your audience without one.

Book cover: Video Encoding, Volume 1 of the Fora Soft Video Engineering Handbook

The book · Volume 1 of 7

Video Encoding: A Complete Guide to Video Compression — From Pixels and Color to H.264, HEVC, AV1, Quality Metrics, and AI

Subjective testing is the only way to be sure viewers like what metrics approve, and a poorly designed test gives noise. Volume 1 explains subjective testing in detail: ITU-R BT.500, ACR and DSCQS methods, viewing conditions, panel size, MOS statistics, and how results calibrate VMAF and other metrics. Written by Nikolay Sapunov, CEO at Fora Soft.

Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook

Need a subjective test to confirm a codec or ladder change before rollout?

Tell us what you are changing: we design a subjective testing protocol, run the comparison and report MOS with confidence intervals and a go or no-go. 250+ video products shipped since 2005.

Book a 30-min call →WhatsApp →Email us →