Pair Comparison (PC), defined in ITU-T P.910 section 8.4 and called stimulus-comparison in ITU-R BT.500-15, shows the viewer two clips and asks the simplest possible question: which one is better? There is no rating scale, just a forced choice. Stripping away the scale removes the hardest part of any rating task - a viewer who cannot decide between a 3 and a 4 can almost always pick the one they prefer - which makes PC the most sensitive method when items are very close, the tool to separate encoders an absolute scale reports as tied. Turning the win-loss votes into a quality scale takes a model: Thurstone's Law of Comparative Judgment or the Bradley-Terry model, often in just-objectionable-difference units. The brutal weakness is arithmetic, not perception: distinct pairs grow as N times (N minus 1) over 2, so 12 conditions need 66 pairs and 40 need 780. It is reserved for small sets or run with active sampling (such as ASAP) that tests only the most informative pairs.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
Pair comparison is the most sensitive subjective method, but the number of pairs grows fast as you add encoders or settings. We wrote seven volumes covering the whole video stack, and for pair comparison start with Volume 1: subjective scores, rating methods and the codec and preset choices being compared. Written by Nikolay Sapunov, CEO at Fora Soft.
Comparing near-identical encodes where only pair comparison will show a difference?
Send us your candidate encodes: we design a pair comparison test that keeps the pair count manageable and give you the analysis plan. 250+ video products shipped since 2005.
