ITU-T P.808 (06/2021) is the framework for crowdsourced subjective testing - viewers rating clips on their own devices at home, instead of in a controlled lab. Written originally for speech quality, its approach has been adopted by the video field. Crowdsourcing trades control for scale and speed: you cannot set or measure the viewer's lighting, display, or distance, so P.808 leans on layered reliability defences in place of an operator in the room - qualification screens, hidden gold-standard clips, attention traps, and time-on-task gates - followed by heavy after-the-fact filtering. Rejection rates can be very large; one 2025 study dropped about 92 percent of participants after rigorous consistency screening. Yet with strict screening, crowdsourced MOS can correlate with lab MOS at around 0.95, which is why the method is trusted for the right questions. The rule of thumb: a lab answers which encode is better under ideal viewing, a crowd answers which is better in the wild, and you must say which you ran.
