VQEG, the Video Quality Experts Group, is a body of researchers and industry engineers who run the formal validation campaigns that decide whether an objective video-quality metric is trustworthy. Its tests gather fresh human ratings on shared databases and compare candidate metrics against them under standardized rules. VQEG's most famous result is also a cautionary tale: in its FRTV Phase I campaign, finalized in 2000, of the nine full-reference models submitted, seven or eight performed about the same as one another and all performed about the same as plain PSNR, leading the ITU to judge none accurate enough to standardize. VQEG later diagnosed why the test could not separate the models: the clips spanned too narrow a quality range, so there was not enough spread in the scores to discriminate, a lesson about range restriction that still bites. VQEG also enforces the held-out-content rule, that a metric trained on a dataset must not be compared against one that was not. Its framework underlies the ITU-T P.1401 statistics, PCC, SROCC, RMSE, and the outlier ratio.
