MSE — Mean Squared Error — is the simplest possible measure of video quality difference: take every pixel in the original, subtract the matching pixel in the compressed version, square the difference, and average across all pixels in the frame (or the whole video). Lower MSE means closer to the original; MSE of zero means perfect reconstruction. The "squared" part means big errors count disproportionately more than small ones, which roughly matches the intuition that one very visible glitch is worse than many tiny ones.

MSE on its own is rarely reported in modern codec work — it's been almost entirely replaced by PSNR, which is just MSE expressed on a logarithmic decibel scale. PSNR is easier to interpret across content of different bit depths (an MSE of 100 means different things on 8-bit vs 10-bit video, but PSNR normalises that). So you'll see PSNR everywhere and MSE almost nowhere in production reporting, but they're the same underlying measurement.

For a product team, MSE is historical context for understanding PSNR — and a useful warning about how easy raw pixel-error metrics are to mislead with. A classic example: take an image, blur it slightly, and add some noise. Take the same image, blur it differently and add different noise. Both modifications can produce identical MSE to the source while looking dramatically different to a human viewer — one might look soft, the other might look "crisp but grainy". MSE can't tell the difference. That's why modern quality assessment uses perceptual metrics (SSIM, MS-SSIM, VMAF) that try to model human vision rather than raw arithmetic — and why MSE/PSNR alone is now considered an inadequate quality measure in serious research.

Book cover: Video Encoding, Volume 1 of the Fora Soft Video Engineering Handbook

The book · Volume 1 of 7

Video Encoding: A Complete Guide to Video Compression — From Pixels and Color to H.264, HEVC, AV1, Quality Metrics, and AI

MSE is the simplest quality number and the basis of PSNR, which makes it easy to compute and easy to misread. Volume 1 covers MSE in detail within the chapter on objective metrics: how it relates to PSNR, why it ignores how humans see, and when to switch to SSIM, MS-SSIM or VMAF. Written by Nikolay Sapunov, CEO at Fora Soft.

Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook

Using MSE or PSNR as your quality gate and seeing results viewers disagree with?

Share your current metric and a few problem clips: we propose a quality gate that combines MSE-based scores with a perceptual metric and sets thresholds for each. 250+ video products shipped since 2005.

Book a 30-min call →WhatsApp →Email us →