The Lagrangian cost is the single mathematical formula that modern video encoders use to decide every coding choice. Written as J = D + λ·R, it combines two competing concerns into one number that can be minimised. D is the distortion (how much visible quality you'd lose with this choice). R is the rate (how many bits this choice would use). λ (lambda) is a tuning factor that translates between bits and distortion — essentially "how many bits is this much quality worth right now?".

Why this is important: every block in every frame has hundreds of possible coding choices — which prediction mode, which block size, which transform, which quantization, which reference frame. The encoder can't pick the best one without a way to score them. Pure quality optimisation would produce huge files; pure size optimisation would produce ugly video. The Lagrangian cost gives one number that balances both, and the encoder picks whichever choice has the lowest J. This is the engine behind rdo — Rate-Distortion Optimisation — which drives almost every decision in a modern encoder.

For a product team, the practical insight is that λ is the dial that converts "quality target" into "compression effort". Lower λ means quality matters more relative to bits — the encoder spends bits aggressively. Higher λ means bits matter more — the encoder accepts more distortion to save them. This is what's actually happening under the hood when you set CRF: CRF is a user-facing knob that the encoder internally translates into a λ value, then runs RDO at that λ across every block. Modern perceptually-tuned encoders (psy-rd, content-adaptive) modify the cost function further to weight different parts of the picture by how much viewers will actually care.

Book cover: Video Encoding, Volume 1 of the Fora Soft Video Engineering Handbook

The book · Volume 1 of 7

Video Encoding: A Complete Guide to Video Compression — From Pixels and Color to H.264, HEVC, AV1, Quality Metrics, and AI

Lagrangian cost J = D + λR is how an encoder weighs quality loss against bits for every decision, and λ tuning moves the whole rate-quality curve. Volume 1 covers Lagrangian cost in detail: rate-distortion optimization, mode decision, the link between λ and quantization, and how H.264, HEVC and AV1 encoders apply it. Written by Nikolay Sapunov, CEO at Fora Soft.

Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook

Tuning RDO and Lagrangian cost parameters in your encoder?

Tell us about your encoder work: we review the rate-distortion setup, test λ choices on your content and report the quality and bitrate effects. 250+ video products shipped since 2005.

Book a 30-min call →WhatsApp →Email us →