Quantization is the bit-saving step inside every lossy codec, and the place where the loss actually happens. After the DCT rewrites a block as wave coefficients, quantization divides each coefficient's strength by a step size and rounds to the nearest whole number, which deletes the small high-frequency waves entirely. The coarser the step - the lower the target bitrate - the more waves round to zero and the more each block collapses toward a single flat tone. This single mechanism is the parent of most spatial artifacts: it flattens neighboring blocks to different tones (blocking), truncates the high frequencies a sharp edge needs (ringing and mosquito noise), discards fine texture (blur), and rounds away the faint variation that held a smooth gradient together (banding). Even content that looked clean at the source can gain banding in the encode because quantization threw away the subtle variation. The coarseness of the step is set by the quantization parameter; quantization is lossy and irreversible, which is precisely why it saves bits.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
Quantization is where every lossy codec throws information away, so it explains most blocking, banding and blur you will ever debug. Of our seven handbook volumes on the video stack, Volume 1 treats quantization in depth: transforms and DCT, quantizers and step sizes, deblocking filters, and how H.264, HEVC and AV1 spend bits. Written by Nikolay Sapunov, CEO at Fora Soft.
Seeing blocking or banding and suspect quantization in your encodes?
We will review your encoder settings and sample frames and tell you which quantization choices cause the artifacts and how to fix them. 250+ video products shipped since 2005.
