A neural codec is a video codec where the compression and decompression are performed by neural networks rather than by hand-designed mathematical operations. Unlike ai-encoding — which uses machine learning to make a traditional codec (H.264, HEVC, AV1) work better — a neural codec replaces the entire codec with learned components: the network learns how to predict frames, how to transform residuals, how to quantize, how to entropy-code. The whole thing is trained end-to-end on huge video datasets to minimise reconstruction error at a target bitrate.
The promise is real and the demos are striking. At extreme low bitrates — where traditional codecs produce ugly blocking and smearing — neural codecs produce smooth, plausible-looking video. Microsoft's DiffuseCodec, several Google research projects (HiFiC for images, follow-ups for video), and academic systems all show neural codecs beating HEVC or AV1 on perceptual quality at the same bitrate, sometimes dramatically. The MPAI organisation is actively standardising an end-to-end video coding (EEV) framework specifically for this category.
For a product team in 2026, neural codecs are firmly in research territory, not deployable production technology. The fundamental blockers: neural codec decoding requires neural inference hardware (GPUs, NPUs) that consumer devices don't standardise on the way they standardise on traditional video silicon. Decoder speed is typically 10–100× slower than hardware AV1 or HEVC. Some neural codecs occasionally hallucinate plausible-but-wrong details (a problem traditional codecs simply don't have). Standards are not yet ratified. Realistic timeline: 2027–2030 for first commercial deployments in niche scenarios; potentially 2030+ for mainstream replacement of traditional codecs. Worth watching closely; not worth deploying yet.
The book · Volume 1 of 7
Video Encoding: A Complete Guide to Video Compression — From Pixels and Color to H.264, HEVC, AV1, Quality Metrics, and AI
Neural codecs post strong perceptual scores in papers, but decode cost and device support still keep them out of most delivery. Volume 1 discusses neural codecs in detail: end-to-end learned compression, how it differs from AI-assisted encoding, where AV2 fits, and what has to change before learned codecs reach production players. Written by Nikolay Sapunov, CEO at Fora Soft.
Wondering whether a neural codec could cut your bitrate in the next two years?
Tell us your content and target devices: we assess neural codec and AI-assisted encoding options against your current ladder and give a realistic timeline. 250+ video products shipped since 2005.