The Future: AV2, Neural Codecs, and End-to-End Learned Compression

Published 2026-05-17 · 26 min read · By Nikolay Sapunov, CEO at Fora Soft

Why this matters

Every codec generation reshapes who pays for bandwidth, who controls the toll booth on patent licensing, and which hardware vendor sells the next decade of chips. If you build, operate, or fund a service that ships pixels — streaming, video conferencing, surveillance, telemedicine, OTT, e-learning, AR/VR — the next codec cycle will move 25–40% of your delivery cost, and it will move it in a direction the past few cycles did not move. AV2 will be patent-friendly and free for AOMedia members. H.267 will be patent-encumbered and excellent. Neural codecs will be neither — they will ship as machine-learning model weights instead of a fixed bitstream specification, and the supply chain for them looks nothing like the supply chain for H.264.

This article is for product managers, founders, video operations leads, and engineers who need to read the next five years of the codec roadmap, understand the three competing tracks well enough to talk to a CFO and a chip vendor in the same meeting, and know where the genuinely surprising risks sit. We walk through each of the three tracks, the technical claims, the practical state of decoders and tooling in 2026, the licensing and hardware story, and a planning frame you can take back to your team.

Three tracks, one question

To understand the next five years of video codecs, it helps to draw three parallel tracks on a single road. Each track is a different answer to the same question — "given that we already have H.265 and AV1 and H.266, how do we compress video another 30% smaller without making the playback unwatchable on a phone?" — and each track has a different group of people working on it.

The first track is the Alliance for Open Media (AOMedia), the consortium of Google, Netflix, Amazon, Apple, Meta, Microsoft, Mozilla, Cisco, Tencent and others, that produced AV1 in 2018 and is now finishing AV2. AV2 keeps the same shape as every codec since H.264 — block-based prediction, transform, quantisation, entropy coding, loop filter — but pushes every box in that pipeline a little further. It is hand-engineered by humans, royalty-free for AOMedia members, and timed for late 2026. ^{1 2}

The second track is the Joint Video Experts Team (JVET) of ITU-T and ISO/IEC, the same standards body that produced H.264, H.265, and H.266. JVET is working on the successor to H.266/VVC, currently labelled the Enhanced Compression Model (ECM) and expected to ship as H.267 around 2029. ECM is the same kind of hand-engineered hybrid codec as AV2, but it is being developed inside a patent-encumbered standards process, and it targets a more ambitious 40% bitrate reduction over H.266. ^{11 12}

The third track is end-to-end learned video compression — neural networks trained to compress and decompress video directly. There is no fixed bitstream specification; the compressed file is whatever the trained model produces, and the decoder is the rest of the model. The Deep Contextual Video Coding (DCVC) family from Microsoft Research, the Deep Render commercial codec, JPEG AI, and the open MPEG NNVC exploration are all on this track. In 2026 the best learned codecs match or slightly beat H.266 on compression and run in real time on a single GPU. ^{17 18 19}

The three tracks are not mutually exclusive. AV2 ships first and will dominate the next four years of free-to-use streaming. H.267 will land at the high end of the patent-licensed ecosystem in 2029–2030. Learned codecs will keep climbing the quality curve in parallel and will, by the late 2020s, ship inside hybrid codecs as drop-in replacements for individual blocks (intra prediction, loop filter) before any pure learned codec replaces a generation outright.

The rest of this article is the operator's view of those three tracks: what each one buys you, what it costs, what the decoder story looks like, and how to plan your pipeline so you are not on the losing end of any of them.

Figure 1. Three parallel tracks define the future of video codecs through 2030. AV2 is the closest, royalty-free for AOMedia members, and the first to ship hardware decoders. H.267 follows the traditional ITU/ISO licensing model and lands at the back of the decade. End-to-end learned codecs run alongside both and will likely first ship as ML modules inside the hybrid codecs rather than as a full replacement.

Track one — AV2: the next royalty-free workhorse

AV2 is what most operators will switch their delivery pipeline to between 2027 and 2029, because it is the cheapest codec in licensing, the easiest to procure decoders for from the AOMedia membership, and the one with the closest match to current pipelines. It is also the smallest jump in technical risk from where most teams are today on AV1.

The Alliance for Open Media spent five years and more than 2,700 commits against its reference software building AV2. The bitstream draft was published on 3 February 2026, and the final 1.0 specification was confirmed for end of 2026 at the AOMedia tenth-anniversary announcement. ^{3 1 4} AV2 will follow the same patent strategy as AV1: AOMedia members commit not to assert patents against implementers; non-members may still attempt to form a patent pool (Sisvel has already indicated intent), but the marketing positioning and contract path remains "royalty-free for the open web". ⁵

What AV2 actually changes versus AV1

AV2 is a hand-engineered hybrid codec — the same block-prediction-transform-quantise-entropy-filter pipeline that every codec since H.264 has shared. Where it differs from AV1 is mostly in how much detail each box of that pipeline now contains. The high-level shifts that matter for an operator are:

• Larger and smarter blocks. The maximum coding unit grows from 128×128 in AV1 to 256×256 in AV2, with fully recursive partitioning into many more shapes. Large smooth regions (sky, walls, animated backgrounds) are coded with fewer overhead bits; fine detail (text, faces, sports motion) can drop down to small partitions when needed. ⁶
• Better intra prediction. AV2 adds Multiple Reference Line Selection (MRLS) — the encoder can pick from four candidate top and left reference lines for every directional intra prediction mode, instead of always copying from the immediately neighbouring row and column. Combined with new data-driven intra modes and improved chroma-from-luma modelling, intra-only frames shrink noticeably. ⁶
• Longer memory for motion. AV2 can reference up to seven past frames for motion-compensated inter prediction, versus AV1's smaller reference list. Slow scenes with periodic motion (sports loops, animated backgrounds, screen content) benefit most. ⁷
• A redesigned loop filter. A single generalised deblocking filter replaces the multi-stage filter chain in AV1, and two new filters — a guided detail filter and a cross-component sample offset — clean up compression noise while preserving more detail. ⁷

The headline coding gain is around 30% lower bitrate than AV1 at equivalent perceptual quality. Netflix's Andrey Norkin reported approximately 28.7% BD-rate improvement on YUV-PSNR and 32.6% on VMAF using AOMedia's Common Test Conditions corpus; independent 4K benchmarks see 18–25% additional savings on top of AV1, with the biggest gains on high-motion and screen content. ^{8 9 6} In headline terms, an HD stream that costs you 4 Mbps on H.264 and 2.7 Mbps on AV1 will cost roughly 1.9 Mbps on AV2 once encoders are tuned.

Decoder and hardware story

AV2 ships into a much more favourable decoder landscape than AV1 did. AOMedia member survey data shows 53% of members plan to deploy AV2 within 12 months of finalisation, and 88% within two years. ^{7 5} Hardware decode support is expected in consumer devices starting in late 2026 and 2027 — phone SoCs, smart TVs, set-top boxes, gaming consoles — because the same Netflix, YouTube, Amazon, Apple, Meta and Microsoft engineering teams that pushed AV1 hardware through the supply chain are already pushing AV2. ¹⁰

Software decoders will appear earlier. The AOMedia avm reference decoder runs today; libavif and dav1d-family forks for AV2 are in active development; FFmpeg integration will follow shortly after the bitstream freeze. For a streaming operator, this means software-decode AV2 on desktop browsers and high-end Android in late 2026, mass-market hardware decode through 2027 and 2028, and the long tail of older devices (TVs from 2022 and before, low-end Android) still on AV1 or H.264 for another four to five years.

The headline practical risk for AV2 is the same risk AV1 had: encoder maturity. The reference encoder, like libaom for AV1, is honest but slow. Production-quality AV2 encoders — the eventual equivalents of SVT-AV1 — will need 18–24 months after the bitstream freeze to reach the speed and tuning of today's best AV1 encoders. Until then, AV2 will be a premium-tier choice for VOD and a poor choice for live, mirroring the early-life trajectory of every codec generation in the last 25 years.

Figure 2. Each codec generation has delivered roughly a 30–50% bitrate reduction over its predecessor at the same perceptual quality. AV2's ~30% over AV1, and ECM/H.267's targeted 40% over H.266, both sit within this historical envelope.

Track two — H.267: the heir to VVC

While AV2 finishes, JVET is already 15 versions into the Enhanced Compression Model (ECM) that will become H.267. ECM v15 has demonstrated approximately 25% bitrate savings over H.266/VVC in random-access configurations, and up to 40% savings on screen content. The final H.267 standard is targeted by JVET for around 2028–2029, with broad deployment expected in 2034–2036 if the historical eight-to-ten-year deployment lag holds. ^{11 12}

H.267 will be a hand-engineered hybrid codec, like AV2 and every codec before it. The technical agenda of ECM is more ambitious than AV2's because JVET is willing to absorb more decoder complexity in exchange for compression: deeper block partition trees, more transform variants, more intra prediction modes, more sophisticated motion compensation, and — increasingly — neural-network blocks for specific stages (intra prediction, in-loop filtering, loop restoration). Some of these neural sub-blocks are being co-developed inside the parallel MPEG Neural Network-based Video Coding (NNVC) exploration. ¹⁴

The headline difference between H.267 and AV2 is licensing and timing. H.267 will be licensed by patent pools — Access Advance, MPEG LA, Avanci, Sisvel — under the same model as H.264, H.265, and H.266. That model is excellent technology, painful licensing: it took H.265 years longer than expected to reach broad deployment because pools disagreed on royalty terms, and H.266 today has minimal browser support five years after spec finalisation because the same fight is repeating. ¹³

For a streaming operator in 2026 the practical takeaway is:

• You will probably never adopt H.267 for open-web delivery. It will live in broadcast contribution, premium VOD with paid client apps, and broadcast distribution to capable smart TVs.
• You will adopt H.265 and H.266 only where AV1 / AV2 cannot reach. That mostly means iOS / Apple-ecosystem playback (because Apple ships HEVC hardware decode universally) and the existing premium VOD pipeline.
• You will watch H.267 for screen-content workloads — remote desktop, cloud gaming, screen-share in video conferencing — because that is where the 40% gain over H.266 is largest.

The thing not to do is to schedule a pipeline migration around H.267. Its timeline is too far out, its licensing too uncertain, and AV2 will be doing 90% of the same job five years earlier, on the same hardware, for free.

Track three — End-to-end learned compression

This is the track that will surprise the most teams in the late 2020s. End-to-end learned video compression replaces the entire hand-engineered hybrid codec pipeline with a single neural network. The network is trained on millions of hours of video to produce a compact internal representation (called a latent, the compressed-bits side of the network) and reconstruct the original frames from it. The compressed file is whatever the network produces; the decoder is the rest of the network.

The earliest end-to-end learned image codec that beat JPEG2000 was published by Ballé et al. in 2017. By 2018–2020 the Microsoft Research DCVC family extended the approach from images to video. By 2024 the DCVC-FM (feature modulation) variant reached the quality of the H.266 reference encoder. In February 2025 DCVC-RT (real-time) was published at CVPR 2025, and it is the milestone that matters most for any practical conversation about neural codecs in 2026. ^{15 16 17}

What DCVC-RT actually delivers

DCVC-RT is the first end-to-end learned video codec that is both competitive with H.266 on compression and fast enough to run in real time on a single GPU. The specific numbers from the CVPR 2025 paper:

• Average encoding speed: 125.2 fps for 1080p video on an NVIDIA A100.
• Average decoding speed: 112.8 fps for 1080p video on an NVIDIA A100.
• BD-rate of −21.0% versus the H.266/VTM reference encoder — that is, DCVC-RT needs 21% fewer bits than H.266 to reach the same perceptual quality. ^{15 18}

A higher-capacity variant, DCVC-RT-Large, pushes BD-rate to −30.8% while staying near real-time. The same paper reports approximately 40 fps encoding and 34 fps decoding for 1080p on a consumer NVIDIA RTX 2080 Ti — three generations old at the time of publication. On the latest consumer GPUs, real-time HD playback is comfortable. ¹⁸

In other words: the gap between "interesting research demo" and "you could ship this in a desktop application" closed in 2025 for high-end consumers, and it is closing for mid-range mobile every quarter.

Figure 3. The compression-versus-speed map of leading video encoders in 2026. Traditional hybrid encoders (x265, SVT-AV1, libaom, vvenc) cover the wide left-to-right speed-quality trade-off. The DCVC family of learned encoders has reached the same Pareto frontier as H.266 and is climbing fast.

Why learned codecs are not a drop-in replacement

The headline numbers above are encouraging, but four properties of learned codecs make them genuinely different from traditional codecs, and these properties define the practical timeline for adoption.

The compressed bitstream is not a fixed specification. A learned codec is a trained model. Two different versions of "DCVC" are two different codecs. There is no equivalent of "the H.264 spec" against which any decoder can validate; there is the specific model weights that produced the file. This is the single biggest cultural change. The industry has built thirty years of interoperability on top of fixed bitstream specifications. Learned codecs throw that frame away. Standards bodies (JPEG, MPEG NNVC) are now writing specifications for how to ship model weights alongside the bitstream so that interoperability becomes possible at all. ²⁰

Decoding is compute-heavy and floating-point. Traditional codecs decode in integer arithmetic on small, well-understood DSP blocks. Neural codecs decode by running a deep network forward, which historically needed floating-point arithmetic on a GPU. Quantising the model to integer precision so it can run on mobile NPUs is an active research area; floating-point cross-platform reproducibility is famously hard. ^{21 22}

Mobile and embedded support is still early. As of 2026, the only published neural video decoders capable of 1080p in real time on a mobile NPU are research prototypes such as Qualcomm's MobileNVC (decodes 1080p YUV420 in real time on a mobile NPU) and MobileCodec. Production deployment is at least one chip generation away. ^{23 24}

The compressed-file size is not the only metric anymore. Some learned codecs (perceptual / generative variants such as HiFiC for images, and emerging generative video codecs) deliberately synthesise reconstructed content that looks like the original but is not pixel-accurate. This is a useful trade-off for some applications (entertainment streaming) and unacceptable for others (medical imaging, surveillance forensics, broadcast contribution). Procurement of a learned codec has to ask "what kind of reconstruction does it produce" alongside "what BD-rate does it achieve".

Where learned codecs will land first

The way neural codecs will actually enter the streaming stack is not as a full replacement for AV1 or H.265 in 2027. It is in three more practical patterns:

1. Neural blocks inside hybrid codecs. ECM/H.267 already integrates neural-network in-loop filters and intra prediction modules. AV2 has data-driven intra modes. By H.268 / AV3, almost every box in the hybrid pipeline will have a neural alternative the encoder can choose. The bitstream stays interoperable; the gains accrue gradually.
2. Pre- and post-processing wrappers. Tools like Netflix's content-aware encoding (the system that powers their per-title and per-shot ladders), Bitmovin's neural ABR, Deep Render's neural pre-encoder, and the SimaBit pre-processing pipeline use neural networks to reshape pixels before a traditional codec runs, recovering 20–30% bits without changing the downstream decoder. These ship today and will be the dominant production pattern through 2027–2028. ^{9 26}
3. Niche end-to-end deployment. Closed-ecosystem applications where one team controls both encoder and decoder — cloud gaming with a custom client, video conferencing with a managed app, machine-to-machine video for ADAS and surveillance — will be the first to ship pure end-to-end learned codecs. Open-web streaming will be the last.

The realistic projection for the open-web: pure end-to-end learned codecs ship in production browsers some time around 2030–2032, and they will arrive labelled as a standard (a future JPEG XS Video, an MPEG-AI, or a successor JVET project) with shipped model weights rather than as a per-vendor proprietary format.

Figure 4. A planning decision tree for codec roadmaps through 2030. The recommendation in 2026 is to plan an AV2 transition, evaluate neural pre-processing on top of your existing pipeline, and treat H.267 and pure end-to-end neural codecs as horizon items rather than near-term commitments.

Comparative landscape — the four codecs that will matter through 2030

If you are reading this in 2026 and trying to plan capital allocation for the next four years of video infrastructure, four codecs sit on the table. We summarise the situation as you should walk into a budget meeting with it.

The lines that read "available" describe when the bitstream is locked, not when broad deployment is realistic. Add three years to "available" for "broad consumer hardware decoder coverage", and another two for "you can drop your AV1 fallback".

Common mistake — extrapolating from BD-rate alone

Every benchmark on the table above reports a single BD-rate number versus a reference codec. The most common procurement mistake we see is to take those numbers literally and assume that the codec with the better BD-rate is the better business choice.

It is not. Codec decisions are joint decisions about compression, decoder availability, licensing cost, encoder cost, ecosystem maturity, and time-to-market. A 40% BD-rate gain on a codec whose decoders exist in 2030 is worth less than a 30% gain on a codec whose decoders exist in 2026, because the bandwidth savings are integrated over the four years of difference. Walk into every benchmark conversation with the integrated cost in mind, and let the BD-rate number be one input among many.

A worked example — the bitrate ladder for a 1080p stream in 2030

Concrete numbers anchor the discussion. Take a 30-minute episode of a typical drama, encoded at 1080p / 24 fps for OTT delivery. Assume a top-rung target of VMAF 95 (broadcast quality) and a baseline of 4.0 Mbps on H.264 today. We can project the bitrate at the same quality through each generation:

H.264         baseline                 = 4.0 Mbps
H.265 / HEVC  −50% vs H.264            = 2.0 Mbps
AV1           −30% vs H.265 (~−65% vs H.264) = 1.4 Mbps
AV2           −30% vs AV1               = 1.0 Mbps
H.266 / VVC   −40% vs H.265 (~−70% vs H.264) = 1.2 Mbps
H.267 / ECM   −40% vs H.266            = 0.72 Mbps
DCVC-RT       −21% vs H.266            = 0.95 Mbps
DCVC-RT-Large −30.8% vs H.266          = 0.83 Mbps

For a service with 10 million MAU streaming 5 hours per month at 1080p, the per-user monthly egress at H.264 is 5 × 60 × 60 × 4 Mbps ÷ 8 ≈ 9 GB. At AV2 that drops to 9 × (1.0 / 4.0) = 2.25 GB. Across 10 million users that is 67.5 PB of monthly egress eliminated compared to a stuck-on-H.264 pipeline. At a typical CDN egress price of 0.5–1 cent per GB at scale, the AV2 transition is worth several million dollars per year for a service of that size. The further codec hops (DCVC-RT, H.267) keep adding margin, but the first hop — from a legacy H.264 / H.265 stack to AV1/AV2 — is the one with the dominant payoff. ²⁵

This is the headline reason AV2 is the priority codec for any operator in 2026, even though H.267 and end-to-end learned codecs are technically more interesting on paper. The bandwidth bill is paid in money today, not in slides about 2030.

Where Fora Soft fits in

We have built video pipelines for OTT, video conferencing, video surveillance, telemedicine, e-learning, and AR/VR customers since 2005, across 239+ shipped projects. The pattern we see most often in customer planning conversations is the inverse of the pattern in the trade press: customers are over-rotated on the codec that will arrive in 2030 and under-rotated on the codec that will arrive in 2026. The practical 2026 task for almost every operator is to run a same-VMAF AV1-versus-AV2 evaluation against their own catalogue, build a hardware decode roadmap by device family, and design a fallback ladder that ships AV2 first to capable devices and AV1 to the rest. We are happy to run that evaluation, design the ladder, or audit one a customer's team has built — none of those engagements need to start with a codec licence purchase.

What to read next

• Codec comparison matrix: MPEG-2, H.264, H.265, VP9, AV1, VVC
• AV1: the new internet standard and where it stands in 2026
• AI inside the encoder: ML-based mode decision, partitioning, scene-cut

Talk to us / See our work / Download

• Talk to a video engineer — book a 30-minute scoping call to plan your AV2 migration or run a same-VMAF benchmark against your catalogue.
• See our case studies — read about our OTT, video conferencing, e-learning, telemedicine and surveillance pipelines at forasoft.com/projects.
• Download the Future Codecs Roadmap Checklist — the one-page version of this article, designed to be taken into a budget or roadmap meeting through 2030.

References