Video Codec Benchmark Methodology: BD-Rate, VMAF, Speed

Updated November 2026: VMAF v1 models (libvmaf 3.2.0), x265 4.2 and SVT-AV1 4.2.0 versions, the AV2 test conditions, and a new section on how we benchmark encoding time.

Why this matters

Short answer: a video codec benchmark can be trusted when it names the content, pins the encoder versions and settings, names the metric model file, shows how per-frame scores were pooled and how sure the result is, computes BD-rate only over the quality range both codecs reach, and times every encode on a stated machine and thread count. If a vendor's comparison cannot answer those six points, read its headline percentage as a claim, not a measurement.

Most codec benchmarks you find online cannot be trusted, and not because the authors are dishonest, but because the method is hidden. A vendor quotes "30% better" with no metric named, no encoder version, no content set, and no way to check it. This article is for the engineer, codec evaluator, or technical decision-maker who is about to read our codec and encoder comparisons and wants to know exactly how the numbers were made before believing them. It is also a working template: if you run your own benchmark, the rules here will keep you from fooling yourself. We publish the method first so the data that follows has a foundation to stand on.

Why the methodology comes before the results

A benchmark result is a claim about the world, and a claim is only as good as the evidence a reader can check. When a codec vendor says one encoder beats another, the useful question is never "by how much?" — it is "measured how, on what, with which settings?" Without those answers, the headline number is marketing, not measurement.

This section of Fora Soft Learn, our benchmarks, exists to publish numbers that other engineers can cite, and a number is only citable if it is reproducible. So we wrote the method down first. Everything in the codec and encoder comparisons that follow (codec comparison on real content, encoder comparison, how results change by content type, and BD-rate explained with our numbers) obeys the rules defined here.

The standards world solved this problem long ago with the idea of common test conditions (CTC) — a fixed, published recipe that says exactly which sequences, which settings, and which measurements every comparison must use, so two labs running the same test get comparable answers. The Alliance for Open Media publishes one for AV1 and another for AV2, the codec it released in June 2026; the Joint Video Experts Team publishes one for its codecs (AOM CTC; JVET-J1010, 2018). The AV2 conditions (CWG-F384o, version 8, October 2025) fix six quantizer points, score with PSNR, SSIM, MS-SSIM, CIEDE2000, VMAF (including the NEG model) and CAMBI, and make encoder runtime a mandatory part of every result. Our methodology adapts that discipline to a streaming-realistic setting. The principle is the same: write the recipe down, then follow it every time.

Figure 1. The benchmark pipeline. Every stage is pinned and disclosed: the source masters, the encoder versions and settings, the decode-and-upscale step, the metric and model, the pooling with its confidence interval, the BD-rate computation over the overlapping quality range, and the provenance record that lets a reader reproduce the whole run.

The content set: real footage, not just the usual test clips

Quality results depend on the content, so the first decision in any benchmark is what to measure on — and it is the decision most often made badly. A codec that wins on a clean, slow-moving test clip can lose on grain, fast motion, or screen text. Measure on the wrong content and the headline number is true only for that content.

The common shortcut is to use the standard academic test sequences — short, pristine clips that everyone has, which makes results comparable across papers. We use a set of those reference clips for exactly that reason: they let anyone line our numbers up against published work. But standard clips alone do not represent what a real product encodes. So we also measure on real client content — the kind of footage our streaming, conferencing, e-learning, OTT, and surveillance projects actually handle — used with permission and reported only as aggregate numbers, never as identifiable frames.

That content set spans the categories that stress codecs in different ways: live action film and television, animation, high-motion sport, screen and presentation content, talking-head conferencing, and user-generated video. Each behaves differently under compression, which is the whole point of how results change by content type. A few rules keep the set honest:

• The source is a pristine master. Every clip is a high-quality mezzanine or original, not a re-encode. Benchmarking against an already-compressed source measures the wrong thing — you would reward an encoder for faithfully copying someone else's artifacts.
• The color format and resolution are fixed and stated. We report the chroma sampling (4:2:0 unless stated), the bit depth, the resolution, and the frame count for every run, because a metric comparison is only valid when these match.
• The clips are long enough to be representative. Very short clips exaggerate start-up and scene-cut effects; we use durations that average over enough shots to be stable.
• The set is documented. Each result names the content category and the number of clips behind it, so a reader knows whether a number rests on three clips or thirty.

The encoders and settings: pinned, disclosed, and identical across the comparison

An encoder is not one thing — it is a specific build with specific settings, and changing either changes the result. So we pin both and publish both. The exact version of every encoder is recorded: the H.264 encoder x264, the HEVC encoder x265 (4.2, released April 2026), the AV1 encoder SVT-AV1 (4.2.0, released July 2026), and any others a given comparison includes. When these tools release new versions, the numbers can move, which is why the version is part of the result, not a footnote.

Settings matter as much as the build. The single most common way a codec comparison goes wrong is a mismatch in encoder effort — comparing a slow, high-effort preset of one encoder against a fast preset of another, then reporting the quality difference as if the codec caused it. We hold the comparison apples-to-apples: the same rate-control mode, comparable speed presets chosen for the use case, the same group-of-pictures structure where the codecs allow it, and the same handling of resolution. The full command line for every encode is recorded in the provenance manifest, so there is no hidden flag.

Tuning is a setting too. SVT-AV1 4.2.0 added a VMAF tune mode for random-access (video on demand) encoding that its changelog says aims at about a 15% VMAF BD-rate gain with minimal PSNR loss (SVT-AV1 changelog, July 2026). An encode tuned for VMAF and then judged by VMAF is graded on its own exam, so we record every tune flag, report PSNR beside VMAF, and add the NEG model whenever any encoder in the comparison is tuned for the metric.

We benchmark at a ladder of operating points, not a single bitrate. Each clip is encoded at several quality or rate settings — typically a spread of constant-rate-factor or fixed-quantizer points — so we can draw a rate-quality curve rather than compare two isolated dots. Standard CTCs use four such points (quantizer values 22, 27, 32, 37 in the JVET conditions) or six (the AOM conditions add lower and higher points); we use enough points to trace the curve across the quality range a streaming service actually ships, and we say how many. The reason for a curve rather than a point is the subject of the next two sections — it is what makes a fair codec comparison possible at all.

Note for codec internals. This article is about measuring codecs, not about how they compress. For how H.264, HEVC, and AV1 actually work, and how the encoder implementations differ, see the Video Encoding section's codec comparison and encoder implementations articles. We link to the cause side and stay on the measurement side.

The metrics and models: named, pinned, and honest about blind spots

A quality score means nothing until you name the metric, the model, and the conditions. "VMAF 95" is not a measurement; "VMAF 95 on the default v0.6.1 model, 1080p, mean-pooled" is. So every score we publish carries that full specification.

Our primary perceptual metric is VMAF (Video Multimethod Assessment Fusion), Netflix's metric that fuses several quality features into a 0–100 score trained to predict human opinion. VMAF's first generation (v0) ships several models, and the model is part of the measurement: the default v0.6.1 model predicts quality on a 1080p living-room television, the 4K model predicts it on a 4K television viewed at 1.5 times the screen height, and the phone model predicts it on a handset, where the same encode scores higher because impairments are harder to see on a small screen (Netflix VMAF documentation; models.md). We state which model every number uses, because reporting a phone-model score as if it were a television score overstates quality. The deeper treatment of model selection lives in VMAF in depth; here the rule is simply that we always name the model and version.

The model now has a generation as well as a name. In June 2026 Netflix released the VMAF v1 models (shipped with libvmaf 3.2.0 on 20 June 2026). The file suffix gives the viewing distance in screen heights: vmaf_v1.0.16_3d0h for 1080p at 3H, _5d0h for a phone at 5H, and _1d5h_2160 and _3d0h_2160 for 4K at 1.5H and 3H, the last scoring on a 0–110 range instead of 0–100; each has a high-frame-rate twin (Netflix VMAF documentation, models_v1.md). The v1 documentation also recommends computing SDR scores at 10-bit precision, even for 8-bit encodes, so the model catches banding. That adds two rules to a benchmark: the result names the model file, not just "VMAF", and v0 and v1 scores never share a chart, nor do scores from models with different ranges. A result scored with v0.6.1 is re-scored under v1, not compared across generations.

We do not report VMAF alone. Alongside it we compute PSNR (Peak Signal-to-Noise Ratio, a measure of pixel error in decibels) and SSIM (Structural Similarity, a 0–1 measure of structural fidelity), because each metric has a different blind spot and seeing them together catches errors a single number hides. Every objective metric is a proxy validated against human scores, and when a metric and a careful human viewing disagree, the viewing wins — that is the whole basis of validating metrics against human scores and of subjective testing as the ground truth. We name where each metric lies for the content under test, because every metric lies somewhere.

One measurement rule is easy to get wrong and quietly invalidates a benchmark: a full-reference metric needs both frames at the same size. When we encode a clip at a lower resolution to test a ladder rung, we decode it and upscale it back to the source resolution before scoring, using a stated upscaler held constant across the whole comparison. Scoring a 540p encode against a 1080p source without upscaling does not measure quality; it measures a size mismatch. The same discipline underlies the convex hull that ties resolution to bitrate.

Table 1. The apples-to-apples controls. The left column is what we hold fixed across a comparison; the right column is the error you get if you do not. A benchmark is only valid when every row matches between the encoders being compared.

Figure 2. The same controls as a visual reference — what we pin, what we record, and where each one lies if it is skipped.

Pooling and confidence: one number, plus how sure we are of it

A metric produces a score for every frame, and the summary number you report depends entirely on how you combine them — a step called pooling that is easy to do in a way that flatters the result. The arithmetic mean is the default, and it is also the most forgiving: it lets a few excellent seconds paper over a stretch of bad ones. Because a viewer remembers the worst moment, not the average, we report a low percentile or the harmonic mean alongside the mean, so the worst seconds are visible. The full treatment of why this matters is pooling per-frame scores into one number; the rule here is that we always state the pooling method and never report the mean alone.

We also report how sure we are of a VMAF number, because a score trained on human opinion carries uncertainty. VMAF can attach a 95% confidence interval to each prediction, computed by bootstrapping — training many models on resampled training data and measuring how much their predictions vary (Netflix VMAF documentation, conf_interval.md, since v1.3.7, 2018). With a bootstrap model, the 95% interval is roughly the score plus or minus 1.96 times the standard deviation of those predictions. The practical consequence is a discipline we apply without exception: if two encoders' intervals overlap, we do not declare a winner. When the model in use has no bootstrap variant, we report the clip-to-clip spread of the result instead and apply the same rule to it. A 0.4-VMAF difference inside a ±1.5-VMAF interval is noise, and calling it a win is the most common way a benchmark misleads.

BD-rate: the right way to say "X% smaller at the same quality"

The headline number in any codec comparison is the bitrate saving at equal quality, and the standard way to express it is BD-rate (Bjøntegaard Delta rate) — the average percentage difference in bitrate between two codecs at matched quality, across a range of quality (Bjøntegaard, VCEG-M33, 2001). The sign convention matters: a BD-rate of −40% means the test codec needs 40% less bitrate than the anchor codec to reach the same quality. It is a saving, not a quality score — a point worth repeating because the two are constantly confused.

The idea is geometric. Plot each codec's rate-quality curve — bitrate on a logarithmic axis, quality on the vertical axis — and BD-rate is the average horizontal gap between the two curves over the quality range where they overlap. "Horizontal gap" is the key phrase: you read the bitrate difference at a fixed quality, not the quality difference at a fixed bitrate.

Figure 3. BD-rate as the average horizontal gap. The anchor and test curves are compared only over the quality range both reach (the shaded band); at each quality level the horizontal distance is the bitrate saving, and BD-rate is the average of those savings in the log-bitrate domain.

A worked example

Take one clip, encoded at four operating points with each codec, scored with VMAF on the default model. The numbers below are illustrative, chosen to make the arithmetic legible — the real measured values appear in the codec comparison, not here.

Table 2. An illustrative rate-quality comparison. At each matched VMAF level the test codec uses about half the bitrate of the anchor. Numbers are synthetic, for demonstration only.

At each matched quality level the test codec uses half the bitrate, so the bitrate ratio is 0.5. BD-rate averages that ratio in the logarithmic domain, then converts back:

log10(0.5)            = -0.301        (the log-domain gap, constant here)
average over overlap  = -0.301        (same at every quality level)
BD-rate = 10^(-0.301) - 1
        = 0.5 - 1
        = -0.50  →  -50%

A BD-rate of −50%: the test codec reaches the same quality at half the bitrate, across the measured range. Real curves are never this clean — the saving varies with quality, the curves are not parallel, and they rarely overlap across the full range — which is exactly why the computation has to be done carefully rather than eyeballed.

Doing the computation right

The original 2001 method fit a single cubic curve through the points. Later work, including the metric author's own follow-up and recent academic analysis, showed that a plain cubic can overshoot between points and distort the result, and that two fixes matter (Bjøntegaard, VCEG-AL22, 2008; Herglotz et al., "The Bjøntegaard Bible," IEEE TIP, 2024). We apply both:

• Use a shape-preserving interpolation, not a plain cubic. A piecewise cubic that cannot overshoot its data points (PCHIP, or the similar Akima method) avoids the spurious wiggles a single cubic introduces. The Bjøntegaard Bible analysis found a plain cubic spline "can lead to overshoots (Runge's phenomenon)... hence CSI should not be used in practice," and recommends the monotone alternatives. We report which interpolation we used.
• Integrate in the log-bitrate domain. Averaging the bitrate difference in the logarithm keeps the result from being dominated by the high-bitrate end, where a few hundred kbps swamps everything below it.

Two more rules guard against the errors that quietly break BD-rate:

• Only compare over the overlapping quality range. BD-rate is defined on the quality interval both codecs actually reach; extending it by extrapolation produces large errors. The Bible reports a case where poor overlap turned a true −43% into −49% — a six-point error from a single non-overlapping point. We check the overlap explicitly and report it.
• A small BD-rate can be inside the noise. When the measured saving is smaller than the method's own error for that content and metric, we say so rather than report a decisive-sounding number. For saturating metrics like VMAF and SSIM, we also apply the log transform the research recommends, because those metrics flatten out at the top of their range and distort the curve fit otherwise.

The downloadable toolkit below computes BD-rate this way — shape-preserving interpolation, log-domain integration, an explicit overlap check, and the VMAF log transform — and reproduces the worked example above exactly.

Encoding time: how we benchmark speed, not just quality

To benchmark HD video encoding time fairly, time every encode on one named machine with the thread count fixed and stated, and report speed as a ratio to an anchor encoder rather than as a bare minutes figure. That is how the AV2 common test conditions do it: runtime is mandatory, measured as CPU user time with the Linux /usr/bin/time utility, and summarized as a geometric-mean ratio against the anchor (AOM CWG-F384o, version 8, 2025).

We record two clocks because they answer two questions. CPU user time is the total compute an encode costs, which is what a cloud bill scales with. Wall-clock time and encoded frames per second say whether the encoder keeps up: a 1080p30 live channel needs at least 30 encoded frames per second, sustained, on the hardware it will actually run on. Thread count changes both, so it is stated for every run. The AV2 conditions run most classes single-threaded for exactly that reason, and they ask for turbo boost off or for anchor and test to run in parallel on the same instance type, so that clock drift does not pass for an encoder gap.

A speed number goes stale as fast as a quality number. x265 4.2 added an experimental threaded motion estimation mode that its release notes say can make 1080p encoding up to 1.5 times faster on multicore machines, plus Arm NEON and SVE optimizations worth about 8% over 4.1 (x265 4.2 release announcement, April 2026). Quote a preset's speed without the version and flags, and you are quoting a different encoder. When speed decides a live design, we measure it on the target hardware before the live streaming pipeline our video and audio streaming development team builds is sized around it. Speed sits next to density and energy as a cost; the trade-offs between the three are in comparing encoders on bitrate and speed at the same VMAF.

Table 3. The speed controls. Encoding time is pinned the same way quality is: one clock definition, one machine, one thread count, reported relative to an anchor.

A common mistake: the benchmark that proves whatever you want

The classic bad benchmark is not faked; it is simply unfair in a way the reader cannot see. Someone compares the slow, high-effort preset of their preferred encoder against a fast preset of the competitor, scores with whichever metric flatters their codec, reports a single mean that hides the content where they lose, picks bitrates where the curves barely overlap, and quotes one number with no version and no confidence interval. Every individual choice is defensible in isolation; together they manufacture a conclusion.

The defense is the discipline in this article, and it is worth stating as a checklist a reader can apply to anyone's benchmark, including ours: name the content, pin and disclose the encoder versions and settings, hold the effort level comparable, name the metric and model, pool with the worst frames visible, compute BD-rate over the real overlap with a shape-preserving fit, and refuse to call a winner inside the confidence interval. A benchmark that cannot answer these questions is an opinion wearing a number's clothes. Reading any quality report this way is the skill taught in reading a quality-metric report without fooling yourself.

Provenance: every result carries its own receipt

The single thing that separates a citable benchmark from a forgettable one is provenance — the record that lets someone else reproduce the number. So every result we publish ships with a manifest that states the content set and clip count, the exact encoder versions and full command lines, the metric, model, and version, the pooling method and confidence interval, the resolution handling, the hardware, and the date. The date matters because encoders and metrics both evolve; a benchmark is a snapshot, valid for the versions and settings it names, and re-validated when those change.

Figure 4. The provenance record. A benchmark number is citable only when a reader can reconstruct it; this manifest is the receipt attached to every result in the block, and the downloadable toolkit generates and checks it.

Where Fora Soft fits in

Fora Soft has built video streaming, OTT, conferencing, e-learning, telemedicine, and surveillance software since 2005, and every one of those products lives or dies on the encoding and delivery choices behind it. We measure delivered quality the same way we ask you to trust it: on real content, with disclosed encoders and settings, against named metrics and models, with the worst frames and the confidence interval in view. The benchmarks in this block are not a literature review — they are our own measurements on the kind of footage our clients actually ship, which is why they are worth citing and why the method is published in full. When a quality target has to be defended to a CDN budget or a product owner, this is the discipline that makes the number hold up. The same measurements drive encoder and ladder choices in the products our video encoding and processing development team builds, and our OTT platform development work uses them to balance quality against CDN cost.

The handbook

Fora Soft Video Engineering Handbook

We wrote a seven-volume handbook that walks the whole video stack, from a single pixel to a running production system, across more than 9,000 pages; an engineer who works through it can design and build the kind of system these benchmarks measure. Volume 1, Video Encoding, is where this method comes from - how H.264, HEVC and AV1 spend their bits, what a rate-quality curve is made of, and how PSNR, SSIM and VMAF each behave on the content you are testing. Volume 3, Video Streaming, takes the same curves on to the encoding ladder, ABR and the bandwidth bill a BD-rate saving actually pays, and Volume 6 covers the learned models that newer no-reference metrics are built on. Written by Nikolay Sapunov, CEO at Fora Soft.

Browse the handbook on Amazon →7 volumes · Kindle & paperback

See our case studies

References

1. G. Bjøntegaard. "Calculation of Average PSNR Differences Between RD-Curves." ITU-T SG16 Q.6 VCEG, document VCEG-M33, Austin, TX, USA, April 2001. Tier 1 (metric author's defining document). The original BD-rate / BD-PSNR method: fit a curve through rate-quality points and report the average bitrate difference at equal quality. Basis for the BD-rate definition and sign convention. https://www.itu.int/wftp3/av-arch/video-site/0104_Aus/VCEG-M33.doc
2. G. Bjøntegaard. "Improvements of the BD-PSNR Model." ITU-T SG16 Q.6 VCEG, document VCEG-AL22, Berlin, Germany, July 2008. Tier 1 (metric author follow-up). Refines the original method toward piecewise interpolation with logarithmic bitrate for more stable results. Basis for the log-domain and shape-preserving-fit rules. https://www.itu.int/wftp3/av-arch/video-site/0807_Ber/VCEG-AL22.doc
3. C. Herglotz, H. Och, A. Meyer, et al. "The Bjøntegaard Bible — Why Your Way of Comparing Video Codecs May Be Wrong." IEEE Transactions on Image Processing, vol. 33, pp. 987–1001, 2024, doi:10.1109/TIP.2023.3346695; arXiv:2304.12852. Tier 5 (peer-reviewed). Analyzes BD computation: cubic splines overshoot and should be avoided (use PCHIP/Akima), saturating metrics (VMAF, SSIM) need a log transform, the quality ranges must overlap (check intersection-over-union), and a few supporting points suffice for rough estimates. Basis for the "doing the computation right" section and the overlap-error figure (−43% vs −49%). https://arxiv.org/pdf/2304.12852
4. Netflix. "VMAF v1 Models" (models_v1.md), VMAF Development Kit documentation, and libvmaf release v3.2.0 (20 June 2026). Accessed 2026-11-03. Tier 1 (metric author documentation). The vmaf_v1.0.16 model files for 1080p at 3H, phone at 5H, 4K at 1.5H and 4K at 3H (0–110 range), their high-frame-rate variants, and the advice to compute SDR scores at 10-bit precision so banding is captured. Basis for the model-generation rule. https://github.com/Netflix/vmaf/blob/master/resource/doc/models_v1.md
5. x265 project. "[ANN] New Release v4.2," x265-devel mailing list, 19 April 2026. Tier 2 (vendor release notes). Experimental threaded motion estimation (--threaded-me), up to 1.5x faster 1080p encoding on multicore machines; Arm NEON/SVE optimizations about 8% faster than v4.1. Basis for the encoder-version and speed-staleness rules. https://mailman.videolan.org/pipermail/x265-devel/2026-April/014533.html
6. SVT-AV1 project. "SVT-AV1 v4.2.0" changelog, 14 July 2026. Tier 2 (vendor release notes). Adds a VMAF tune mode for random access aimed at about a 15% VMAF BD-rate gain with minimal PSNR loss, plus RTC rate-control and API changes. Basis for the encoder-version and tune-disclosure rules. https://gitlab.com/AOMediaCodec/SVT-AV1/-/releases/v4.2.0
7. Netflix. "VMAF — Confidence Interval" (conf_interval.md), VMAF Development Kit documentation. Accessed 2026-06-25. Tier 1 (metric author documentation). Since v1.3.7 (June 2018) each VMAF prediction can carry a 95% confidence interval computed by bootstrapping; with a bootstrap model the 95% CI is approximately the score ± 1.96 × the bootstrap standard deviation. Basis for the confidence-interval and overlapping-interval rules. https://github.com/Netflix/vmaf/blob/master/resource/doc/conf_interval.md
8. Netflix. "VMAF Models" (models.md) and VMAF FAQ, VMAF Development Kit documentation. Accessed 2026-06-25. Tier 1 (metric author documentation). The default v0.6.1 model targets a 1080p living-room TV; the 4K model targets a 4K TV at 1.5× screen height; the phone model targets handset viewing; the NEG model resists enhancement gaming. Basis for the model-selection rule. https://github.com/Netflix/vmaf/tree/master/model
9. Z. Li, C. Bampis, J. Novak, et al. "VMAF: The Journey Continues." Netflix Technology Blog, 2018. Tier 4 (credible deployer engineering blog). How VMAF fuses features trained on subjective scores, why a single metric needs uncertainty quantification, and how Netflix deploys it at scale. Basis for the VMAF framing. https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12
10. Alliance for Open Media. "AV2 Common Test Conditions," document CWG-F384o, version 8, October 2025. Accessed 2026-11-03. Tier 1 (standards-body methodology). The current AOM benchmark recipe, used to develop AV2: natural (A1–A5) and synthetic (B1 gaming and animation, B2 screen sharing) classes in 4:2:0, six random-access quantizer points (base_q_idx 110, 135, 160, 185, 210, 235), PSNR, SSIM, MS-SSIM, CIEDE2000, VMAF including NEG, and CAMBI, plus mandatory runtime measured as user time with /usr/bin/time and reported as geometric-mean ratios to the anchor. Its AV1-era predecessor used CRF/QP points {17,22,27,32,37,42}. Basis for the common-test-conditions discipline, the operating-point ladder and the encoding-time rules. https://aomedia.org/docs/CWG-F384o_AV2_CTC_v8.pdf
11. Alliance for Open Media. "Alliance for Open Media Releases AV2 Codec," press release, June 2026. Tier 1 (codec owner). Announces the AV2 release with the libavm reference software; proposed tools were tested under common test conditions. Basis for the AV2 release date. https://aomedia.org/press%20releases/Alliance-for-Open-Media-Releases-AV2-Codec/
12. JVET (ITU-T/ISO/IEC). "JVET Common Test Conditions and Software Reference Configurations," document JVET-J1010, 2018. Tier 1 (standards-body methodology). The CTC for VVC-era codec testing: test-sequence classes, random-access GOP-16 configuration, and the four QP points {22,27,32,37}. Basis for the operating-point and apples-to-apples rules. https://www.itu.int/wftp3/av-arch/jvet-site/2018_04_J_SanDiego/JVET-J1010.zip
13. Recommendation ITU-T P.1401 (07/2020). "Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models." International Telecommunication Union, 2020. Tier 1 (official standard). The procedure for evaluating an objective metric against subjective scores — Pearson correlation, Spearman rank correlation, RMSE, and the rule that a difference smaller than the uncertainty is not a difference. Basis for the metric-validation and noise-threshold rules. https://www.itu.int/rec/T-REC-P.1401
14. Recommendation ITU-R BT.500-15 (2023). "Methodologies for the subjective assessment of the quality of television pictures." International Telecommunication Union, 2023. Tier 1 (official standard). The subjective ground truth every objective metric is validated against; when a metric and a careful viewing disagree, the viewing wins. Basis for the measurement-honest framing. https://www.itu.int/rec/R-REC-BT.500
15. Recommendation ITU-T P.910 (2023). "Subjective video quality assessment methods for multimedia applications." International Telecommunication Union, 2023. Tier 1 (official standard). Defines the subjective test methods (ACR, DCR, PC) and viewing conditions used to validate objective metrics. Basis for the subjective-validation reference. https://www.itu.int/rec/T-REC-P.910
16. Y. Reznik, et al. "Revisiting Bjøntegaard Delta Bitrate (BD-BR) Computation for Codec Comparison." ACM Mile-High Video (MHV), 2022. Tier 5 (peer-reviewed). Reviews practical BD-BR computation choices — interpolation, integration domain, and supporting-point placement — for streaming codec comparison. Basis for the BD-rate implementation choices. https://www.reznik.org/papers/MHV22_BD_BR-CameraReady.pdf
17. FFmpeg. "libvmaf, psnr, and ssim filter documentation." FFmpeg Filters Documentation, accessed 2026-06-25. Tier 3 (first-party tooling). The filters that compute VMAF, PSNR, and SSIM in a reproducible command line, including model selection and the scaling that full-reference metrics require. Basis for the measurement-tooling and upscaling rules. https://ffmpeg.org/ffmpeg-filters.html