Video Anomaly Detection in 2026: Models, Cost & Compliance

Why Fora Soft wrote this playbook

Fora Soft is a custom software development company founded in 2005, specializing in video streaming and computer vision. We’ve shipped 250+ projects with 50 in-house engineers, and our surveillance practice ships to real customers, not demos. The most-cited example is VALT, a video-surveillance platform we’ve been the sole development team for across 10+ years; it runs in 770+ US organizations and serves 50,000+ users under HIPAA. We’ve also built construction-site monitoring (EyeBuild, solar sites) where the anomaly that matters is a person in a fall zone, not a bounding box around a hard hat.

So this isn’t a survey rewritten from other surveys. It’s the guide we wish existed when a CTO asks us “can you add anomaly detection to our cameras, and what will it actually cost to run?” We’ll name the model families, show the benchmark traps, draw the production pipeline, price the hardware, and walk the 2026 compliance map — including the parts most articles still get wrong. Where we haven’t measured something ourselves, we’ll say so and point at the primary source.

Our computer-vision lead, Denis, runs the surveillance and CV vertical, and the opinions here are his team’s, argued from projects we’ve shipped. If you want the short version for a specific site, our video surveillance engineering team will give you a straight answer on a call.

What is video anomaly detection?

Video anomaly detection (VAD) is the task of automatically flagging moments in video that deviate from learned normal patterns. Instead of matching a fixed catalogue of objects, a VAD model learns what “ordinary” looks like for a scene — the usual motion, paths, speeds and interactions — and scores each frame or clip by how far it strays. High score, likely anomaly. That framing matters, because the interesting events in surveillance (a fall, a fight, a wrong-way vehicle, an abandoned bag) are exactly the ones nobody labelled in advance.

Contrast that with plain object detection. A detector answers “what is in the frame and where”: person, car, backpack. Anomaly detection answers a harder question: “is this pattern normal for this place, right now?” A backpack on a bench is a backpack; a backpack left alone for ten minutes while its owner walks away is an anomaly. You need the object layer to see the bag, and the anomaly layer to judge the behaviour. Most real systems run both, which is why we treat computer vision for video surveillance and anomaly detection as one stack, not two projects.

The reason anomaly detection in video surveillance is treated as its own field is the label problem. You can’t enumerate every bad thing that might happen on a loading dock, and you certainly can’t collect thousands of clean examples of each. So the whole discipline is organised around one question: how little supervision can you get away with? That question defines the four regimes below.

The four supervision regimes, and which one you actually have

Every VAD approach sits on a spectrum of how much labelling it needs. The literature sorts VAD into three classic regimes — fully-supervised, weakly-supervised and one-class/unsupervised — with zero-shot foundation-model methods the newer fourth lane. Picking the right one is less about which paper tops a leaderboard and more about which labels exist in your world.

Figure 1. The four supervision regimes, ordered by how much labelling they demand. You rarely get to choose the fanciest one; you choose the one your data supports.

Fully-supervised needs frame-level anomaly labels — someone marks the exact frames where the bad thing happens. It’s accurate and almost nobody uses it, because labelling anomalies frame by frame across thousands of hours is prohibitively expensive and the anomalies are rare to begin with.

Weakly-supervised asks only for video-level labels: this clip contains an anomaly somewhere, that clip is clean. That’s cheap to collect and it’s where most of the strong 2021–2024 methods live. It’s the pragmatic default for teams that can tag a few hundred incident clips.

One-class (self-supervised) trains on normal video only and flags whatever doesn’t fit. You need zero anomaly examples, which is perfect for a brand-new site where nothing bad has happened yet — but it’s also the noisiest, because “unusual” and “dangerous” aren’t the same thing.

Zero-shot (foundation models) uses pre-trained vision-language or large-language models to judge anomalies with no task training at all. It’s the newest lane and the answer to a cold start with unknown anomaly classes. We’ll come back to it once the classics are on the table.

Weakly-supervised VAD: the pragmatic default

Weakly-supervised VAD is usually framed as Multiple-Instance Learning (MIL). A video is a “bag” of short clip “instances,” and you only tell the model whether the bag as a whole is normal or abnormal. The model learns to push up the anomaly score of the most suspicious clip in an abnormal bag while keeping normal bags quiet. This is the paradigm that Sultani et al. introduced with the UCF-Crime dataset (2018), and it still anchors the field.

On UCF-Crime’s frame-level AUC, the well-known milestones read like a short history: RTFM around 84.0% (ICCV 2021), MGFN and UR-DMU both near 87.0% (AAAI 2023), and VadCLIP at 88.02% (AAAI 2024), which also reports 84.51% AP on XD-Violence. A cluster of 2025 methods report pushing past 90% AUC; treat those as promising rather than settled until you’ve read each results table. The trend is clear enough: video-level labels plus a good temporal backbone get you most of the way.

Here’s the catch we tell clients up front: those numbers are on curated academic clips. Your loading dock isn’t UCF-Crime. Weakly-supervised models transfer well when your anomaly classes resemble the training ones and badly when they don’t, so budget for a round of on-site labelling. The good news is that video-level tagging is fast — an operator can label a week of incident clips in an afternoon.

One-class and self-supervised VAD: learn normal, flag the rest

When you have no anomaly examples, you train only on normal footage and treat deviation as the signal. Two classic families do this. Reconstruction-based models (autoencoders) learn to rebuild normal frames; when they meet something unfamiliar, reconstruction error spikes. Prediction-based models learn to predict the next frame from the last few; when the actual future diverges from the prediction, you flag it. Both assume normal video is, in a sense, boring and predictable.

This is where benchmark hygiene bites hardest. On UCSD Ped2 the one-class setting is effectively solved (~97–99% AUC), CUHK Avenue sits around 90–92%, and ShanghaiTech in the one-class setting is much harder at roughly 74–78%. Quote a ShanghaiTech number without saying “one-class” and you can be off by twenty points. We’ll make that trap explicit in the benchmarks section.

One-class is the honest way to start a greenfield site: no incidents to learn from, so learn normality and raise a hand at the weird stuff. The trade-off is precision. “Unusual” catches a delivery van parked in a new spot as readily as it catches an intruder, so one-class systems lean hard on a triage layer and human review before anyone gets paged.

Transformer and vision-language VAD in 2026

The last two years moved the frontier from “better temporal networks” to “borrow a foundation model.” VadCLIP (AAAI 2024) adapts a frozen CLIP model for weakly-supervised VAD, aligning video with text so the model reasons about anomalies in language, not just pixels. It’s a big part of why weakly-supervised scores jumped.

More striking is LAVAD (CVPR 2024), which does training-free VAD: it captions frames with a vision-language model and prompts a large language model to produce temporal anomaly scores — no training, no data collection. It reports beating one-class and unsupervised methods on UCF-Crime and XD-Violence despite never seeing a training label. In the same wave, open-vocabulary VAD (CVPR 2024) detects and names both seen and unseen anomaly categories, breaking the old closed-set, binary framing.

Should you ship a zero-shot VLM pipeline today? For a cold start with unknown classes, it’s a genuinely useful first layer — you get reasonable alerts before you’ve labelled anything. But VLM/LLM scoring is heavier and slower than a tuned MIL model, so for high-camera-count, real-time sites we still put a lightweight scorer on the hot path and reserve the big models for triage and explanation. That’s a per-project call, and we’re happy to argue it out on your numbers.

Benchmarks and datasets that state the setting

If you remember one thing from this article, make it this: a VAD accuracy number is meaningless without its setting. The same dataset yields wildly different scores depending on supervision regime, split and metric. The chart below puts six common benchmark points side by side to make the trap obvious.

Figure 2. Frame-level AUC across benchmarks and settings. ShanghaiTech leaps ~76 → ~97 purely by protocol; UCSD Ped2 is saturated. Read the small print before you trust a number.

The datasets you’ll meet, oldest to newest: UCSD Ped2 (2010) and CUHK Avenue (2013) are small, low-resolution one-class sets — useful for sanity checks, retired for bragging rights. ShanghaiTech Campus (2017) has 13 scenes, 437 videos and 130 anomalous events and is used in both one-class and weakly-supervised setups, which is exactly why its numbers are so often quoted misleadingly.

UCF-Crime (2018) is the weakly-supervised workhorse: 1,900 untrimmed real CCTV videos, 128 hours, 13 anomaly classes, scored by frame-level AUC. XD-Violence (2020) is the largest public set at 4,754 videos and 217 hours, and it’s audio-visual and scored by average precision (AP), so its numbers aren’t comparable to AUC. Street Scene (2020) is a harder, high-resolution single-scene benchmark with 203,257 frames.

One more subtlety the leaderboards hide: frame-level AUC and AP say nothing about how fast a model reacts. A 2025 paper shows two detectors with identical AUC can flag the same event seconds apart, and proposes latency-aware metrics instead. For surveillance, where a ten-second delay changes the outcome, detection latency belongs in your acceptance test right next to accuracy.

The production pipeline, camera to alert

A shipping VAD system is not one model; it’s a chain. Cameras stream in over RTSP or ONVIF, frames are decoded and down-sampled, an object detector finds people and vehicles, a tracker stitches them into tracks over time, and only then does the anomaly scorer judge behaviour. The output isn’t an alarm; it’s a score that a triage layer turns into an alert, or quietly drops.

Figure 3. The production pipeline. Detection and tracking answer “what and where”; the scorer answers “is this normal”; triage decides whether a human is worth interrupting.

The mistake we see most often is treating the scorer as the whole system. In practice the detector and tracker do the heavy lifting of turning raw pixels into objects and trajectories, and the scorer reasons over those trajectories. Skip the tracker and your model has no temporal context — it can’t tell a person standing still from a person who has fallen, because both are “a person” in a single frame.

The other under-built stage is triage. A raw anomaly score fires constantly; the triage layer applies a threshold, simple rules (zones, schedules, dwell times) and de-duplication so an operator sees a handful of credible alerts a shift, not hundreds. We’ll quantify why that matters when we get to false alarms — it’s the difference between a system operators trust and one they mute.

The detection and tracking layer: YOLO26 and ByteTrack

The object layer is where 2026 actually moved. Ultralytics released YOLO26 in September 2025 as an edge-first detector that is NMS-free and end-to-end by default, drops Distribution Focal Loss to simplify the head, and runs roughly 43% faster on CPU than YOLO11n. If a vendor is still calling YOLO11 “the newest,” their stack is already a year behind. For reference, YOLO11 tops out at 54.7 mAP50-95 on COCO (the “95% mAP” you’ll see in some blogs is simply wrong, since COCO mAP caps well below that).

Detection alone isn’t enough; you need tracking to give the scorer time. Three trackers cover most cases. DeepSORT pairs Kalman motion with a deep appearance embedding — the legacy reference, still fine for sparse scenes. ByteTrack is motion-only but associates every detection box, including low-confidence ones, reaching about 80.3 MOTA on MOT17; it’s the throughput pick for busy scenes. BoT-SORT adds appearance and camera-motion compensation on top of ByteTrack for the best accuracy when the camera moves or crowds overlap.

Our default recipe for a fixed-camera site is YOLO26 for detection, ByteTrack for tracking, and a weakly-supervised scorer on top. Then swap ByteTrack for BoT-SORT only if the footage has real occlusion or moving cameras. Simpler is faster, and on the edge every millisecond you spend tracking is a millisecond the scorer doesn’t get.

Which approach fits your constraints

Put the regimes next to the constraints that actually decide a project — labels on hand, latency budget, accuracy ceiling and where it breaks — and the choice usually makes itself.

Notice the last row. If your “anomaly” is really “someone crossed this line after hours,” you may not need a learned model at all — a rule is cheaper, faster and easier to defend in court. We’ll say more about that in the when-not-to-use section.

Edge vs cloud inference: where the model runs

Once you’ve picked a model, the next decision shapes the whole budget: does inference run on a box at the site, or on a GPU in the cloud? It isn’t a religious question; it’s four trade-offs you can weigh.

Figure 4. Edge vs cloud, side by side. Edge trades capex and per-site boxes for low latency and on-site privacy; cloud trades network hops for elastic scale.

Edge inference keeps frames on-site, answers in well under a second, and costs a flat capex per camera. It’s the right call for fixed cameras, strict privacy and connectivity you can’t trust. The price is operational: you’re now shipping, updating and monitoring a box at every location. Modern edge silicon makes this very doable, as the next section shows with exact chips and prices.

Cloud inference centralises everything on GPUs you can scale up and down, which suits bursty load, many sites and heavy models like VLM scorers. The cost is a network round-trip on every stream, an opex bill that grows with your camera count, and video leaving the building, which, as the compliance section explains, is a real regulatory consideration, not just a latency one. Plenty of systems split the difference: cheap detection at the edge, heavy scoring and analytics in the cloud.

Hardware footprint and cost math

Edge VAD is cheaper than most people expect. The NVIDIA Jetson Orin Nano Super (December 2024) delivers 67 sparse-INT8 TOPS for $249 — it replaced a 40-TOPS unit that cost $499, so performance per dollar roughly tripled in one generation (about 1.7× the compute at half the price). Need more headroom? The Jetson AGX Orin reaches up to 275 TOPS. Prefer a co-processor on existing hardware? Hailo’s accelerators run 13–40 TOPS, and the Hailo-8L shipped as the $70 Raspberry Pi AI Kit. And some cameras skip the box entirely: Axis runs object analytics directly on its ARTPEC system-on-chip, so lighter anomaly logic can live on the camera itself.

A published edge system (RTFM with an I3D backbone, optimised through Torch-TensorRT) hit 47.56 FPS on a Jetson using 3.11 GB of RAM — comfortably real-time for a single stream. That “single stream” caveat matters: the detector, tracker and scorer all share one budget, so a sub-$300 box handles a few cameras, not forty. Plan boxes per camera group, not per site.

Figure 5. Worked cost math for 40 cameras. Edge is a one-time capex; cloud is a recurring opex that scales with GPUs. The crossover depends on how many hours a day you run.

Work a concrete example. Say 40 cameras, running 24/7. On the edge, if one Jetson Orin Nano Super handles ~8 streams, you need five boxes: 5 × $249 = $1,245, plus cameras, PoE and install (call it ~$8,000, which varies wildly), so roughly $9,300 in year-one capex and near-zero monthly cost. The cameras and PoE are needed either way, so the honest comparison is really the compute delta. In the cloud, if one AWS g6.xlarge (an L4 GPU at about $0.805/hour on-demand) fans out ~12 down-sampled streams, you need four GPUs: 4 × $0.805 ≈ $3.22/hour, times 730 hours a month ≈ $2,350/month, or about $28,200 a year before egress and storage. For fixed cameras running around the clock, edge pays for itself in months; for bursty or multi-site load, cloud’s elasticity wins. Cloud GPU prices drift, so re-check them on the day you size the cluster.

Mini case: monitoring a construction site

A construction-site client came to us with a familiar failure: their cameras had “AI analytics” that paged the site manager so often he’d turned the alerts off entirely. The system flagged every vehicle, every shadow, every worker crossing a zone. In effect they were paying for anomaly detection and getting a very expensive motion sensor. The events they actually cared about, a person in an active fall zone or a vehicle reversing near workers, were lost in the noise.

Our twelve-week plan was unglamorous on purpose. We put YOLO26 and ByteTrack on an edge box per camera cluster, trained a weakly-supervised scorer on a few hundred incident clips their safety team tagged, and — the part that actually fixed it — rebuilt the triage layer with zones, dwell times and a threshold tuned against the base rate. The arithmetic is worth seeing: if genuine anomalies are ~0.01% of frames, even a 99% specific detector produces far more false alarms than true ones, so the whole game is raising the bar and adding rules until the alerts a human sees are mostly real.

The outcome we optimised for wasn’t a benchmark score; it was reviewed-alert precision the safety team would trust again. In round numbers, the old setup fired on the order of a few hundred alerts a day; after we retuned triage to the base rate, what remained was a few dozen a single reviewer could clear in one shift. By moving the decision from “did anything move” to “is this pattern dangerous in this zone,” the manager turned notifications back on. Want a similar assessment of your own alert volume? Grab a 30-minute slot and we’ll look at a day of your footage together.

Pick your approach in five questions

1. Do you have labelled incidents? If yes, go weakly-supervised for the accuracy. If no, start one-class or zero-shot and collect labels as incidents happen, then graduate.

2. Is the event really an anomaly, or a rule? Line-crossing, loitering and after-hours presence are rules. If a hand-written rule covers 80% of your value, ship that first and add learned VAD for the fuzzy 20%.

3. What’s your latency budget? If a ten-second delay changes the outcome, keep a light scorer on the hot path and run the heavy VLM off to the side for explanation, not for paging.

4. Where must the video live? If frames legally can’t leave the site, that decides edge before any accuracy argument does. Compliance is a hard constraint, not a preference.

5. Who reviews the alerts? If nobody has time to clear a queue, no model is accurate enough — invest in triage and rules before you invest in a bigger model. An unwatched alert is worse than no alert.

Five pitfalls that sink VAD projects

1. Ignoring the base rate. As an order of magnitude, anomalies are a tiny fraction of footage — often well under 0.1%. A detector that’s “99% accurate” still floods operators with false positives; abandoned-object detectors have been cited with false-positive rates near 50%. Design for precision at your real base rate, not for headline accuracy.

2. Trusting a benchmark number without its setting. A ShanghaiTech score means nothing until you know one-class versus weakly-supervised. Ask which dataset, which split, which metric — and whether it was measured on anything like your scene.

3. Dropping the tracker. Without tracking, the scorer has no temporal context and can’t distinguish a person standing from a person who fell. Detection plus tracking is the foundation; the fancy scorer comes after.

4. Sizing the edge box for one stream and deploying forty. Real-time on a single stream doesn’t mean real-time on a whole site. The detector, tracker and scorer share the compute budget, so measure streams-per-box before you buy.

5. Measuring the wrong thing. AUC on a public set won’t tell your operators anything. Track false positives per 1,000 reviewed hours at a target recall on your priority classes, plus detection latency. Those are the numbers that decide whether people trust the system.

Compliance in 2026: EU AI Act, GDPR and BIPA

This is where most VAD articles are quietly out of date, so read carefully. The EU AI Act entered into force on 1 August 2024, its prohibited-practice and AI-literacy rules applied from 2 February 2025, and general-purpose-AI duties from 2 August 2025. The twist: the Commission’s “Digital Omnibus,” proposed in November 2025 and agreed in 2026, pushed the heaviest high-risk (Annex III) obligations from the original August 2026 date to 2 December 2027 (and embedded high-risk products to August 2028). Transparency (Article 50) and AI-literacy (Article 4) duties were not delayed, so “we have until 2027” is only half true.

On biometrics specifically: Article 5 prohibits real-time remote biometric identification in public spaces for law enforcement, with narrow, pre-authorised exceptions (searching for trafficking victims or missing persons, an imminent threat to life, or locating a serious-crime suspect). It is not a blanket ban, and it binds law enforcement rather than private CCTV. But any remote biometric identification system is classified high-risk under Annex III, so if your anomaly system also identifies people, you’re in the heavy regime. Behaviour-only anomaly detection that never tries to identify individuals is a much lighter footprint.

GDPR governs the cameras regardless of the AI Act. Ordinary surveillance usually rests on legitimate interest under Article 6(1)(f), but the moment you turn a face into a template to uniquely identify someone, you’re processing biometric special-category data under Article 9 and generally need explicit consent; the controlling guidance is the EDPB Guidelines 3/2019 on video devices. In the US, there’s no federal statute, but Illinois BIPA carries statutory damages of $1,000 to $5,000 per violation — though 2024’s SB 2979 limited that to a single violation regardless of how many scans occur. None of this is legal advice; loop in counsel for your jurisdiction before you deploy.

When NOT to use AI video anomaly detection

Honest answer: plenty of the time, you shouldn’t. If your requirement is “alert me when someone crosses this line after 8pm,” that’s a rule, and a rule is cheaper, faster, fully explainable and far easier to defend to a regulator than a learned model. Reach for VAD when the interesting behaviour is genuinely open-ended, not when a tripwire would do.

Skip learned VAD, too, when nobody can review the alerts. A model that pages an empty desk is worse than no model, because it trains everyone to ignore the system. If you have three cameras and no operations team, a good VMS with basic analytics beats a bespoke anomaly pipeline. And if your compliance context forbids the processing outright — certain biometric uses, certain jurisdictions — the right answer is to not build it, or to build a behaviour-only version that never identifies anyone.

We’d rather tell you this on a call than sell you a pipeline you don’t need. When VAD is the wrong tool, we say so — and when it’s the right one, we can point at the surveillance products we’ve run for a decade as proof we’ll build it properly.

FAQ

What to read next

Ready to put anomaly detection on your cameras?

Video anomaly detection has stopped being a research demo and become an engineering decision with known parts: choose the supervision regime your labels support, read benchmark numbers with their setting attached, build the detector-tracker-scorer-triage chain in full, and price edge against cloud on your real duty cycle. The models are good enough in 2026; the projects that fail, fail at triage and at the base rate, not at accuracy.

The last mile is judgement — which regime, which hardware, which events are rules and which are genuinely anomalies, and what the law lets you do with faces. That’s the part we do every week, on surveillance products that have run for a decade. Bring us your site and we’ll give you a straight answer, including “you don’t need this” when that’s the truth.