An analytics pipeline is the ordered chain of steps that turns raw video into useful events. A typical pipeline decodes the incoming stream, pre-processes the frames (resize, colour-convert), runs detection to find objects, optionally tracks them across frames and classifies or recognises them, applies rules ("crossed this line", "loitered here"), and finally emits an event and metadata the VMS can record, alert on, and make searchable. Each stage feeds the next.
Seeing analytics as a pipeline matters because every stage has a cost and an accuracy, and the weakest stage caps the whole. If detection misses an object, no amount of clever downstream tracking or search will recover it; if decoding cannot keep up, the GPU starves. The pipeline also defines where work can be split — early, cheap stages on the edge, heavier stages on a server or cloud — which is the heart of the edge-vs-cloud decision.
The pitfall is optimising one stage in isolation. Teams obsess over the detection model's accuracy while the real loss is an under-provisioned decode step dropping frames, or a rule stage generating false alarms that bury true ones. Design and measure the pipeline end to end — throughput and accuracy at every stage — and remember it is only ever as good as its worst link. The detection and tracking model internals belong to the AI for Video Engineering section.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
An analytics pipeline decides where decoding, detection and tracking run, and that placement drives latency, bandwidth and hardware cost per camera. We wrote seven volumes covering the whole video stack, and for an analytics pipeline start with Volume 7: real-time pipeline latency, serving and inference with TensorRT and Triton, cost per stream-hour and vertical playbooks. Written by Nikolay Sapunov, CEO at Fora Soft.
Deciding where each stage of your video analytics pipeline should run?
Tell us your camera counts and detection needs and we will deliver an analytics pipeline design with stage placement, hardware sizing and cost per camera. 250+ video products shipped since 2005.
