A GPU server is a machine fitted with one or more graphics processing units, used in surveillance to run video analytics at scale. GPUs excel at the parallel maths that neural networks need, so a single GPU server can decode many camera streams and run detection, tracking, or recognition across them — far more than a CPU could, and far more than any single camera's NPU. It is the workhorse of the on-prem and cloud analytics tiers alike.
Its job is concentrated inference. Rather than putting modest AI on every camera, a GPU server centralises the heavy lifting: dozens of streams in, models running on the GPU, metadata and events out. Renting an equivalent GPU in the cloud does the same job for bursty or temporary workloads — and is dramatically cheaper than per-minute analytics APIs for continuous analysis, often tens of dollars a month versus thousands.
The pitfall, as with any GPU-based video tier, is that decoding — not inference — is frequently the real bottleneck. Each incoming stream must be decoded before the GPU can analyse it, and decode capacity (sometimes a separate hardware function) can cap camera density well below what the GPU's raw inference throughput suggests. Size a GPU server by decode-plus-inference for the specific models and resolutions in play, build in headroom and redundancy, and decide deliberately whether the GPUs live on-prem or in the cloud.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
A GPU server carries on-prem analytics from roughly 50 cameras up, and sizing it wrong wastes money or drops streams. Our seven-volume handbook covers the full video stack, and for GPU server sizing start with Volume 7: serving with vLLM, Triton and TensorRT, and cost per stream-hour. Volume 6 covers the detection and tracking models it runs. Written by Nikolay Sapunov, CEO at Fora Soft.
How many camera streams can one GPU server handle for your analytics?
Share your models, resolutions and camera count, and we will benchmark and size GPU servers with streams per GPU and cost per camera. 250+ video products shipped since 2005.
