Video AI Inference Serving — Decision Checklist

One page: how to serve video AI models in production. The two ideas that make an LLM/VLM server fast (PagedAttention cutting KV-cache waste from 60-80% to under 4%; continuous batching lifting GPU utilization from 30-40% to 75-90%, together ~2-4x throughput on the same card); the serving choices and when to use each (vLLM the open default; Ollama for one user/one machine only; SGLang for prefix-heavy work; TensorRT-LLM for max NVIDIA speed with its 2026 PyTorch backend; Triton for many models on one fleet, with NVIDIA Dynamo as its datacenter successor; serverless GPUs for spiky traffic); the per-stage decision for a video pipeline (decode, detect on TensorRT, transcribe on faster-whisper/CTranslate2, describe and summarize on vLLM, wired as a Triton ensemble); the one batching cost example (85%/35% = 2.4x -> 58% cheaper per token); and the questions to ask before committing. Routes to /services/ai-software-development.