Attention scores how much each input element should influence each output element, so the model focuses on the relevant words or image patches. It is what gives transformers their grasp of context, and its compute cost shapes how long inputs can be.
The book · Volume 6 of 7
AI for Video: A Complete Guide to Intelligence in Video Products: From Models and Inference to Computer Vision, Speech, Search, and Generative Video
Attention is why transformers grasp context, and its compute cost is why long videos must be sampled or chunked before a model sees them. Volume 6 covers the ML basics needed for product decisions, with model artifacts, deployment topologies and the vision and speech models built on attention, so you can reason about context length and cost. Written by Nikolay Sapunov, CEO at Fora Soft.
Video models running out of context or budget because of attention cost on long footage?
We will review your pipeline and recommend frame sampling, chunking and model choices, with the cost per hour of video for each. 250+ video products shipped since 2005.
