Speech-to-speech systems hear and reply in voice within one model, cutting the delay and information loss of chaining ASR, an LLM, and TTS. Realtime APIs and Gemini Live use this approach to make voice agents feel responsive and lifelike.
The book · Volume 7 of 7
AI for Video: A Complete Guide to Intelligence in Video Products: From Real-Time Pipelines and Agents to Serving, Cost, Compliance, and Verticals
Speech-to-speech models cut the delay of chaining ASR, an LLM and TTS, which is what makes a voice agent feel responsive. Volume 7 covers speech-to-speech in detail: real-time voice pipelines, call latency budgets, agents as engineering with tools and memory, serving infrastructure and the cost per stream-hour of live voice AI. Written by Nikolay Sapunov, CEO at Fora Soft.
Building a voice agent on speech-to-speech models instead of an ASR-LLM-TTS chain?
Tell us the scenario and target latency, and we will compare speech-to-speech with a chained pipeline and estimate cost per conversation minute. 250+ video products shipped since 2005.
