
LiveKit AI voice agents are the difference between a “press 1 for support” IVR and a caller who thinks they’re talking to a person. A production-grade agent in 2026 can qualify a lead, book an appointment, or clear a tier-1 support call with a sub-second feel — at roughly 5–10% of the cost of a human-staffed call centre. Get the latency budget wrong, though, and you ship a robot that people hang up on.
This is the build-and-deploy guide we wish we’d had when we shipped our first LiveKit agent back on the framework’s 0.x releases. We’ve been building real-time audio and video products since 2005 — 250+ projects, 99% Upwork success rating — and voice agents are now a big part of that. Below: the exact stack to pick, the numbers you have to hit, real per-minute costs, the five pitfalls that kill projects, and how LiveKit stacks up against Vapi, Retell, Pipecat and Bland. Want the architecture theory first? Our LiveKit for AI agents guide in Learn covers how the pieces fit; this article is how you actually build and ship one.
Key takeaways
• Sub-500 ms perceived latency is the whole game. Under ~300 ms feels human, over 600 ms and callers revert to touch-tone, over 1.5 s and they hang up. Every stack decision serves that one number.
• Default to cascade (STT→LLM→TTS); reach for speech-to-speech only where naturalness is the product. Hybrid — S2S for chit-chat, cascade for tool-calls — is the growing 2026 pattern where voice quality is the product.
• Below ~10,000 minutes/month, buy Vapi or Retell. Above it, build on LiveKit. The framework path undercuts managed platforms 60–80% per call at scale; managed wins once you count engineering time at low volume.
• Ship the eval suite in week 2, not month 6. Every voice-agent rollback we’ve diagnosed traces back to missing observability or skipped compliance — not to the model.
• EU AI Act Article 50 transparency has applied since 2 August 2026. If you serve EU users, the AI disclosure must already be live on every in-scope call; the July 2026 Digital Omnibus did not move this date.
Updated October 2026: LiveKit Agents 1.8 code and features (audio turn detector, adaptive interruptions, built-in evals), October 2026 LiveKit Cloud and model prices, a coding-agent workflow, LiveKit vs Retell for small businesses, and EU AI Act status after the Digital Omnibus.
Why product owners pick LiveKit for voice agents in 2026
LiveKit is the open-source WebRTC stack that OpenAI (for ChatGPT’s voice mode), Character.ai, and thousands of smaller vendors run their real-time audio on. LiveKit Agents — the framework this guide is about — went 1.0 in April 2025 and, as of October 2026, is at version 1.8.5 (released 6 October 2026, per PyPI), with an audio-based turn detector, adaptive interruption handling, built-in testing and simulations, and native Model Context Protocol (MCP) tool support. LiveKit itself raised a $100M Series C at a $1B valuation in January 2026, so the framework is not going anywhere soon.
Three reasons product owners pick LiveKit over a managed platform like Vapi or Retell:
- Cost at scale. Above roughly 10,000 minutes/month, the framework path undercuts managed platforms by 60–80% per call. Below that, managed is cheaper once you count engineering time.
- Vendor freedom. Bring your own STT (Deepgram, AssemblyAI, Whisper), LLM (Claude, GPT, Gemini, open models), and TTS (Cartesia, ElevenLabs, Azure). No lock-in on any layer.
- Telephony is first-party. LiveKit shipped native SIP and Phone Numbers in 2025, so inbound and outbound calling no longer needs a Twilio bridge.
We’ve shipped LiveKit-style agents for customer support, outbound qualification, in-app voice companions, and regulated workflows — including a medical AI voice appointment-booking assistant that verifies caller identity, looks up records, offers slots, books the visit, and schedules reminders end to end. That production work is what our AI voice agent development team on LiveKit brings to every new build, and the patterns below are the ones that held up under real traffic.
What a LiveKit AI voice agent actually is
A LiveKit agent is a process, not a chatbot. It joins a LiveKit “room” as a participant — exactly the way a human would if they dialed in — subscribes to the caller’s audio track, runs that audio through speech recognition, feeds the text to an LLM (optionally with tool-calling), and publishes the reply back as synthesized speech on its own track. Real-time, bi-directional, fully parallel.

Figure 1. The four stages run concurrently inside one AgentSession, so the caller never waits for a previous stage to finish.
The core primitive is AgentSession, which unified the older VoicePipelineAgent and MultimodalAgent abstractions into one orchestrator in the 1.0 release. You declare STT, LLM, and TTS as pluggable components, register tool functions, and the SDK wires up streaming, turn detection, and interruption handling.
Why “a process in a room” matters: because the agent is just another participant, you can drop a human into the same room to take over, record the whole conversation, run a supervisor agent that watches the first, or bridge to a SIP phone line — without changing your architecture. That flexibility is why LiveKit beats bespoke stacks on iteration speed.
New to the underlying transport? Our WebRTC explainer covers what makes real-time media possible. LiveKit sits on top of WebRTC and runs the server-side SFU you’d otherwise have to operate yourself.
The latency budget that separates useful from unusable
Every voice agent lives or dies on one number: the time from the caller finishing their turn to the first syllable of the agent’s reply. Under 300 ms it feels human. 300–600 ms feels sluggish but acceptable. Above 600 ms callers revert to touch-tone mental models and start tapping keys. Above 1.5 s they hang up.
Here’s the budget that hits sub-500 ms perceived latency in production:

Figure 2. Each stage’s target p50 stacked from zero to the ~450 ms a caller perceives as near-human.
| Stage | Target p50 | Target p95 | Who owns it |
|---|---|---|---|
| Endpointing / VAD | 80 ms | 160 ms | Semantic VAD model |
| Final ASR transcript | 120 ms | 250 ms | Deepgram / AssemblyAI |
| LLM first token | 180 ms | 400 ms | OpenAI / Claude / Gemini |
| TTS first audio | 70 ms | 140 ms | Cartesia / ElevenLabs |
| Perceived total | ~450 ms | ~950 ms | The whole pipeline |
A hard truth from 2025–2026 production data: published medians across millions of real calls sit around 1.4–1.7 s, and p99 runs 3–5 s. The 450 ms number is achievable — but only with streaming at every stage, co-located regions, pre-warmed model contexts, and disciplined observability. It won’t happen by default. The budget also starts when the turn detector decides the caller is done: the silence threshold you tune per vertical (below) and network or telephony transit sit on top, so a phone agent always lands above the 450 ms figure end to end; measure the full path, not just the pipeline.
The big gotcha is first-token LLM latency. A model that feels fast in chat (1 s first token) blows the whole voice budget on its own. Pick a model that streams its first token in under 300 ms, even if its reasoning is slightly weaker.
Thinking about a voice agent for your product?
30 minutes with a Fora Soft engineer who has shipped LiveKit agents in production. We’ll size the rollout, pick a stack, and hand you a realistic per-minute cost.
LiveKit Agents 1.x: the 2026 framework
At runtime the pipeline is four layers, each running concurrently so the caller never waits on a previous stage. The runtime pieces around AgentSession:
- Worker. A long-lived process that dispatches one Job per incoming room. One worker runs dozens of concurrent sessions, each job in its own pre-warmed process by default, so size memory per job.
- Job. A single agent-on-a-call lifecycle. Each Job has its own LLM context, STT stream, and TTS buffer.
- Plugins. Drop-in implementations of STT, LLM, TTS, and VAD. Swapping Deepgram for AssemblyAI is a one-line change.
- Tool registration. Decorated Python functions become callable by the LLM mid-turn, via OpenAI tool-calling or MCP.
The streaming transport is the same WebRTC LiveKit uses for human video calls, so the agent can also join a video conference, watch a screen share, and answer questions about what’s on screen — useful for support agents that need to see what the user sees. How to capture and publish that screen on mobile is covered in our Android WebRTC screen-sharing guide.
Building LiveKit agents with Claude Code and other coding agents
Yes, you can build a LiveKit agent with Claude Code, Cursor or Codex, and LiveKit now supports it first-party. Point the coding agent at the LiveKit Docs MCP server (claude mcp add --transport http livekit-docs https://docs.livekit.io/mcp), install LiveKit’s Agent Skills (npx skills add livekit/agent-skills, also packaged as a Claude Code plugin), and start from the Python or Node.js starter, which ships an AGENTS.md tuned for coding agents (LiveKit docs, October 2026).
The loop that works for us: the coding agent writes the agent class and tools, writes text-mode tests for each tool and handoff, and runs the agent locally through lk agent debugger, which a script or coding agent can drive turn by turn. A senior engineer still owns the parts a model gets wrong most often: the latency budget, interruption settings per vertical, write-tool confirmations and the compliance script. That split is how we ship LiveKit work faster than a hand-coded build. The same approach works for Pipecat pipelines; our Pipecat vs LiveKit Agents comparison covers which framework to point it at.
Speech-to-speech vs cascade: which pipeline to pick
There are two architectural options in 2026, and most production agents now combine them.
Cascade (STT → LLM → TTS). The traditional pipeline: three vendors, three models, three logs. More moving parts, but you pick the best model at each layer, redact PII between stages, and swap a vendor without a rewrite. This is what about 90% of LiveKit production agents still run in 2026.
Native speech-to-speech (S2S). OpenAI’s Realtime API (gpt-realtime-2.1 and the cheaper gpt-realtime-2.1-mini) and Google’s Gemini Live API (Gemini 3.8 Live, released 15 September 2026) take audio in and emit audio out — no explicit text stage. Latency drops because there is no text hand-off between three vendors, and pauses and prosody sound more natural. OpenAI lists gpt-realtime-2.1 audio at $32 per 1M input tokens and $64 per 1M output tokens; Google quotes Gemini 3.8 Live at about $0.005 per minute of audio in and $0.018 per minute out (vendor price pages, October 2026). The trade-offs: less predictable cost (we break down what the OpenAI Realtime API really costs per minute), harder to log and redact, single-vendor lock-in.
Hybrid is the growing 2026 pattern: use S2S for the natural-feeling parts of a call (greetings, rapport, clarifications). Drop to cascade when the agent needs to call a tool — tool-calling is more reliable on text-mode LLMs — then return to S2S for the response. In LiveKit you build this as an agent handoff between a realtime-model agent and a cascade agent inside the same session.
If your use case involves PII redaction (healthcare, finance), start with cascade. If it’s long conversational flows with no tool calls (coaching, companions, storytelling), start with S2S.
Turn detection, VAD and barge-in that feel natural
The single most common complaint about bad voice agents is interruption handling. Either the agent barrels on after the caller starts speaking (robotic), or it stops on every breath (annoying). The 2026 answer is a two-signal model:
- Acoustic VAD (Silero). Sub-millisecond inference per audio chunk. Detects whether someone is speaking. Fast but naive — can’t tell “umm” from end-of-turn.
- Model-based turn detection. LiveKit’s turn detector now listens to the audio, not only the transcript. The
v1audio model runs on LiveKit Inference (free for agents deployed to LiveKit Cloud) andv1-miniruns locally on CPU; both cover 14 languages (LiveKit docs, October 2026). The older text-based MultilingualModel, a Qwen2.5-0.5B distillation that cut false interruptions by 39% in its December 2025 release, is deprecated and slated for removal in Agents 2.0.
Barge-in (caller interrupts the agent mid-sentence) is handled by the runtime: when VAD fires on the caller’s track while the agent is speaking, TTS is cancelled, the interrupted LLM turn is rolled back, and the new input is processed. Since March 2026 adaptive interruption handling is on by default (Python Agents 1.5.0+): an audio model checks the first few hundred milliseconds of overlapping speech, decides in 30 ms or less, and, per LiveKit’s own numbers, rejects 51% of VAD-triggered barge-ins as backchannels like “uh-huh” instead of cutting the agent off.
One practical tip: tune the VAD silence threshold to your vertical. Sales calls want ~400 ms of silence before end-of-turn (people think out loud). IVR replacement wants ~250 ms (callers are purposeful). Healthcare intake wants ~600 ms (older callers, longer pauses). A single default will feel wrong for at least two of the three.
Tool-use: letting the agent actually do the work
The difference between a voice chatbot and a voice agent is the ability to take action mid-conversation. In LiveKit Agents 1.x, tools are ordinary async methods on your Agent class marked with @function_tool; the docstring becomes the description the LLM sees. (This replaced the 0.x ai_callable decorator, so older tutorials will not run.) Tools from an MCP server plug into the same session alongside your own.
from livekit.agents import Agent, RunContext, function_tool
class SupportAgent(Agent):
def __init__(self) -> None:
super().__init__(instructions="You are a concise phone support agent.")
@function_tool()
async def get_order_status(self, context: RunContext, order_number: str) -> dict:
"""Look up order status by order number."""
return await crm.orders.fetch(order_number)
@function_tool()
async def book_callback(self, context: RunContext, phone: str, iso_time: str) -> str:
"""Schedule a follow-up call at an ISO 8601 time."""
return await scheduler.book(phone, iso_time)
During a turn the LLM emits a tool call, the runtime runs the function, streams the result back into the LLM’s context, and the model continues with the updated information. The caller hears a short filler (“one moment, let me check that”) so the pause feels intentional, not laggy. Three patterns we recommend:
- Read tools are cheap, write tools are expensive. Let the agent look up anything. Put writes (sending email, charging a card, cancelling an appointment) behind explicit caller confirmation.
- Tools fail — design for it. Every tool wrapper should time out after ~2 seconds and return a graceful “system unavailable” message the LLM can verbalize naturally.
- Log every tool call. For debugging, evals, and audit trails. It also becomes a legal duty if your use case is classed as high-risk under the EU AI Act.
Our voice AI agents guide walks a complete tool-using agent step by step, including confirmation patterns and the prompt shape that keeps tool-calls reliable.
SIP and telephony: putting the agent on a phone number
Until mid-2025, getting a LiveKit agent onto a real PSTN phone line meant bridging through Twilio or Telnyx with fiddly SIP glue. LiveKit SIP went GA and LiveKit Phone Numbers shipped in 2025, so a voice agent now accepts a call from any phone on Earth with about four lines of configuration.
For inbound, the pattern is: point a SIP trunk (LiveKit’s own, Telnyx, or any provider) at LiveKit’s SIP endpoint; the trunk forwards the call into a room; the worker spawns a Job on that room. For outbound, the agent initiates the SIP INVITE via LiveKit’s server API. Both paths are documented and have code samples in the agents-python repo, and Agents 1.8 added warm transfer through the Twilio connector, so the agent can brief a human before handing the caller over. If the phone line is the product, our AI call agent development team handles the trunking, warm transfer and CRM hand-off as one build.
Telephony is where these agents earn their keep. We built a hospital over-the-phone interpretation platform-class system on SIP/FreeSWITCH so a doctor picks up any landline, selects a language from an IVR menu, and reaches a live interpreter in seconds — the same telephony plumbing a voice agent rides on.
Pricing note: LiveKit Cloud lists US local numbers at $1.00 per month (one free per project), US local inbound at $0.01 per minute, toll-free inbound at $0.02 per minute, and third-party SIP trunk minutes at $0.004 per minute on the Ship plan (LiveKit pricing page, October 2026). Bring your own Twilio or Telnyx trunk when you already own the numbers or need coverage outside the US; otherwise native LiveKit numbers are the simpler path.
Vendor matrix: ASR, LLM and TTS in 2026
Our October 2026 defaults for a production English-language agent, with alternatives for common edge cases. Prices are LiveKit Inference list prices on the Build and Ship plans (livekit.com/pricing/inference, checked 9 October 2026); direct vendor contracts can differ.
| Layer | Default pick | Why | Swap to… |
|---|---|---|---|
| ASR | Deepgram Flux | Conversational STT with built-in end-of-turn; $0.0065/min (Nova-3: $0.0048/min) | AssemblyAI Universal-Streaming ($0.0025/min), Whisper (self-host) |
| LLM | GPT-4.1 mini / GPT-5 mini | Low first-token latency, reliable tool-calling; $0.40/$1.60 and $0.25/$2.00 per 1M tokens in/out | Claude Sonnet (harder reasoning), Gemini 3 Flash or 2.5 Flash-Lite (cheap scale) |
| TTS | Cartesia Sonic 3.6 | Fast first audio, natural prosody; $50 per 1M characters ($37.50 on Scale) | ElevenLabs (quality), Deepgram Aura-2 or Inworld (price) |
| VAD | LiveKit turn detector (audio v1) + Silero | Best open turn detection available; free on LiveKit Cloud | Deepgram Flux end-of-turn for single-vendor |
| S2S | OpenAI gpt-realtime-2.1 | Most mature, wide voice library | Gemini 3.8 Live (cheaper audio minutes) |
Running on Azure? LiveKit agents work with Azure end to end: the Azure plugin covers Azure AI Speech STT and TTS, Azure OpenAI models plug in through the OpenAI plugin, and LiveKit Inference serves the GPT-4.1 and GPT-5 families from Azure at the same list prices as OpenAI. For data-residency projects we self-host the LiveKit server in the customer’s Azure region on AKS and keep agent workers next to the Azure OpenAI deployment.
Need a face, not just a voice? LiveKit’s avatar plugins (Tavus, Hedra, Beyond Presence, Simli, Anam, bitHuman) publish a lip-synced video track into the same room, so the agent logic does not change. Our interactive AI avatar development guide compares the vendors.
Deploying: LiveKit Cloud vs self-hosted
Three deployment modes to choose from:
- LiveKit Cloud. Managed SFU, agent dispatch, observability dashboard, global PoPs. The “just works” option, fastest to MVP.
- Self-hosted server + cloud agents. You run the SFU on your own Kubernetes or ECS; agent workers live wherever. A good fit if you already operate real-time video infrastructure.
- Fully self-hosted via a SIP partner. LiveKit + Telnyx or Wavix, no LiveKit Cloud. Disclosed savings of ~50% per call for high-volume deployments — but you need an ops team.
LiveKit Cloud plans start free (Build: 1,000 agent session minutes and 5 concurrent sessions), then Ship at $50/month with 5,000 minutes and 20 concurrent sessions, and Scale at $500/month with 50,000 minutes and up to 600 concurrent sessions; overage is $0.01 per agent session minute (LiveKit pricing page, October 2026). Self-hosted agents connecting to Cloud bill as WebRTC participant minutes instead. For almost every project below 100k minutes/month, LiveKit Cloud wins on total cost of ownership once engineering time is counted. Above that, the self-hosted path starts to pay for itself in 6–9 months. Autoscaling in all three modes is driven by Worker concurrency: each worker holds N sessions, and you add workers linearly with call volume. Plan for bursts — a 5x spike from a marketing push is common — by keeping a pool of pre-warmed workers so cold-start doesn’t show up in p99 latency.
Cost model: what a production call actually costs
Real per-minute numbers from production deployments we’ve seen in 2025–2026. Your mileage varies with voice verbosity and LLM token usage.

Figure 3. Per-minute cost stacked from zero across three stacks — every one is a fraction of a $7–12 human BPO call.
| Component | Budget stack | Balanced stack | Premium stack |
|---|---|---|---|
| LiveKit session | $0.010 | $0.010 | $0.010 |
| ASR | $0.010 (Deepgram) | $0.010 (Deepgram) | $0.015 (AssemblyAI) |
| LLM | $0.008 (Gemini Flash) | $0.020 (GPT-4o mini) | $0.050 (Claude Sonnet) |
| TTS | $0.015 (Azure Neural) | $0.030 (Cartesia) | $0.090 (ElevenLabs) |
| Telephony (if used) | $0.010 | $0.013 | $0.015 |
| Total / min | ~$0.05 | ~$0.08 | ~$0.18 |
Against a typical $7–12 human-BPO per-call cost, even the premium stack stays under $1 for a typical 4–5 minute call, at least 8x cheaper per call. Reality check: expect your first production deploy to land 20–30% above the table once you count retries, unused tokens, and ops overhead. For how we price this kind of work end to end, our software estimating guide covers the three-number method we use for voice-agent builds. With Agent Engineering (senior engineers directing AI coding agents) we’re routinely at the lower end of these ranges.
Treat the table as a conservative ceiling: list prices kept falling through 2026. Via LiveKit Inference in October 2026, Deepgram Nova-3 is $0.0048/min, Deepgram Flux $0.0065/min and Cartesia Sonic 3.6 $50 per 1M characters, while the LiveKit agent session itself is $0.01/min beyond the plan allowance. For the platform line item alone, our LiveKit vs Agora pricing analysis compares the two real-time platforms at different scales.
Need a working prototype this month?
We routinely ship a LiveKit voice-agent pilot — tool-use, live telephony, observability — in 4–6 weeks. Free scoping call with a senior engineer.
Observability and evals: you can’t ship what you can’t measure
The single biggest cause of voice-agent rollback we’ve seen is a team shipping without traces. A bad turn is usually invisible in logs — you need the audio plus the transcripts plus the LLM response plus the tool result to understand what the agent did wrong. The 2026 stack that works:
- Turn-level traces. LiveKit’s OpenTelemetry hooks emit one span per turn with ASR, LLM, TTS, and tool-call timings. Agents 1.8 follows the full OpenTelemetry GenAI semantic conventions and strips PII in-process, and 1.7 added semantic PII redaction for chat history and call recordings. Wire this into your existing APM.
- Call recordings. Dual-track audio archived to S3 or equivalent, with retention matched to your compliance (30 days for most, 7 years for finance).
- Eval suite. A nightly job replays 100–500 canned scenarios through the agent and grades responses against a rubric. Catches silent regressions when a vendor updates a model. LiveKit now ships the tooling: text-mode unit tests that assert on messages, tool calls and handoffs in pytest or Vitest, and Agent Simulations that run multi-turn scenarios against a simulated user, in text or over audio, from CI.
- Error taxonomy. Every failed turn gets a label: tool-timeout, hallucinated-fact, barge-in-misfire. Track the trend over time.
Ship the eval before the first customer call: a voice agent without regression evals silently degrades the moment any vendor updates a model. Build the eval rig in week 2, not month 6. 100 scenarios, 20 golden transcripts, one CI job.
Use cases shipping in 2026
Categories where LiveKit voice agents are in live production at scale as of Q3 2026:
- Tier-1 customer support. Refunds, returns, password resets, basic troubleshooting. 40–70% containment on narrow domains.
- Appointment booking. Dental, automotive, salon, veterinary, medical. 24/7 intake with calendar integration and reminders.
- Outbound qualification. B2B lead follow-up, BANT scoring, scheduling a human rep for promising ones. Replaces SDR power-dial.
- Collections (soft). Payment reminders, plan setup, account updates. Regulatory-heavy — needs TCPA compliance.
- Healthcare intake. Pre-visit forms, symptom capture, insurance verification. HIPAA scope requires a Business Associate Agreement with every vendor.
- In-app voice companions. Embedded in mobile and web for coaching, tutoring, accessibility. Shorter calls, higher concurrency. For an agent that sits in a multi-person call and takes notes instead of talking, see our guide to the LiveKit AI meeting assistant: architecture and cost.
Still struggling in 2026: emotionally sensitive long-form dialogue (therapy, grief), multi-language round-trips with code-switching, and any write-action where a hallucination is expensive (wire transfers, policy cancellation).
LiveKit vs Vapi vs Retell vs Pipecat vs Bland
A 2026 comparison across the five options most product teams evaluate. For an open-source-only, head-to-head read, see our Pipecat vs LiveKit Agents comparison; for architecture trade-offs and alternatives beyond these five, our LiveKit voice agent architecture and alternatives guide goes deeper.
| Option | Type | Time to 1st call | Cost / min | Best fit |
|---|---|---|---|---|
| LiveKit Agents | Open-source framework | 2–6 weeks | $0.05–0.18 | 10k+ min/mo, custom integrations |
| Vapi | Managed, code-first | 2–3 hours | $0.05 platform fee + models (~$0.10–0.30 all-in) | <10k min/mo, fast MVP |
| Retell AI | Managed, visual builder | 3–6 hours | $0.055 infra + LLM + $0.015 telephony (~$0.10–0.30 all-in) | Non-technical owners, <10k min/mo |
| Pipecat | Open-source framework | 2–6 weeks | $0.04–0.17 | Custom orchestration, video+voice |
| Bland AI | Managed telephony | 1–2 days | $0.08–0.20 | Regulated outbound, TCPA-heavy |
Our short-form rule: pick Vapi or Retell for anything below ~10k minutes/month while you validate; switch to LiveKit Agents (or Pipecat if you need tighter video+voice coupling) once volume or customization crosses that threshold.
LiveKit Agents vs Retell AI for a local business. For a single-location business (a dental clinic, salon or HVAC company) handling a few hundred to a few thousand call minutes a month, Retell AI is the faster and cheaper choice: $0.055/min voice infrastructure, 20 concurrent calls included and $10 of free credit (Retell pricing page, October 2026), and no engineering team needed. LiveKit Agents wins when the same automation has to scale across many locations or franchises (tens of thousands of minutes), call in-house systems, or keep audio and transcripts in your own cloud: the platform fee is $0.01 per agent session minute, but you pay for the build. Our Vapi vs Retell vs custom breakdown runs the per-minute math at higher volumes, and if you have already outgrown a managed platform, our team handles migrating voice agents off Vapi or Retell to LiveKit without redesigning the conversation.
Compliance: EU AI Act, consent, and PCI
Three compliance areas that voice-agent projects repeatedly under-scope:
- EU AI Act — Article 50 transparency, in force since 2 August 2026. AI systems that interact with people must disclose it. If your agent serves EU users, you need an AI-disclosure at the start of the call (“You’re speaking with an AI assistant”), unless that is already obvious. That disclosure is the legal minimum; we also log every interaction and keep your model provider’s compliance documentation, which Article 50 does not require but which enterprise customers ask for and high-risk use cases must have. (The general-purpose-AI obligations landed a year earlier, on 2 August 2025. The Digital Omnibus, in force since 27 July 2026, left Article 50 disclosure untouched; it moved stand-alone high-risk Annex III deadlines to 2 December 2027 and gave generative systems already on the market until 2 December 2026 for machine-readable output marking.)
- Call-recording consent. US all-party states (California, Florida, Illinois, Maryland, Massachusetts, Montana, Nevada, New Hampshire, Pennsylvania, Washington) require every party to consent. GDPR in the EU requires a lawful basis plus a right-to-erasure path for audio and transcripts.
- PCI DSS. If the agent ever takes a card number by voice, you need in-call DTMF-or-audio redaction before the data hits the LLM or the logs. Several vendors ship drop-in pause-resume patterns.
Practical note: document, then enforce. Before the first customer call, write down (a) the AI-disclosure script, (b) the recording-retention policy, (c) the tools the agent may call, and (d) the escalation rule. That document is your regulator-ready audit trail.
Mini-case: a 6-week LiveKit support agent rollout
A mid-market SaaS client running ~12,000 inbound support calls (roughly 85,000 minutes) per month through a human team asked us to pilot a LiveKit agent for the first-response layer. The goal: deflect 40% of calls without hurting CSAT. Here’s what six weeks looked like.
| Week | Milestone | Outcome |
|---|---|---|
| 1 | Scope 20 intents, write eval transcripts | 100 scenarios, 20 golden turns |
| 2 | AgentSession + Deepgram + GPT-4o mini + Cartesia | p95 latency 780 ms |
| 3 | Tool-use (CRM read, ticket create) + eval CI | Eval pass-rate 82% |
| 4 | SIP trunk, AI disclosure, recording pipeline | First live call at 5% traffic |
| 5 | Prompt tuning, escalation rules, barge-in calibration | 20% traffic, CSAT parity with humans |
| 6 | 50% traffic, dashboards, incident runbook | 47% containment, cost ~$0.09/min |
Total external build cost: about $72,000 across six weeks (two senior engineers, one designer for the disclosure UX). Estimated annualized savings against the previous human-only ops: ~$420,000 after the ~$8,000/month vendor stack. Payback under 10 weeks. Our AI integration services on LiveKit shave another ~25% off build time on projects like this, because the scaffolding — eval rig, trace pipeline, disclosure UI, SIP wiring — is reusable across clients. Want the same for your call volume? Book a 30-minute scoping call.
Decision framework: buy, build, or hybrid
Five questions decide whether you build on LiveKit, buy a managed platform, or run a hybrid:

Figure 4. The volume line that flips the math, and the five questions that decide build vs buy.
- Volume. Above ~10k minutes/month, LiveKit wins on TCO. Below, managed wins.
- Integration depth. Does the agent need bespoke internal APIs, not standard CRMs? If yes, LiveKit.
- Latency ceiling. Is sub-500 ms perceived latency the product? If yes, LiveKit gives you the most control over each stage.
- Compliance. Regulated verticals (health, finance, legal) need auditable logging, PII redaction, BAAs. LiveKit makes that easier.
- Team. Do you have a Python team that can ship and operate async streaming code? If not, managed is safer.
Three yeses out of five is usually enough to justify building on LiveKit. At one or two, start managed, measure volume for a quarter, then revisit.
Five pitfalls that kill LiveKit voice-agent projects
Every rollback we’ve diagnosed has one or more of these. Rule them out early and you’ll ship.
- No eval suite until after launch. A silent regression in a vendor model breaks your agent overnight. Build evals in week 2.
- Over-engineering turn detection. Teams hand-tune interruption heuristics instead of using LiveKit’s turn detector and adaptive interruption handling out of the box. Use the defaults, then tune only the silence threshold per vertical.
- LLM picked for reasoning, not latency. A model with 1.2 s first-token latency blows the budget. Pick the lowest-latency model that clears the quality bar, not the smartest.
- Write tools without confirmation. An agent that can send an email or charge a card without a readback is a liability incident waiting to happen. Always confirm.
- Compliance deferred to legal. AI disclosure, recording consent, and PII logging are engineering work. If legal owns them, they ship after the rollback.
Planning to ship a voice feature inside a mobile app? App Store review adds another pitfall: since November 2025, Apple’s guideline 5.1.2(i) requires apps to tell users and get their permission before sending personal data, voice included, to a third-party AI service. Our AI video analytics case study covers how we handled similar disclosure patterns for an e-learning product.
KPIs that tell you the agent is working
Watch these weekly. If the leading KPIs trend right and the lagging ones don’t, the agent is a toy, not a product.
| Category | Leading KPI | Lagging KPI |
|---|---|---|
| Performance | ASR-to-first-audio p95 | Call abandonment rate |
| Quality | Eval pass-rate (100 scenarios) | CSAT delta vs human baseline |
| Containment | Tool calls per call | Human-handoff rate |
| Cost | Cost per call (rolling 7-day) | Cost per resolved incident |
| Safety | Hallucination flag rate | Customer complaint ratio |
When to not build a LiveKit voice agent
LiveKit voice agents are powerful, but they’re not the answer to every customer-interaction problem. Don’t build if:
- Your call volume is under 500 minutes/month. A managed platform or a better FAQ page ships faster.
- Your callers are mostly elderly or unfamiliar with voice AI — without clear signposting they’ll mistrust the interaction.
- Every call needs a human empathetic touch (grief counselling, crisis support). Voice AI can hurt more than help.
- You already have a well-adopted chat channel with sub-10-second human response. A voice agent solves a latency problem you don’t have.
- Your legal team hasn’t signed off on AI disclosure, consent, or recording retention for your jurisdictions. Build that first.
Saying no to a voice-agent project with evidence is a better outcome than saying yes and rolling back six months later.
FAQ
How long does it take to build a first LiveKit voice agent?
For a well-scoped tier-1 support agent with CRM tool-use and telephony, 4–6 weeks from kickoff to first live call with a senior two-engineer team. Add 2–3 weeks for regulated verticals that need BAAs or PCI pauses.
Can LiveKit agents handle multiple languages?
Yes, with caveats. English, Spanish, French, German, and Mandarin have mature ASR and TTS. LiveKit’s turn detector covers 14 languages; others work but with higher latency and weaker turn detection. Code-switching mid-call (Spanglish) is still fragile in 2026.
Does a LiveKit agent work for outbound calls too?
Yes. LiveKit SIP supports outbound INVITE from the agent worker. Just watch TCPA and GDPR rules for unsolicited outbound — the regulatory exposure is much higher than inbound.
What’s the cheapest way to experiment before committing?
Start on LiveKit Cloud’s free Build plan (1,000 agent session minutes, 5 concurrent sessions and one free US number as of October 2026), run the agent quickstart in Python, and point your own phone at it via a cheap SIP provider. Two days of tinkering tells you more than a month of PRD writing.
Do we need GPUs to run LiveKit agents?
No, unless you self-host the ASR or TTS models. The agent worker is CPU-bound; the heavy inference happens at your ASR/LLM/TTS vendors. Self-hosting Whisper or a TTS model is the only case that needs a GPU.
How do we stop the agent from hallucinating about our product?
Three layers: a tight system prompt with an explicit “say I don’t know” fallback, RAG over product docs surfaced as a tool, and an eval suite that specifically tests for fabrication. Don’t rely on any single one.
Is LiveKit open source? Can we self-host everything?
Yes. LiveKit Server and LiveKit Agents are Apache 2.0; LiveKit Cloud is the paid managed version. You can run the whole stack on your own Kubernetes cluster, but budget 3–4 weeks of infra work just to reach parity with Cloud.
How does the EU AI Act affect our voice-agent rollout?
Since 2 August 2026, if EU users are in scope, Article 50 transparency requires you to disclose the caller is talking to an AI unless that is already obvious. Interaction logs and provider documentation are not Article 50 duties, but enterprise buyers ask for them and they become mandatory for high-risk use cases. In the mini-case below, disclosure plus the recording and audit pipeline fit into one week of a six-week pilot.
Can you use Claude Code to build LiveKit agents?
Yes. Add the LiveKit Docs MCP server (https://docs.livekit.io/mcp) and LiveKit’s Agent Skills to Claude Code, start from a starter project with its AGENTS.md, and let the coding agent write tools plus text-mode tests. Keep a senior engineer on latency, interruption tuning and compliance.
LiveKit Agents or Retell AI for a local business?
Retell AI for a single location with a few thousand minutes a month or less: $0.055/min infrastructure, 20 concurrent calls included, no code. LiveKit Agents once you run many locations, need in-house integrations, or want the audio and transcripts in your own cloud.
Does LiveKit work with Azure?
Yes. Use the Azure plugin for Azure AI Speech STT and TTS, Azure OpenAI through the OpenAI plugin, and self-host the LiveKit server on AKS if data must stay in your Azure region.
How do you test and evaluate a LiveKit agent?
Use LiveKit’s built-in tests to assert on messages, tool calls and handoffs in pytest or Vitest, then run Agent Simulations with a simulated user in CI. Add a nightly replay of 100–500 real scenarios graded against a rubric.
Which AI avatar vendors work with LiveKit voice agents?
Tavus, Hedra, Beyond Presence, Simli, Anam and bitHuman have LiveKit avatar plugins that publish a lip-synced video track into the room, so the voice-agent logic stays the same.
Can you build a LiveKit voice widget in plain JavaScript, without React?
Yes. The livekit-client JavaScript SDK is framework-agnostic: connect to a room, publish the microphone and play the agent’s audio track from vanilla JS. React components are optional helpers.
What to read next
Voice AI
Build voice AI that sounds human with LiveKit
The deeper-dive companion: prompt shapes, turn handling, and voice-agent UX.
Frameworks
Pipecat vs LiveKit Agents
Open-source, head-to-head: which orchestration framework fits your build.
Costs
What the OpenAI Realtime API really costs
The per-minute math behind the speech-to-speech option.
Pricing
LiveKit vs Agora: 2026 cost analysis
Where each real-time platform wins on price at your scale.
Real-time
What is WebRTC
The transport layer under LiveKit, in one primer.
Ship your first LiveKit voice agent in 2026
Voice AI in 2026 isn’t a research preview — it’s a deployable feature with known unit economics, proven vendors, and transparency rules that are already in force. The gap between teams shipping it and teams still evaluating already shows up in cost-per-call, CSAT, and how fast you can staff support. The fastest way across that gap is a 6-week pilot on real traffic, not a six-month PRD.
If you want help scoping yours, we’ve shipped LiveKit voice agents for support, outbound, in-app, and regulated verticals since the framework’s 0.x days. We’ll bring the scorecard, the stack choices, and realistic estimates to the first meeting.
The book · Volume 7 of 7
AI for Video: A Complete Guide to Intelligence in Video Products: From Real-Time Pipelines and Agents to Serving, Cost, Compliance, and Verticals
This playbook takes one LiveKit voice agent from a latency budget to a working phone number. Volume 7 goes under the hood of the same system: where AI sits in a WebRTC pipeline, agent loops with memory and tools, serving on vLLM and TensorRT, cost per stream-hour, and what the EU AI Act asks of a voice product. It is written for the team that has to keep the agent fast and affordable long after the demo. Written by Nikolay Sapunov, CEO at Fora Soft.
Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook
Ship your first LiveKit voice agent in 6 weeks
Free 30-minute scoping call with a Fora Soft senior engineer. You’ll leave with a stack, a per-minute cost estimate, and a realistic timeline.