
Key takeaways
• Live real-time translation in teleconferencing is now an end-to-end pipeline of four stages: ASR (speech-to-text), MT (machine translation), TTS (text-to-speech), and optional voice cloning. Target end-to-end latency is < 800 ms for captions and 2–3 s for translated voice.
• Two production lanes now exist. The cascaded stack (Deepgram Nova-3 + DeepL + ElevenLabs Flash) is still the default when you need glossaries, per-stage debugging and data-residency control. Since May–June 2026 single-model speech-to-speech APIs are production options too: OpenAI gpt-realtime-translate (70+ input, 13 output languages, $0.034/min) and Gemini 3.5 Live Translate (70+ languages, public preview).
• Realistic per-minute cost is ~$0.08–$0.10 for cascaded voice. At October 2026 list prices: $0.009/min ASR + ~$0.025/min MT + ~$0.045/min TTS = $0.079/min ($4.74/hour) per language pair, plus 20–30% for retries and idle stream time. Captions-only runs ~$0.034/min.
• WebRTC architecture choice changes everything. SFU + server-side translation worker is the production-grade pattern. P2P breaks at >3 participants; MCU adds transcoding lag. We covered this in detail in our P2P vs MCU vs SFU piece.
• Build vs buy break-even sits around 2,000 paid calls/month. Below that, off-the-shelf KUDO, Interprefy, Wordly or Zoom AI Companion is cheaper. Above that — or with telehealth, courtroom, broadcast or eSports verticals where data residency or domain glossaries matter — custom WebRTC + translation usually wins.
Updated October 2026: speech-to-speech APIs from OpenAI, Google and DeepL, native voice interpretation in Zoom and Teams (Meet in preview), and a re-priced cost model.
More on this topic: read our complete guide — 7 Best Video Call Translation Tools Compared (2026).
Why Fora Soft wrote this real-time translation playbook
We’ve been building video and AI products since 2005 — 250+ projects across those 21 years. One of them is Translinguist — a custom AI-driven live event interpretation platform that combines streaming ASR, real-time MT, multilingual TTS and human interpreter fallback for events where Zoom’s default translated captions don’t cut it.
This playbook is the buyer-and-builder guide we wish more product teams had on day one. We compare the 2026 ASR / MT / TTS stack honestly with WER, BLEU/COMET, latency and price. We walk the WebRTC architecture for piping translated audio back into the call. We give the cost model at scale, the compliance traps for HIPAA / GDPR / FERPA, and the build-vs-buy line. And we end with a five-question framework for picking the right path.
If your product is a webinar platform, telehealth tool, e-learning service, courtroom-recording stack or eSports broadcaster, the rest of this article is for you.
Adding real-time translation to your video product?
A 30-minute scoping call gives you a vendor-neutral architecture, a 2026-grade stack, and a budget for translated captions or full voice translation.
What live real-time translation in teleconferencing actually is
Live real-time translation is the live conversion of one speaker’s audio into another language — as captions, as a translated voice, or as both — with low enough latency that listeners can follow the conversation as it happens. It used to require a human interpreter and an isolation booth. In 2026 it’s a software pipeline you ship inside your video conferencing product.
Two delivery modes coexist. Translated captions show the listener written subtitles in their language; latency target is under 800 ms and accuracy is mostly an ASR + MT problem. Translated voice renders the speaker’s sentences as spoken audio in the listener’s language, optionally in a cloned voice; latency target is 2–3 s end-to-end and the chain adds TTS plus voice-cloning quality risks. Most production deployments ship both, and let the listener pick.

Figure 1. The four-stage cascaded pipeline — ASR → MT → TTS → optional voice cloning, feeding captions and dubbed voice.
Can audio translators translate in real time? What changed in 2026
Yes. In 2026 AI audio translators translate speech in real time with a delay of one to two sentences: a few seconds for translated voice and under a second for translated captions. The change this year is that one model can now do the whole speech-to-speech step, and the three big meeting platforms ship it natively.
| Product | Where you get it | Launched | Languages | Price |
|---|---|---|---|---|
| OpenAI gpt-realtime-translate | API (WebRTC / WebSocket) | May 7, 2026 | 70+ in → 13 out | $0.034/min |
| Gemini 3.5 Live Translate | Gemini Live API (public preview), Meet (private preview) | June 9, 2026 | 70+ | Not published for the API |
| DeepL Voice-to-Voice | Voice for Meetings (Teams, Zoom), API early access | April 17, 2026 | 40+ | Enterprise quote |
| Microsoft Teams Interpreter | Teams meetings and 1:1 VoIP calls | GA; docs updated Sept 2026 | 9 | Copilot licence, 20 h/user/month included |
| Zoom Voice Translator | Zoom desktop app 7.0.0+, Pro/Business/Enterprise | July 22, 2026 | 5 at launch, 4 more planned in 2026 | ZoomMate or Live Translation add-on |
What it means if you build a video product. You no longer have to assemble ASR, MT and TTS just to get translated voice. OpenAI gpt-realtime-translate accepts a WebRTC offer and returns translated audio on a remote track plus transcripts on a data channel (OpenAI cookbook, May 2026), so an SFU can forward it like any other participant. The limits are real, though: 13 output languages, no custom glossary prompt, no voice selection, and your audio goes to the vendor. Those limits are why the cascaded pipeline below is still the default for medical, legal and EU-resident workloads.
Accuracy and limits. Expect the familiar weak spots: names and numbers, domain jargon, overlapping speakers, noisy rooms and low-resource languages. DeepL’s own April 2026 demo ran one to two sentences behind the speaker because word order differs between languages. That delay is built into the task and will shrink but not vanish.
The four-stage pipeline — ASR → MT → TTS → voice cloning
A real-time translation pipeline is a chain. Each stage adds latency and accuracy risk; you optimise the chain, not the components.
1. ASR (speech-to-text). Streams the speaker’s audio into partial transcripts. Modern engines (Deepgram Nova-3 and Flux, AssemblyAI Universal-Streaming, OpenAI gpt-realtime-whisper) deliver 100–500 ms latency on streaming endpoints. Word error rate (WER) is the headline accuracy metric — 5–7% on conversational English is the 2026 frontier.
2. MT (machine translation). Converts the transcript into the target language. Cascaded MT (DeepL, Google Cloud Translation, Azure Translator, NLLB-200) typically adds 100–200 ms per sentence. Speech-to-speech models skip ASR + MT + TTS altogether: research models like Meta’s streaming M4T research model, and since May–June 2026 commercial APIs (OpenAI gpt-realtime-translate, Gemini 3.5 Live Translate) that cover 13–70+ languages.
3. TTS (text-to-speech). Renders the translated text as audio. ElevenLabs (Flash v2.5 at ~75 ms first audio, Eleven v3 for expressive voices), Cartesia Sonic, Microsoft Azure Neural TTS and Resemble AI cover the production-quality range. The trade-off is naturalness vs latency — expressive voices add 200–400 ms.
4. Voice cloning (optional). Renders TTS in the original speaker’s voice. ElevenLabs and Azure Personal Voice ship this as APIs, and Microsoft Teams Interpreter simulates the speaker’s own voice by default (2026), without storing voice samples. It’s the quality multiplier that makes a translated meeting feel like the speaker really learned your language. Compliance traps are real: HIPAA and GDPR both treat voice biometrics as sensitive data.
Reach for cascaded ASR → MT → TTS when: you need 100–120 language pairs, debuggability per stage, custom domain glossaries, and the freedom to swap one component without rebuilding the pipeline. Reach for a speech-to-speech API when your languages are on its list, you don’t need glossaries, and sending audio to the vendor is acceptable.
The 2026 ASR engines — what to ship into production
Numbers below are 2025–2026 vendor-published or independently benchmarked; prices are streaming list rates checked in October 2026. Re-test on your own audio before you commit; conversational accents, code-switching and background noise change the rankings.
| ASR engine | WER (conv.) | Streaming latency | Languages | Price/min | Use case |
|---|---|---|---|---|---|
| Deepgram Nova-3 | 5.3–6.8% | ~450 ms | 40+ | $0.0092 | General real-time |
| OpenAI gpt-realtime-whisper | Not published | Streaming | Multilingual | $0.017 | One vendor with Realtime |
| AssemblyAI Universal-Streaming | Vendor-reported | ~300–600 ms | English + multilingual | $0.0025 (Universal-3.6 Pro: $0.0075) | Cost-sensitive, alphanumeric |
| Google Cloud STT | 7–10% | ~500 ms | 125+ | $0.016 / vol $0.004 | GCP estates |
| Azure Speech | 7–9% | ~500 ms | 100+ | $0.012 | Microsoft estates, HIPAA |
| NVIDIA Riva (on-prem) | 8–10% | ~300 ms | 12 core + custom | GPU capex | Air-gapped, regulated |
Deepgram’s $0.0092/min is the Nova-3 multilingual list rate; it shows $0.0058 as a limited-time promo, and English-only Nova-3 lists at $0.0077 (October 2026). For more on the ASR options we put together a deeper review of speech-recognition platforms, and our real-time speech translation vendor benchmarks compare end-to-end delay.
The 2026 MT engines — what translates well and fast
DeepL. Quality leader on European languages by COMET and human-judgement studies; 30+ languages. Sentence-by-sentence streaming, ~150–200 ms. API usage is about $25–$28 per million characters on 2026 plans (≈ $0.0025–$0.0028 per 100 characters). In April 2026 DeepL also launched Voice-to-Voice: 40+ languages, Voice for Meetings in Teams and Zoom, and an API in early access. The default for general business meetings in EMEA.
Google Cloud Translation v3. 135 languages. Custom AutoML models. Basic NMT is $20 per million characters with 500,000 characters free each month (2026). Wins on language coverage and AutoML domain customisation.
Azure Translator. 100+ languages, deep Microsoft Teams integration, HIPAA/GDPR-attested. The natural pick if you’re embedding translation inside a Teams-native or healthcare workflow.
Meta NLLB-200 + M4T v2. 200 languages in NLLB-200, multimodal text+speech in M4T v2. Open weights, deployable on-prem, but released under non-commercial licences, so clear the licence before you ship them in a paid product. BLEU lags DeepL by 5–10 points on European pairs but wins on low-resource languages and on workloads that can’t leave a private cloud.
LLMs (GPT, Claude, Gemini) for translation. LLMs translate fluently and handle code-switching beautifully, but they introduce hallucination risk on names and numbers and add 300–800 ms on the API call. Google’s LLM Translation lists at $10 per million input and $10 per million output characters (2026). Good for high-stakes single-pass translation, less good for hot streaming paths.
TTS and voice cloning — the make-or-break stage for translated voice
Listeners forgive a translated caption that’s slightly clunky; they don’t forgive a translated voice that sounds robotic, mid-sentence, or detached from the speaker. Pick TTS like you’re picking the voice of your product.
ElevenLabs Eleven v3 + Flash v2.5. Flash first-audio latency ~75 ms (vendor figure), expressive v3 voices, 30+ languages, voice cloning from a few minutes of source audio. API cost is about $0.05 per 1,000 characters for Flash and $0.10 for v3 (August 2026). The 2026 production default for most video products.
Microsoft Azure Neural TTS & Personal Voice. Wider language coverage, HIPAA/GDPR posture out of the box, voice cloning gated behind responsible-AI consent flow. Slightly higher latency (~200 ms first-token) but enterprise-friendly.
Cartesia Sonic, Resemble AI, open-source. Cartesia claims sub-90 ms; Vapi measured a 166 ms median in June 2026. Resemble AI is a credible commercial alternative with on-prem options. PlayHT is gone: Meta acquired PlayAI in July 2025 and the platform shut down on December 31, 2025, so migrate any PlayHT voices. Open-source models can be self-hosted by teams who can’t use cloud TTS at all; check each model’s licence before commercial use.
For more on how voice cloning works under the hood and what it costs in production, see our voice cloning & synthesis guide.
WebRTC architecture for live translated audio — the production pattern
A translation pipeline is useless if the WebRTC layer can’t pipe the result back into the call without jitter. The 2026 production pattern looks like this:
1. SFU not MCU. An SFU (Selective Forwarding Unit) — the core of any modern video conferencing build — gives every client a fan-out of the original media plus the option of additional translated audio tracks. An MCU adds transcoding latency and CPU cost that real-time translation can’t afford.
2. Server-side translation worker. The SFU forks a copy of the active speaker’s audio to a translation worker that runs ASR, MT and TTS, or holds one speech-to-speech API session per target language. Captions return over a WebRTC data channel; translated voice returns as an additional audio track that the listener client can swap in for the original. This is the part our custom WebRTC architecture team designs first, because track routing decides your latency budget.
3. VAD + speaker labelling. Voice-activity detection chunks the speech into translatable segments without breaking sentences. Speaker labels (diarisation) keep the right voice attached to the right cloned TTS profile when speakers swap mid-call.
4. Latency budget & jitter buffer. Aim for < 800 ms total for captions and about 2–3 s for translated voice. The jitter buffer needs to be tunable per-listener — deaf or hard-of-hearing users on captions only can run with a tighter buffer; voice-mode listeners need a slightly bigger one to absorb TTS variance.

Figure 2. SFU + server-side translation worker — the 2026 production architecture for piping translated audio back into the call.
Off-the-shelf platforms — what Zoom, Teams, KUDO, Interprefy and Wordly actually deliver
Zoom Voice Translator and translated captions. Zoom launched spoken translation on July 22, 2026: English, Chinese, Spanish, French and Japanese at launch, with Portuguese, Arabic, German and Italian planned for later in 2026. It runs in the desktop app 7.0.0+ on Pro, Business and Enterprise accounts, via ZoomMate or a Live Translation add-on. Translated captions remain on paid plans. Good for general business meetings; weak on domain glossaries.
Microsoft Teams Interpreter. Speech-to-speech interpretation in 9 languages (Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Spanish) in meetings and 1:1 Teams VoIP calls, not in Teams events. It needs a Microsoft 365 Copilot licence and includes 20 hours per user per month; listeners can hear the speaker’s simulated voice. Translated live captions are available too. Best fit if you’re already in the Microsoft 365 estate.
Google Meet speech translation. Started with five languages paired with English. Gemini 3.5 Live Translate (June 9, 2026) brings 70+ languages and any-to-any pairs without an English pivot; Meet access is in private preview for business Workspace customers, with wider rollout later in 2026. Translated captions cover the major languages. The most consumer-friendly default; thinner on enterprise controls.
KUDO AI & Interprefy AI. Specialised event-grade translation platforms with hybrid AI + human-interpreter workflows. Per-event pricing; meant for conferences, governmental meetings, regulated webinars. Strong in language coverage, weak as a developer SDK.
Wordly, Verbit, Maestra, Akkadu, Translanguage. Mid-market simultaneous interpretation as a service, priced per event or per attendee-hour; June 2026 coverage put AI-only interpretation at $8–$35 per attendee-hour versus $60–$200 per human interpreter-hour. Useful at the event tier; rarely embeddable into a third-party product. For a side-by-side of the ready-made tools, see our comparison of real-time meeting translation platforms.
Off-the-shelf doesn’t fit your domain?
We built Translinguist for clients whose translation needs Zoom couldn’t cover. We can build the same for you in 12–16 weeks.
Cost model — what live translation actually costs at scale
A simple back-of-envelope per minute of translated audio at October 2026 list prices, assuming ~150 words or ~900 characters of speech per minute:
- ASR: Deepgram Nova-3 multilingual streaming at $0.0092/min
- MT: DeepL at ≈ $0.0028 per 100 chars ≈ $0.025/min
- TTS: ElevenLabs Flash at ≈ $0.05 per 1,000 chars ≈ $0.045/min
- Subtotal: $0.079/min ≈ $4.74/hour of audio per language pair
- Add 20–30% for VAD retries, idle stream time, language detection, dropped frames — real cost lands at $0.095–$0.10/min
Translated captions only (no TTS) come in at ~$0.034/min. Voice + cloning is the same as the cascaded cost above plus an enrollment cost per cloned voice. A speech-to-speech API such as OpenAI gpt-realtime-translate is $0.034/min for voice and transcript together (May 2026), which is less than half the cascaded voice cost if its 13 output languages and lack of glossaries fit. Multiply by listener language count: a meeting with 4 active translation languages costs 4× the per-minute number.

Figure 3. Per-minute cost stack — ASR + MT + TTS = $0.079, near $0.10 with a real-world buffer; a speech-to-speech API costs $0.034.
Reach for white-label off-the-shelf when: you have < 2,000 translated calls/month, you operate in < 4 language pairs, and your listeners tolerate generic TTS voices. Below that scale the per-call savings of building your own pipeline don’t cover the engineering investment.
Compliance & data residency — HIPAA, GDPR, FERPA, AI Act
Real-time translation routes raw voice and translated text through third-party APIs. That alone breaks several common compliance regimes unless you architect carefully.
HIPAA (US healthcare). Voice of a patient is PHI. Most consumer ASR/MT/TTS APIs aren’t HIPAA-eligible without a signed BAA. The shortlist that signs BAAs in 2026: Microsoft Azure (Speech, Translator, Neural TTS), Google Cloud (STT, Translation), AWS (Transcribe, Translate, Polly), Deepgram (with enterprise plan). Open-source self-hosting on a HIPAA-aligned cloud is the more conservative pattern for telehealth.
GDPR (EU). Voice used to clone or identify a speaker is biometric data, a special category under GDPR Article 9. EU-region processing is the safe default; the EU Data Boundary commitments from Microsoft and Google help, but read the fine print. Auto-delete of audio after the call and keeping only the translated transcript is the production pattern.
FERPA (US education). Student voice falls under FERPA when stored as identifiable data. Most off-the-shelf translation platforms don’t document FERPA posture; build-your-own with auto-delete and on-prem ASR is the safer route for K-12 and US higher-ed.
EU AI Act. Article 50 transparency obligations apply from August 2, 2026: people must be told when they hear AI-generated or manipulated audio, so a translated voice needs a visible “AI voice” label. The AI Omnibus, adopted by the Council on June 29, 2026, gives systems already on the market until December 2, 2026 for machine-readable marking under Article 50(2); new systems get no grace period. Bake this into the listener UI, not the legal small print.
Mini case — Translinguist, the live event interpretation platform we built
Situation. An interpretation client needed real-time interpreting for international events where off-the-shelf platforms fell short — broad language coverage, a marketplace of certified human interpreters as fallback, closed captioning in dozens of languages, and a transcript-and-dubbed-audio archive that holds up in regulated settings.
What we built. Translinguist — a WebRTC platform (MediaSoup SFU, RabbitMQ, Node.js, React, Deepgram and Google STT) that combines AI speech-to-speech translation in 16+ languages, closed captioning in 22 languages, a marketplace of 30,000+ certified interpreters for human fallback, plus recording, transcription, a sign-language UI and Zoom / Meet / Teams integration. It supports 75+ languages end to end and is part of our custom AI language-interpretation work.
Outcome. Translinguist won the NHS (NOE CPC) National Framework for Language Services across the UK and was featured twice in Slator as a disruptive remote-interpreting player. Clients report 50% cost savings, an 80% reduction in interpreting costs, 30% higher engagement, 53% higher attendance and 2× ROI within the first two years.

Figure 4. Translinguist in action — live translated captions per language and a sign-language interpreter tile in one meeting.
Build vs buy — when custom translation pays back
Buy off-the-shelf when: < 2,000 translated calls/month, < 4 language pairs, generic content, no domain glossary, listeners tolerate generic voices, and your data-residency story can live with a US/EU cloud vendor.
Build a custom WebRTC + translation pipeline when: you operate > 2,000 calls/month, you need domain glossaries (legal, medical, financial, technical), data residency is a hard constraint (HIPAA, FERPA, EU on-prem), the listener UX has to feel native to your product (your own caption styling, your own voice swap UI), or your business model is selling translation as a feature, not consuming it.
Vertical anchors that force custom every time: courtroom interpretation, telehealth voice dubbing, eSports live broadcast, religious-service interpretation, simultaneous interpretation for live events with regulated speakers, K-12 / FERPA classroom translation. We’ve built or scoped each of these in the last three years.
Worked break-even (illustrative). 2,000 calls a month × 45 minutes × 2 target languages = 180,000 translated minutes. On your own pipeline that is about $18,000 a month at $0.10/min cascaded, or about $6,100 on a $0.034/min speech-to-speech API. A platform billed per listener-hour at $8–$35 (June 2026 market range), with 3 translated listeners per call, costs $36,000–$157,500 for the same month. The gap pays back a $90–$160K build within a year even after hosting, on-call and model upgrades; at a few hundred calls a month it doesn’t.
Honest cost shape. A custom WebRTC + cascaded translation MVP runs from roughly $90–$160K and 12–16 weeks with our team using Agent Engineering acceleration; comparable off-the-shelf integrators tend to quote $250K+ and 6–9 months. Where the scope demands voice cloning, low-resource languages, or full HIPAA/GDPR posture, we scope discovery first instead of guessing at totals.

Figure 5. Buy-vs-build at a glance — the same axes point off-the-shelf below ~2,000 calls/month and custom above it.
Decision framework — pick a translation strategy in five questions
Q1. Captions only or full translated voice? Captions deliver most of the value at under half the cost ($0.034 vs $0.079 per minute) and avoid synthetic-voice disclosure duties. Voice is the conference-grade experience; budget the extra latency, cost and compliance work.
Q2. How many language pairs in the same call? 1–3 pairs → off-the-shelf is fine. 4–10 pairs concurrently → you’re past the sweet spot of every consumer platform; plan a custom SFU with parallel translation workers.
Q3. What’s the latency budget for your audience? Telehealth + 1:1 conversations → tight (about 2 s voice, captions first). Conference broadcast → relaxed (3–5 s voice acceptable). Recorded webinars → minutes are fine.
Q4. What compliance regime are you in? HIPAA, FERPA, GDPR, MAS, OSFI, SOC 2, AI Act — each adds non-negotiable architecture constraints. Pick the engine vendors that sign the right paper.
Q5. Is translation a product feature or a hidden infrastructure piece? If it’s a product feature you market, custom is almost always the answer. If it’s a hidden infrastructure piece nobody notices, off-the-shelf is fine.
Five pitfalls we see in almost every live translation rollout
1. Optimising for one stage, not the chain. Teams chase Whisper-grade ASR accuracy then plug it into a slow MT and a 400 ms TTS — net latency 4 s, listeners drop. Fix: budget the full pipeline up front and benchmark end-to-end, not stage-by-stage.
2. Ignoring domain vocabulary. Generic ASR misses "tachycardia," "anti-money-laundering," "sovereign immunity" or game-specific terms. WER explodes from 6% to 25% on real content. Fix: per-event glossaries, custom vocabulary, fine-tuned acoustic models for the verticals that need it.
3. Hallucinated names and numbers. LLM-based MT happily invents plausible-but-wrong translations of proper nouns, dates, dosages and dollar amounts. Fix: pre-extract entities and pin them through translation; ground numerics with a separate parsing pass.
4. Voice cloning without consent UX. EU AI Act and several US state laws now require explicit notice when listeners hear an AI-generated voice. Fix: visible "AI voice" badge, opt-out toggle, recording disclosure, signed enrollment for any cloned speaker.
5. Forgetting accessibility for the source language. Translation is for cross-language listeners; live captions in the source language are for hearing-impaired listeners. They’re different features but the same engine. Fix: ship both from day one.
KPIs — how to measure that real-time translation actually works
Quality KPIs. WER on conversational audio (target ≤ 8% for English, ≤ 12% for non-English mainstream); BLEU/COMET against reference translations on rehearsed content (target ≥ 35 BLEU on European pairs); MOS naturalness on TTS output (target ≥ 4.0/5); listener-reported intelligibility on a 5-point scale (target ≥ 4).
Latency KPIs. Caption latency p50 / p95 (< 800 ms / < 1.5 s); translated voice latency p50 / p95 (< 2 s / < 3.5 s); jitter buffer underrun rate (< 1% of seconds); sentence-completion delay (the “wait-k” trade-off, target k = 3 words).
Reliability KPIs. Translation worker uptime (≥ 99.9%); language-pair availability (every supported pair ≥ 99% week-over-week); fallback-to-human-interpreter rate when AI confidence drops (≤ 5% on rehearsed content); audio packet loss compensation success rate (≥ 98%).
When NOT to add live translation
Don’t add translated voice for clinical-decision conversations where mistranslation could cause harm. Captions plus a documented human interpreter SLA is the responsible default for those calls.
Don’t add translation in 12 languages because someone asked for it on a sales call. Each language adds compute cost, support burden and quality variance — ship 2–3 strong languages first, then expand.
Don’t build a custom translation stack to save $2,000/year in vendor licences. Below ~2,000 translated calls/month, the engineering hours don’t pay back; off-the-shelf is the right answer.
Reach for hybrid AI + human interpreter when: the meeting is regulated, recorded, broadcast or contractually consequential. AI handles the bulk of the volume; humans take over the moments that can’t fail.
Want a translation pipeline that’s yours, not a third-party widget?
We’ll scope a custom WebRTC + translation MVP for your video product in 48 hours — latency, languages, compliance, cost.
Streaming protocols and end-to-end speech-to-speech models
Two architectural lanes coexist in 2026. The cascaded ASR → MT → TTS pipeline is still the default for regulated and glossary-heavy deployments. End-to-end speech-to-speech (S2S) became a commercial option this year: OpenAI gpt-realtime-translate (May 2026), Gemini 3.5 Live Translate (June 2026) and DeepL Voice-to-Voice (April 2026, API in early access). Meta’s streaming M4T research model and Google’s Translatotron line remain research models.
Cascaded streaming wins on debuggability. When a translated line goes wrong you can isolate the failure to ASR, MT or TTS. Custom glossaries, per-event vocabulary, and language-pair-specific tuning all live on the cascaded pipeline. Cost per minute is also better understood and easier to budget.
End-to-end S2S wins on prosody and simplicity. S2S preserves emotion, emphasis and speaker style across languages in ways that cascaded TTS struggles with, and it removes two network hops. OpenAI’s model follows the speaker’s tone and pitch but offers no voice choice or custom prompt; Meta’s streaming M4T research model reaches roughly 100 input and 36 output speech languages at about 2 s delay.
WebRTC delivery options. Translated voice lands as either a second audio track on the existing peer connection (cleanest) or via a dedicated channel (older clients). Captions ride a WebRTC data channel, so they skip the audio jitter buffer entirely. SRT or HLS are options for one-to-many video streaming scenarios where direct WebRTC isn’t viable.
Reach for end-to-end speech-to-speech when: your language pairs are on the API’s list, prosody matters more than glossaries, you can tolerate a one-sentence delay, and your compliance regime allows audio to reach the vendor. For everything else, cascaded is still the answer.
FAQ
What is live real-time translation in teleconferencing?
It’s the live conversion of one speaker’s voice into another language — either as captions, as translated voice, or both — with low enough latency that participants follow the conversation as it happens. The pipeline is ASR + MT + TTS + optional voice cloning, with target latency < 800 ms for captions and 2–3 s for translated voice.
Can audio translators translate in real time?
Yes. In 2026 AI translators turn speech into translated captions in under a second and into translated voice one to two sentences behind the speaker. Accuracy is good on mainstream languages and clear audio, and weakest on names, numbers, jargon and overlapping speakers.
Do Zoom, Teams and Google Meet have live voice translation?
Zoom and Teams do; Google Meet is in preview. Zoom Voice Translator launched July 22, 2026 with 5 languages; Microsoft Teams Interpreter covers 9 languages with a Copilot licence; Google Meet is rolling out Gemini 3.5 Live Translate (70+ languages) from a private preview that started in June 2026.
How do I integrate real-time translation into my video conferencing software?
Fork the active speaker’s audio from your SFU to a translation worker, run either a cascaded ASR → MT → TTS chain or a speech-to-speech API session per target language, and send captions back over a data channel and translated voice as an extra audio track. Budget under 800 ms for captions and 2–3 s for voice.
Which ASR engine should I pick in 2026?
Deepgram Nova-3 is the balanced default (~450 ms streaming) at $0.0092/min multilingual list price (October 2026). AssemblyAI Universal-Streaming is the low-cost option at $0.0025/min. OpenAI gpt-realtime-whisper ($0.017/min) fits teams already on OpenAI Realtime. Azure Speech wins on enterprise/HIPAA posture. Pick by your dominant language, latency budget and compliance regime.
DeepL vs Google vs Azure for live machine translation?
DeepL leads on COMET-graded quality for European languages and adds ~150–200 ms; Google Translation v3 leads on language coverage (135) and AutoML; Azure Translator wins inside Microsoft 365 estates. NLLB-200 + M4T are the open-source on-prem option for regulated workloads.
What latency is acceptable for live translation?
Below 800 ms feels real-time for captions; 2–3 s for translated voice keeps conversational flow; above 4 s breaks turn-taking. Target p95 caption latency < 1.5 s and p95 voice latency < 3.5 s in production.
How much does live translation cost per minute?
Cascaded ASR + MT + TTS at October 2026 list prices lands at roughly $0.079/min ($4.74/hour) per language pair, or about $0.10/min with a 20–30% buffer. Captions-only is ~$0.034/min. OpenAI gpt-realtime-translate charges $0.034/min for speech-to-speech. Multiply by simultaneous listener language count.
When should I build vs buy a translation feature?
Buy off-the-shelf below ~2,000 translated calls/month with < 4 language pairs and generic content. Build custom above that, or when you need domain glossaries, data residency (HIPAA / FERPA / EU on-prem), product-native UX, or voice cloning on regulated content.
Is end-to-end speech-to-speech translation production-ready in 2026?
Partly. Since May–June 2026 OpenAI gpt-realtime-translate ($0.034/min, 13 output languages) and Gemini 3.5 Live Translate (public preview, 70+ languages) are available to developers; OpenAI’s is in the API today, Gemini’s is still a public preview. Cascaded ASR + MT + TTS still wins when you need custom glossaries, per-stage debugging, 100+ language pairs or audio that stays in your own cloud.
How do I make live translation HIPAA-compliant?
Use ASR/MT/TTS vendors that sign HIPAA BAAs (Azure, GCP, AWS Transcribe/Translate/Polly, Deepgram Enterprise). Auto-delete raw audio after the call; keep only translated transcripts. Run voice cloning only with documented consent. Or self-host open-source ASR/MT (Whisper, NLLB, a self-hosted TTS model) on your HIPAA-aligned infrastructure.
What to Read Next
Voice cloning
Voice cloning & synthesis: ultimate guide
How modern voice cloning works in production — quality, cost, compliance.
ASR engines
Top AI speech-recognition software
A deeper review of the ASR engines we shortlist for live translation.
WebRTC
P2P vs MCU vs SFU
The architecture choice that decides whether translation latency works.
Integration guide
Video call with translator: WebRTC integration
A practical walk-through of wiring translation into a WebRTC stack.
Ready to ship live translation in your video product?
Live real-time translation in teleconferencing is no longer the unobtainable feature it was three years ago. The 2026 stack ships with sub-second captions, sub-3-second translated voice, voice cloning that fits the speaker, and a per-minute cost that earns its place in your pricing. The decisions left are which ASR / MT / TTS to combine, which compliance regime to design around, and whether to wrap it in an SFU you own or a third-party widget you rent.
If you’re anywhere on that decision — from "we want translated captions next quarter" to "we’re building the next Translinguist" — we’ve done the work many times. Bring the spec; we’ll bring the architecture, the per-minute math, and a 12–16 week MVP plan.
The book · Volume 7 of 7
AI for Video: A Complete Guide to Intelligence in Video Products: From Real-Time Pipelines and Agents to Serving, Cost, Compliance, and Verticals
This guide maps the ASR, MT and TTS pipeline for a translated call; Volume 7 picks up where it stops, with real-time budgets in production. It covers call latency and where AI plugs into a WebRTC pipeline, live captions and translation, serving on vLLM and Triton, cost per stream-hour and EU AI Act obligations. Written by Nikolay Sapunov, CEO at Fora Soft.
Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook
Talk to the team that built Translinguist
30 minutes with a Fora Soft solutions architect — vendor-neutral, latency-honest, compliance-aware.