End-to-end encryption (E2EE) means that only the endpoints of a conversation — the participants' own devices — can decrypt the media. No server in the path can read it, and that includes your own infrastructure: a Selective Forwarding Unit (SFU), the media server that routes video between participants, sees only ciphertext. In WebRTC, the browser-based real-time media stack, transport is already encrypted hop-by-hop with DTLS-SRTP; E2EE is layered on top of that, typically using insertable frame encryption or a group-key scheme based on Messaging Layer Security (MLS), so the keys never reach the server.
The reason this matters to a telemedicine team is that E2EE collides directly with features that assume the server can read media. Server-side recording, live captions, AI scribes, and automated quality monitoring all depend on decrypting the stream centrally — under true E2EE they break, or must be redesigned to run at the endpoints, which is harder and limits scale. So E2EE is a deliberate architectural trade, not a default to switch on everywhere.
A point teams frequently get wrong: HIPAA does not require E2EE. The Security Rule's encryption requirement (45 CFR §164.312) is satisfied by strong encryption in transit and at rest, which WebRTC's DTLS-SRTP plus encrypted storage already provides. E2EE answers a stronger threat model — one where the server operator itself is not trusted to see content, such as highly sensitive behavioral-health or VIP scenarios. Choose it for that reason explicitly, and document which features you are giving up, so that a later request for server-side recording does not quietly undermine the guarantee you promised patients.
The handbook · 7 volumes
Fora Soft Video Engineering Handbook
E2EE keeps the server blind to media, which also blocks server-side recording, captions and AI scribes unless they move to the client. We wrote seven volumes covering the whole video stack, and for E2EE decisions start with Volume 3: WebRTC, SFU topologies and real-time delivery. Volume 7 covers where AI plugs into a call, Volume 6 speech recognition. Written by Nikolay Sapunov, CEO at Fora Soft.
Weighing E2EE against recording, captions and AI scribes in your calls?
We will map which features survive E2EE in your product and propose an architecture with clear trade-offs and an implementation estimate. 250+ video products shipped since 2005.
