SoundStream is Google's pioneering end-to-end neural audio codec and the basis of Lyra's later versions. It introduced, in a fully learned codec, the residual vector quantization (RVQ) bottleneck that most neural codecs now use: a stack of vector quantizers that successively refine the encoded representation, letting one model serve a range of bitrates by using more or fewer quantization stages. Trained with adversarial and reconstruction losses, it reconstructs speech and music at very low bitrates with quality that traditional codecs can't match there. SoundStream's architecture became a template — EnCodec and others followed the same RVQ-autoencoder pattern — making it a foundational work in the move from hand-engineered to learned audio compression.
The book · Volume 5 of 7
Audio for Video: A Complete Guide to Sound in Video Products: From Jitter Buffers and Lip Sync to Atmos, Quality Metrics, and Neural Codecs
SoundStream introduced the residual vector quantization design most neural codecs now follow, so it is the reference point for judging newer ones. This volume examines SoundStream in depth among neural audio codecs, with Lyra, EnCodec and quality metrics like ViSQOL and MUSHRA, so you can assess neural codecs on evidence. Written by Nikolay Sapunov, CEO at Fora Soft.
Evaluating SoundStream-style neural codecs for low-bitrate audio in your product?
Tell us your bitrate and device targets and we will benchmark SoundStream-based codecs against Opus with quality and compute numbers. 250+ video products shipped since 2005.
