MediaPipe Selfie Segmentation: Web Background Blur Guide

Updated October 2026: MediaPipe Tasks Vision 1.1.1 still runs its GPU path on WebGL2 (WebGPU belongs in your compositor); new browser matrix with WebGPU in Safari 26 and Firefox 141+, and Firefox still without MediaStreamTrackProcessor.

What Is MediaPipe Selfie Segmentation?

MediaPipe selfie segmentation is Google's on-device model that labels every pixel of a webcam frame as either person or background, so an app can blur or replace the room behind the user. In the current MediaPipe Tasks API it runs through the Image Segmenter task with the selfie_segmenter model: a 256×256 input (or 144×256 in the landscape variant), a float16 file of about 244 KB, and two output classes, 0 for background and 1 for person (Google AI Edge model table, checked October 2026). In a browser you load it from the @mediapipe/tasks-vision package, whose latest release is 1.1.1, published on npm on 14 October 2026.

Three things are easy to get wrong, so here they are up front. First, the selfie segmenter is binary; the six-class model that separates hair, body skin, face skin, clothes and accessories is a different and much heavier download. Second, MediaPipe's GPU delegate in the browser runs on WebGL2, not WebGPU; WebGPU earns its place in the compositor you write around the model. Third, the legacy Selfie Segmentation solution was folded into the Image Segmenter task on 4 April 2023, so tutorials that import the old @mediapipe/selfie_segmentation package describe an API Google has moved away from.

The latency column is Google's whole-task average on a Pixel 6, with Android pre- and post-processing included, so read it as a relative cost between models rather than as browser time. File sizes are the files Google serves for the Image Segmenter task, measured in October 2026.

Why This Matters

If your product is a video conferencing tool, a telemedicine consultation app, an asynchronous video messaging product, an online tutoring platform, or anything that puts a webcam on screen, your users expect to blur the room behind them. They expect it to work without installing software, without giving up frame rate, without paying for it, and without making the call look like a green-screen TV anchor with a fuzzy halo around their hair. Budget-wise, the model is free and the core pipeline is about 350 lines of TypeScript; the real cost is cross-browser testing across the fallback ladder and, in regulated products, the privacy paperwork. The pipeline described here belongs to the same family Google Meet uses on the web (MediaPipe running in WebAssembly, as Google Research described in 2020) and that LiveKit wraps in its open-source @livekit/track-processors package (version 0.8.2, October 2026, built on @mediapipe/tasks-vision), and it is the pattern we build into white-label conferencing tools. This article assumes you have read lesson 2.1 on computer-vision preprocessing and lesson 2.4 on SAM 2 — because background blur is a special case of semantic segmentation, and the segmentation primitives in those lessons sit at the foundation of every effect we are about to describe. By the end you will be able to specify the pipeline, pick the right rendering backend, plan the cross-browser fallback, and avoid the three mistakes that account for almost every blur-quality complaint we see in production.

What "Background Blur" Actually Is — Pixels, Not Magic

The effect that everyone calls background blur is a two-stage image processing pipeline. The first stage is semantic segmentation — for every pixel of the input frame, a small neural network emits a probability between 0 and 1 that the pixel belongs to a person rather than to the room. The second stage is compositing — using that probability mask, the renderer mixes the sharp original frame (kept for the person pixels) with a Gaussian-blurred copy of the same frame (used for the room pixels). The result looks like depth-of-field on a real camera, but no camera was involved — the blur is software, the depth is inferred, and the boundary between sharp and blurred is dictated by a 256×256 grayscale mask.

A useful analogy is rotoscoping in traditional animation. The mask is the same idea as a painter cutting a silhouette out of a sheet of paper and laying it over a different background. The neural network is the painter; the rest of the rendering pipeline is the act of laying the silhouette over a blurred copy of the room. The model is the only "intelligent" component; everything around it is plumbing.

The reason the effect feels modern is the model. Five years ago, segmenting a person from the background in real time required either a depth camera (Microsoft Kinect, iPhone TrueDepth) or a server-side GPU. The smartphone-grade mobile architectures that landed between 2019 and 2022 — MobileNetV3 and its derivatives — made it possible to run real-time segmentation on the same CPU that decodes your webcam feed. The model used by MediaPipe's selfie segmentation, and by ML Kit, and by a variant of Google Meet's blur, is one of those mobile architectures.

The MediaPipe Selfie Segmentation Model — Numbers That Matter

The MediaPipe selfie segmenter is a binary semantic segmentation network with about 106,000 parameters and a file size of 447 KB in float32 (Qualcomm AI Hub model card); the float16 file Google serves for the Image Segmenter task is about 244 KB. The architecture is a custom MobileNetV3 backbone modified with the encoder-decoder structure used in U-Net-style segmenters: the backbone produces multi-scale features that get up-sampled and combined to produce a single-channel output the same height and width as the input. The model card describes two variants: a general model that takes a 256×256 RGB input and emits a 256×256 mask, and a landscape model that takes a 144×256 input and emits a 144×256 mask. The landscape model needs fewer floating-point operations than the general one, and a variant of it powers Google Meet's background effects.

Google publishes whole-task latency for a Pixel 6: 33.5 ms on CPU and 35.2 ms on GPU for the square model, 34.2 ms and 33.6 ms for the landscape one (Google AI Edge, checked October 2026). Those figures include Android pre- and post-processing, so they are not raw model time. In a browser the cost splits differently. On a modern laptop (Apple M2, 12th-gen Intel Core i7), the WebAssembly SIMD path of @mediapipe/tasks-vision runs the square model in roughly 5–10 ms per frame in our tests, fast enough for 30 fps input. The GPU delegate, which runs on WebGL2, brings inference down to a couple of milliseconds and leaves most of the 33 ms frame budget for compositing.

The selfie segmenter emits two classes, background and person, as either a category mask (one byte per pixel naming the winning class) or a confidence mask (a float from 0 to 1 per pixel). For blur, ask for the confidence mask: it already is the soft edge your compositor blends with. If you need hair, skin or clothes as separate regions, that is the SelfieMulticlass model, about 16 MB in float32 and two to six and a half times slower on Google's Pixel 6 numbers, so it belongs in beauty-filter features rather than in a default blur.

Figure 1. The end-to-end pipeline. The model itself is tiny; most of the engineering work lives in the input preprocessing, mask upscaling, and compositing stages.

The model has two known weaknesses, and every production deployment has to engineer around them. First, the boundary is only as sharp as the 256-pixel resolution allows — a 1080p webcam feed gets its mask up-sampled by roughly 4× vertically and 7.5× horizontally, and a naive bilinear up-sample produces a soft edge that looks fine on the cheek but bad on individual strands of hair. Second, the model was trained on single-person, head-and-shoulders, indoor footage taken at consumer-camera distance; it generalises poorly to multiple people, to extreme close-ups, to back-lit subjects, and to scenes shot at a wide-angle distance from the camera. Both weaknesses are addressable, and we cover the fixes below.

Where WebGPU Actually Fits In The Pipeline

Start with a correction that many 2025–2026 tutorials get wrong: MediaPipe's own inference in the browser does not run on WebGPU. In @mediapipe/tasks-vision 1.1.1 the delegate option accepts only "CPU" or "GPU", the GPU path initialises a WebGL context on the canvas you pass in, and masks come back as WebGL textures (package type definitions, October 2026). WebGL works because the segmentation network is a stack of convolutions, and a fragment shader can be coerced into multiplying matrices by encoding tensors as textures. TensorFlow.js was built on the same workaround, and for a 106K-parameter CNN it is fast enough.

WebGPU was designed with general-purpose compute as a first-class feature next to graphics, rather than as a graphics API bent into doing maths. A compute shader has direct buffer access, shared workgroup memory and explicit synchronisation; it can dispatch a 16×16×1 workgroup of threads against an arbitrary image without pretending to be a triangle pipeline. That makes it the right home for everything around the model: the two blur passes, mask feathering, temporal smoothing and the final blend are image-processing work a shader finishes in a few milliseconds. If you want the model itself on WebGPU, run the same .tflite file through Google's LiteRT.js (@litertjs/core with the webgpu accelerator) and take over the resizing and mask handling that MediaPipe otherwise does for you.

The other reason WebGPU matters now is coverage. Chrome and Edge turned it on for Windows, macOS and ChromeOS in version 113 (May 2023) and for Android 12+ in version 121 (January 2024); on Linux, Chrome 144 enabled it for Intel Gen12+ GPUs and Chrome 147 for NVIDIA on Wayland. Firefox 141 shipped it on Windows in July 2025, Firefox 145 on macOS Tahoe on Apple silicon, and Firefox 147 on older macOS versions on Apple silicon. Safari 26 turned it on by default across macOS, iOS, iPadOS and visionOS in September 2025 (WebGPU implementation status, edited August 2026). The gaps that remain, such as Firefox on Linux, Android and Intel Macs, Chrome on most non-Intel Linux setups, and Safari 18 and earlier, are why the WebGL compositor stays in your build as the fallback.

The Full Pipeline — From getUserMedia To WebRTC

Production background blur is not a single API call. It is a four-stage pipeline that starts at the camera and ends at the WebRTC peer connection.

Stage one is capture. The navigator.mediaDevices.getUserMedia() call returns a MediaStream containing a single MediaStreamTrack of video. That track is the source for every downstream stage. For 720p input at 30 fps, the track processor delivers them as 1280×720 WebCodecs VideoFrame objects.

Stage two is inference. Each frame is handed to MediaPipe Image Segmenter via segmentForVideo(videoFrame, timestamp, callback), which resizes the input to 256×256, runs the model on the GPU delegate (WebGL2 inside MediaPipe) and gives you an MPMask. Use the callback form: the return-value form copies every mask, and the API documentation warns against it for high-throughput use. The mask lives in a WebGL texture. If your compositor is WebGPU, either read it back with getAsFloat32Array(), a GPU-to-CPU copy of 256 × 256 floats × 4 bytes = 262,144 bytes (about 256 KB) every frame, or copy the canvas into a WebGPU texture with copyExternalImageToTexture().

Stage three is compositing. A WebGPU fragment-shader pipeline takes three inputs: the original 1280×720 frame, a Gaussian-blurred copy of the same frame at the same resolution, and the 256×256 mask. For each output pixel, the shader samples the mask (with bilinear filtering), uses the mask value as the blend weight between the sharp and blurred frames, and writes the result to an offscreen canvas. The Gaussian blur itself is two separable passes (horizontal, then vertical) of a 1-D 21-tap kernel, run on a quarter-resolution copy of the frame: once blurred, the lost detail is invisible, and 21 taps then cover the whole kernel. Sigma, counted in full-resolution pixels, sets the look: sigma 8 for a Zoom-like medium blur, sigma 16 for a heavy blur, sigma 4 for the "barely there" professional look.

Stage four is encoding. The offscreen canvas becomes a VideoFrame that you write into a MediaStreamTrackGenerator (Chrome and Edge) or a VideoTrackGenerator (Safari 18+, the W3C name). Either exposes a MediaStreamTrack that looks to WebRTC exactly like a camera track. Add it to the RTCPeerConnection with pc.addTrack(processedTrack, processedStream), and the recipient cannot tell a blurred sender from an unblurred one. Firefox has neither generator as of October 2026, so it takes the canvas path in the ladder below.

The Insertable Streams machinery ties stages two through four together. The MediaStreamTrackProcessor reads the camera track as a ReadableStream of VideoFrame objects; you transform those frames through stages two and three; and you write the result into the generator. In Safari the processor and the generator live inside a dedicated Worker, and the worker posts the generated track back to the page. In Chrome both objects are exposed on the main thread, so you create them there and transfer the processor's readable stream and the generator's writable stream to the worker with postMessage. Either way the per-frame work runs off the main thread, which keeps the UI responsive even on a four-year-old laptop.

// Skeleton running inside a dedicated Worker.
// MediaPipe's GPU delegate uses WebGL2; compositorRun() is your WebGPU code.
// Chrome: create MediaStreamTrackProcessor + MediaStreamTrackGenerator on the
//   main thread, add generator to the call, transfer readable/writable here.
// Safari 18+: create MediaStreamTrackProcessor + VideoTrackGenerator here and
//   postMessage(generator.track) back to the page for pc.addTrack().
import { ImageSegmenter, FilesetResolver } from "@mediapipe/tasks-vision";

const vision = await FilesetResolver.forVisionTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@1.1.1/wasm",
);
const segmenter = await ImageSegmenter.createFromOptions(vision, {
  baseOptions: {
    modelAssetPath: "selfie_segmenter.tflite",  // float16, about 244 KB
    delegate: "GPU",                        // WebGL2 under the hood
  },
  canvas: new OffscreenCanvas(1, 1),         // GPU delegate needs a canvas in a Worker
  runningMode: "VIDEO",
  outputCategoryMask: false,
  outputConfidenceMasks: true,               // soft 0-1 mask = blend weight
});

self.onmessage = async ({ data: { readable, writable } }) => {
  const reader = readable.getReader();
  const writer = writable.getWriter();
  while (true) {
    const { value: frame, done } = await reader.read();
    if (done) break;
    let maskData;
    // Callback form skips MediaPipe's extra mask copy; the mask is valid only
    // inside the callback, so read it out here (a 256 KB GPU readback).
    segmenter.segmentForVideo(frame, frame.timestamp / 1000, (result) => {
      const masks = result.confidenceMasks;
      maskData = masks[masks.length - 1].getAsFloat32Array();
    });
    // Returns a new VideoFrame that keeps frame.timestamp.
    const composited = await compositorRun(frame, maskData, blurSigma);
    await writer.write(composited);
    frame.close();
  }
};

The full implementation is roughly 350 lines of TypeScript including the WebGPU shader. If you are on LiveKit, its open-source @livekit/track-processors package ships ready background-blur and virtual-background processors on the same MediaPipe model. Version 0.8.2 (October 2026) still pins @mediapipe/tasks-vision 0.10.14, so test it against the 1.x line before you rely on it.

How To Get A Subject Cut-Out With Alpha Transparency In A Video-Call SDK

Ask the segmenter for the confidence mask, not the category mask, and use it as the alpha channel: each value from 0 to 1 says how sure the model is that the pixel belongs to the person. Upsample it to the frame size, feather the edge with the sigmoid remap described in the pitfalls section, and multiply the colour by it. You now have a premultiplied cut-out of the subject on a transparent background inside the sender's browser.

The catch is the wire. WebRTC video tracks carry no alpha channel: VP8, VP9, H.264 and AV1 as negotiated in WebRTC all encode opaque frames, so transparency disappears the moment a frame enters the encoder. That leaves two designs. The common one is to composite on the sender, putting the subject over a blurred room, an image or a branded background, and send an ordinary opaque frame; this is what nearly every SDK does. The other, for layouts where the receiver must composite (a presenter floating over shared slides, for example), is to pack the mask next to the colour image in the same frame, colour on one side and the mask as grayscale on the other, and unpack it in a shader on the receiving side. That "stacked alpha" frame is wider and costs noticeably more bitrate for the same quality, so use it only when the layout really needs receiver-side compositing. If you would rather replace the room than blur it, the virtual backgrounds lesson covers the compositing details.

Cross-Browser Reality — A Capability Detection Ladder

Insertable Streams is still not uniformly supported as of October 2026 (MDN browser-compat data). Chrome and Edge have shipped Google's earlier, non-standard pair, MediaStreamTrackProcessor and MediaStreamTrackGenerator, since version 94; they are exposed on the main thread and handle audio as well as video. Safari 18 shipped the W3C-standard pair, MediaStreamTrackProcessor and VideoTrackGenerator, inside dedicated workers and for video only. Firefox has shipped neither: Mozilla announced a worker-only, video-only implementation of the standard in September 2024, but it has not reached release Firefox. One code path will not cover all three engines, and Firefox users always land on the canvas fallback.

The right engineering response is a capability-detection ladder that picks the best available pipeline at runtime, rather than a hard browser-version check. The ladder looks like this, top to bottom:

The reason the ladder matters is that landing on the CPU tier gives a visibly worse result than tier 1 on the same hardware, and users on the CPU tier will report it as a bug. We have learned to surface the active tier to the user in a small unobtrusive UI hint — "background blur is using your CPU; quality may be reduced" — because users who understand the trade-off file fewer support tickets than users who don't.

A common production mistake is to pick tier 1 because navigator.gpu exists. Some Chrome desktop users have the API defined but get no adapter, because their GPU or driver is on the blocklist (often older integrated graphics on Windows). Capability detection means checking that requestAdapter() actually returns an adapter, and for tier 1 or 2 also checking that MediaStreamTrackProcessor exists in the context where you plan to use it.

A Numeric Example — Frame Budget On A 30 Fps Webcam

A 30-fps feed gives you 33.33 milliseconds per frame to do everything: read the frame, downsample it, run the model, upsample the mask, blur the frame, composite, and hand it back to WebRTC. Anything that overruns the budget either drops frames or adds latency, both of which the user notices.

Worked example on a MacBook Pro M2 with Chrome 138 and a 1280×720 webcam input. The model runs on MediaPipe's GPU delegate and reports 1.5 ms median inference time over a 1-minute capture; everything after it runs in a WebGPU compositor. The down-sample from 1280×720 to 256×256 takes 0.3 ms (a single GPU draw). The Gaussian blur, two separable passes at sigma 8, takes 1.8 ms total. The composite shader takes 0.4 ms. The WebCodecs encode-back-into-a-VideoFrame step takes around 0.6 ms. Total: 4.6 ms per frame. The browser's compositor and the WebRTC encoder consume another ~5 ms before the frame leaves the box, and the budget still has 23 ms of headroom.

On a 6-year-old Intel Core i5 laptop without a discrete GPU, the same pipeline runs on WebGL at tier 2: 7 ms model inference, 1 ms down-sample, 4 ms blur, 1 ms composite, 1 ms encode — 14 ms total. Still under budget. If GPU acceleration is unavailable on the same hardware (a blocklisted driver, or WebGL switched off by policy), tier 5 kicks in: 18 ms model on WASM SIMD CPU, 6 ms blur on 2D canvas, 2 ms composite — 26 ms total, leaving almost no headroom and a visibly soft mask edge.

The takeaway is that the GPU pipeline, MediaPipe's GPU delegate plus a WebGPU compositor, is not just faster; it is the only pipeline that leaves enough headroom for the compositor to do clean work. The CPU tier is running, but the visual quality is meaningfully lower.

Three Pitfalls We See Teams Ship

These are the three failure modes we have walked into in the field and that we now check for in every code review before a background-blur feature ships.

The first pitfall is the soft-edge halo. The 256×256 mask, naively bilinear-upsampled to 1280×720, produces a 3-to-4-pixel soft band around the silhouette where the mask value is somewhere between 0 and 1. In that band, the compositor mixes sharp and blurred pixels, and the result reads as a bright halo on a dark background or a dark halo on a bright one. The fix is a two-step post-processing step on the mask itself: a small Gaussian blur with sigma 2 (to remove the staircase artefacts), followed by a steep sigmoid remap that pushes mid-mask values toward 0 or 1 (mask = 1 / (1 + exp(-12 * (mask - 0.5)))). Both steps live in the WebGPU compositor shader; they cost nothing measurable. The result is a one-pixel-thick transition band that reads as natural depth of field rather than a fluorescent outline.

The second pitfall is hair flicker. The model was trained at 256×256 resolution, and the hair-versus-background boundary at that resolution is on the order of a single pixel wide. Frame-to-frame the model emits slightly different boundary masks even on perfectly stationary footage — the result reads as the hair "shimmering" when the user holds still. The fix is temporal smoothing: keep a small ring buffer of the last 3 frames' masks, and emit the average. Three frames is enough to remove the high-frequency flicker; more than five frames introduces visible lag when the user moves. The cost is one extra texture sample per output pixel per buffered frame; on WebGPU it is invisible in the budget.

The third pitfall is multi-person scenes. A meeting room with two people sitting side by side at one camera reveals that the model treats both people as "person" pixels — there is no instance segmentation, no left-person-versus-right-person separation. For background blur this is usually fine; users want both faces sharp. But the moment a product feature requires single-person segmentation (a beauty filter that should only apply to the user, an avatar that should only replace one face), the right answer is not to fix MediaPipe — it is to upgrade to SAM 2 (lesson 2.4) with the user's face as the prompt. MediaPipe is for the room versus people boundary; SAM 2 is for the this specific person versus everyone else boundary. Using the wrong tool for the wrong job is the single most common architectural mistake we see in this space.

Figure 2. The three pitfalls and their fixes. The middle and right panels are the production-quality references; the left panel is what ships when the pipeline is taken from the MediaPipe README without further work.

One newer problem shows up only in long calls. A GitHub issue opened against @mediapipe/tasks-vision 1.0.1 in August 2026 reports that continuous segmentForVideo calls on the GPU delegate grow internal WebGL handle tables without bound (about 1.2 MB of renderer memory a minute in Chrome 152), until the page turns sluggish after about an hour. The issue is still open. The reporter's workaround, closing and recreating the segmenter every few minutes, costs one frame of re-initialisation and is worth adding to any product with hour-long sessions.

Build Versus Buy — When MediaPipe Is The Wrong Choice

For most web-based video products, MediaPipe Image Segmenter with a WebGPU compositor around it is the right primitive. It is free and permissively licensed (Apache 2.0), it runs on the user's device, so there is no per-user cloud cost, and the quality is high enough that the average user cannot tell it from Zoom or Meet. One privacy detail matters for regulated products: the package's privacy notice (last modified 5 June 2026) says images and video are processed on the device and never sent to Google, but that the APIs do send performance and usage metrics to Google and that you are responsible for users' informed consent. Put that in your data-processing notes before a GDPR or HIPAA review asks.

There are three situations where buying a commercial SDK (NVIDIA Maxine, Banuba, Agora's virtual background extension) makes more sense than building on MediaPipe.

The first is when you also need real-time noise suppression, voice activity detection, and AI-based echo cancellation in the same product, and you want one vendor to support all of them. A bundled SDK costs money per user, but one contract and one integration can save more engineering time than the per-feature quality difference is worth.

The second is when you need a hardware-accelerated effect on a native desktop app rather than the browser. NVIDIA Maxine runs on the user's RTX GPU via CUDA and delivers higher-quality results on supported hardware — but it requires an RTX-class GPU on the user's machine, which excludes the long tail of laptops and Macs. For a Windows-only enterprise desktop product targeting recent RTX hardware, Maxine is a legitimate choice. For a cross-platform web product, it is not.

The third is when you need signed, audited vendor terms for background blur on a regulated product (healthcare, defence, finance). Commercial vendors sell SOC 2 reports, HIPAA BAAs and indemnification on the model's behaviour. MediaPipe is Apache-licensed code; the audit story is on you. For a telemedicine product where you negotiate vendor agreements with hospital procurement, the BAA from a commercial vendor is sometimes worth the per-user fee.

For everything else — and that is the vast majority of products — MediaPipe with a WebGPU compositor is the right answer in 2026. Cross-link to lesson 6.6 on Krisp / Maxine / Dolby build-versus-buy for the more detailed vendor-by-vendor breakdown.

Common Questions About MediaPipe Selfie Segmentation

Does MediaPipe selfie segmentation run on WebGPU?

Not through MediaPipe itself. As of @mediapipe/tasks-vision 1.1.1 (October 2026) the browser GPU delegate runs on WebGL2; use WebGPU for the blur and compositing around the model, or run the same .tflite file through LiteRT.js with its WebGPU accelerator.

What is the difference between the selfie segmenter and the selfie multiclass model?

The selfie segmenter is a binary person-versus-background model of about 244 KB in float16; SelfieMulticlass splits the person into hair, body skin, face skin, clothes and accessories, weighs about 16 MB in float32 and runs two to six and a half times slower on Google's Pixel 6 benchmarks.

Does MediaPipe background blur work in Firefox?

Yes, but not through Insertable Streams: Firefox has no MediaStreamTrackProcessor as of October 2026, so you render to a canvas and send canvas.captureStream(). A WebGPU compositor is available on Windows (Firefox 141+) and Apple-silicon Macs (Firefox 145+ on macOS 26, 147+ on older versions).

Is MediaPipe selfie segmentation free for commercial use?

Yes. MediaPipe is Apache 2.0, and frames are processed on the device; the APIs do send usage and performance metrics to Google, so cover that in your consent and privacy notes.

How much of the frame budget does browser background blur use?

In our worked example on an Apple M2 laptop in Chrome, the whole pipeline takes about 4.6 ms of the 33 ms available per frame at 30 fps; a CPU-only fallback on an older laptop takes about 26 ms and leaves almost no headroom.

Is there a MediaPipe Selfie Segmentation V2?

Google's documentation does not use a "V2" name. People usually mean the selfie_segmenter model in the MediaPipe Tasks Image Segmenter, which replaced the legacy Selfie Segmentation solution in April 2023.

Where Fora Soft Fits In

Fora Soft has shipped background blur as part of WebRTC-based conferencing products since 2021, and we maintain the WebGPU compositor described in this article as a reusable component across our video conferencing, telemedicine, e-learning, and AR/VR projects. We have rolled out the same pipeline behind whitelabel conferencing tools for European telehealth platforms (where the on-device processing satisfies GDPR data-locality requirements), for online tutoring products serving children (where the room background can include other family members that should be blurred), and for OTT post-production workflows where editors review raw interviews and want a clean preview without renting a studio. If you are building any of those, the build pattern is the same; the difference is the verticalisation around the rest of the product. Our video conferencing app development team ships background blur and replacement in WebRTC calls, and our real-time video processing development covers the segmentation, effects and filters behind it. When the blur has to slot into an existing call stack, our WebRTC developers take on the Insertable Streams wiring, the Chrome-versus-Safari worker split and the Firefox canvas fallback.

The book

AI for Video: A Complete Guide to Intelligence in Video Products: From Real-Time Pipelines and Agents to Serving, Cost, Compliance, and Verticals

This lesson spends 4.6 ms of a 33 ms frame; volume 7 of the Fora Soft Video Engineering Handbook is about the rest of that budget. It works through where AI attaches to a WebRTC pipeline and what each insertion point costs in latency, then the neighbours of background blur on the same path — live captions, in-call translation, moderation while the call runs — and finishes with serving, cost per stream-hour, and the EU AI Act and biometrics rules that decide whether a camera feature ships at all. Written by Nikolay Sapunov, CEO at Fora Soft.

Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook

See our case studies

References

1. Google AI Edge. Image segmentation guide — MediaPipe Image Segmenter task documentation. https://developers.google.com/edge/mediapipe/solutions/vision/image_segmenter (accessed 2026-10-20). Source of the model table: input sizes, float16 and float32 precision, Pixel 6 task latency and output classes.
2. google-ai-edge/mediapipe. Selfie Segmentation (legacy solution) — documentation. https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/selfie_segmentation.md (accessed 2026-10-20). Source of the 256×256 and 144×256 inputs, the MobileNetV3 base, the Google Meet landscape variant and the April 2023 upgrade notice.
3. W3C WebRTC WG. MediaStreamTrack Insertable Media Processing using Streams — W3C draft specification. https://www.w3.org/TR/mediacapture-transform/ (accessed 2026-10-20). The standard MediaStreamTrackProcessor / VideoTrackGenerator API, implemented in Safari 18 and not yet in release Firefox.
4. MDN Web Docs. Insertable Streams for MediaStreamTrack API. https://developer.mozilla.org/en-US/docs/Web/API/Insertable_Streams_for_MediaStreamTrack_API (accessed 2026-05-24). Browser-support matrix and worker-vs-main-thread differences.
5. W3C. WebGPU — W3C Candidate Recommendation. https://www.w3.org/TR/webgpu/ (accessed 2026-05-24). Authoritative source for the WebGPU API used by the compositor.
6. W3C. WebCodecs — W3C Candidate Recommendation. https://www.w3.org/TR/webcodecs/ (accessed 2026-05-24). The VideoFrame API that wraps each camera frame as a GPU-backed handle.
7. IETF. RFC 8829 — JavaScript Session Establishment Protocol (JSEP). https://www.rfc-editor.org/rfc/rfc8829 (accessed 2026-05-24). The WebRTC offer/answer machinery into which the processed MediaStreamTrack is wired.
8. Mozilla WebRTC blog. Unbundling MediaStreamTrackProcessor and VideoTrackGenerator (September 2024). https://blog.mozilla.org/webrtc/unbundling-mediastreamtrackprocessor-and-videotrackgenerator/ (accessed 2026-10-20). Mozilla's plan for a worker-only, video-only implementation of the standard API.
9. Chrome Platform Status. MediaStreamTrack Insertable Streams (Breakout Box). https://chromestatus.com/feature/5499415634640896 (accessed 2026-05-24). Chrome's pre-spec implementation, shipped in 2021.
10. WebRTC Hacks / WebRTC.Ventures. Background Removal Using Insertable Streams (2023). https://webrtc.ventures/2023/02/background-removal-using-insertable-streams/ (accessed 2026-05-24). Detailed walkthrough of the Insertable Streams + segmentation pattern; the architectural ancestor of the pipeline described in this article.
11. GPU for the Web Community Group. WebGPU Implementation Status. https://github.com/gpuweb/gpuweb/wiki/Implementation-Status (accessed 2026-10-20; page edited August 2026). Source of the per-platform WebGPU versions for Chrome, Firefox and Safari.
12. Howard, A., et al. Searching for MobileNetV3. arXiv:1905.02244 (2019). https://arxiv.org/abs/1905.02244 . The architecture family from which the selfie-segmenter backbone derives.
13. Qualcomm AI Hub. MediaPipe-Selfie-Segmentation model card. https://aihub.qualcomm.com/models/mediapipe_selfie (accessed 2026-10-20). Source of the 106K parameter count and the 447 KB float model size.
14. WebRTC Project. getUserMedia — Media Capture and Streams W3C Recommendation. https://www.w3.org/TR/mediacapture-streams/ (accessed 2026-05-24). The standard for the camera-track input side of the pipeline.
15. NVIDIA Maxine. Background Blur — Video Effects (VFX) SDK User Guide v1.1.0. https://docs.nvidia.com/maxine/vfx/1.1.0/Filters/BackgroundBlur.html (accessed 2026-05-24). Reference for the commercial-vendor build-versus-buy comparison.
16. MediaPipe. @mediapipe/tasks-vision package README and type definitions, version 1.1.1 (published on npm 14 October 2026). https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@1.1.1/README.md (accessed 2026-10-20). Type definitions (delegate: "CPU" | "GPU", WebGL canvas for GPU processing, WebGL-texture masks, callback guidance for segmentForVideo) and the privacy notice last modified 5 June 2026.
17. MDN Web Docs. MediaStreamTrackProcessor — browser compatibility. https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackProcessor (accessed 2026-10-20). Limited availability: Chrome 94 (main thread), Safari 18 (workers), not in Firefox.
18. web.dev. WebGPU is now supported in major browsers (November 2025). https://web.dev/blog/webgpu-supported-major-browsers (accessed 2026-10-20). Chrome 113 and 121, Firefox 141 and 145, Safari 26.
19. Google AI Edge. LiteRT for Web with LiteRT.js. https://developers.google.com/edge/litert/web (accessed 2026-10-20). Running .tflite models in the browser with the WebGPU, WebNN or Wasm accelerator.
20. google-ai-edge/mediapipe. Issue #6352: ImageSegmenter (Web, GPU delegate) GL handle tables grow unbounded during continuous segmentForVideo (2026). https://github.com/google-ai-edge/mediapipe/issues/6352 (accessed 2026-10-20). Long-session memory growth on tasks-vision 1.0.1 and the recreate-the-segmenter workaround.
21. Google Research. Background Features in Google Meet, Powered by Web ML (2020). https://research.google/blog/background-features-in-google-meet-powered-by-web-ml/ (accessed 2026-10-20). How Meet runs MediaPipe segmentation in the browser with WebAssembly.
22. LiveKit. track-processors-js (npm @livekit/track-processors 0.8.2, October 2026). https://github.com/livekit/track-processors-js (accessed 2026-10-20). Open-source background-blur and virtual-background processors built on @mediapipe/tasks-vision 0.10.14.