Updated October 2026: Depth Anything 3 (November 2025) and Video Depth Anything licences, corrected SmolVLM pixel-shuffle and memory figures, and a 2026 small-VLM comparison with Gemma 4 E2B and Qwen3.5-0.8B.
SmolVLM And Depth Anything: The Short Answer
SmolVLM is Hugging Face's family of small open vision-language models: SmolVLM2 ships in 256M, 500M and 2.2B parameters, reads images and video, and is released under Apache 2.0 (Hugging Face, February 2025). Depth Anything is a family of monocular depth models: Depth Anything V2 Small (24.8M parameters, Apache 2.0, NeurIPS 2024) turns one RGB frame into a per-pixel depth map, and Depth Anything 3 (ByteDance Seed, November 2025) adds multi-view geometry and camera poses, with Apache 2.0 Small, Base, Metric-Large and Mono-Large checkpoints. Together they cover the two halves of on-device vision: depth measures the scene, the small VLM describes it. Hardware floor: a device with 4 GB of RAM and an AI accelerator runs Depth Anything V2 Small plus SmolVLM2-500M; the 2.2B model needs about 5 GB for video.
What is the smallest VLM? As of October 2026, SmolVLM-256M and SmolVLM2-256M are the smallest widely used instruction-following vision-language models: a 93M-parameter SigLIP encoder plus the 135M-parameter SmolLM2 language model, running single-image inference in about 0.8 GB of GPU memory (SmolVLM paper, arXiv 2504.05299, 2025). The next tier is LFM2-VL-450M (Liquid AI, 2025), SmolVLM2-500M and Qwen3.5-0.8B (Alibaba, March 2026).
Is there a SmolVLM 3? No. As of October 2026 Hugging Face has not released a SmolVLM3. SmolLM3 (July 2025) is a 3B text-only language model, and SmolVLA (June 2025) is a ~450M vision-language-action model for robots built on SmolVLM2-500M.
Why This Matters
The interesting thing about 2026 is not that AI got bigger. It is that small models got useful. Two years ago, depth from a single camera was a research problem; today a 25-megaparameter model trained with a synthetic-data teacher outputs depth maps good enough for occlusion, background blur and coarse distance checks without a stereo rig. Two years ago, a vision-language model that could describe a video clip required a multi-GPU server; today a 500-megaparameter model running inside an iPhone app can caption short clips and answer simple questions about them. If you ship a product that touches video — surveillance, telemedicine, e-learning, OTT, video conferencing, AR — the question is no longer whether small on-device CV is feasible. It is which two or three of these primitives you stack together to answer the product question. This article is for the product manager, founder, or video engineer who needs to scope a feature that involves understanding what is inside a frame — its geometry, its content, the relationship between its objects — without paying a per-call API price or asking the user's phone to upload every frame to a server.
The Two Halves Of On-Device Vision
A useful way to think about modern small-model computer vision is that it splits into two complementary jobs. The first is measurement — turning pixels into numbers about the physical scene: where surfaces are, how far away they are, how they move. The second is understanding — turning the same pixels into natural language about the scene's content and meaning. Depth Anything and SmolVLM each own one half of that split.
Depth Anything is a measurement primitive. It takes one RGB frame and outputs a depth value for every pixel. The default models output relative depth, which keeps the shape and ordering of the scene up to scale; the metric variants output metres. The output is dense (one number per pixel, not one number per detected object) and geometric (the values follow the scene's surfaces, not just a front-to-back ranking of objects). Downstream tasks consume those numbers directly: a planner uses depth to avoid obstacles; an AR app uses depth to occlude a virtual object behind a real one; a surveillance system uses depth to convert a 2D bounding box into a 3D position.
SmolVLM is an understanding primitive. It takes one or more frames plus a natural-language prompt, and it outputs text. The text might be a description of what is in the frame, an answer to a question about the frame, a structured extraction of a fact, or a classification. The model is small enough to run locally, but the interface is the same one you would use against a hosted frontier VLM — give it pixels and a prompt, get text back.
You almost never pick one of these and ignore the other. The interesting on-device products in 2026 use both: depth to handle the geometry layer, a small VLM to handle the semantics layer, and a thin orchestration layer on top that turns the combined signal into product behaviour. The rest of this article goes deep on each model and then shows how to stack them.
Depth Anything V2 — What It Is And How It Works
Depth Anything V2 was released by the original Depth Anything team in 2024 and accepted as a NeurIPS 2024 paper. It is a foundation model for monocular depth estimation — a single neural network that, given any RGB image from any camera in any scene, outputs a dense depth map. The "anything" in the name is a claim about the breadth of inputs it handles: indoor, outdoor, daylight, low light, real photos, paintings, generated images. The training methodology is what makes that claim hold.
The architecture is a Vision Transformer encoder plus a Dense Prediction Transformer decoder. The encoder is reused from DINOv2 — Meta's self-supervised foundation model — which provides general-purpose image features. The decoder, called DPT, aggregates features from multiple transformer layers and upsamples them into a full-resolution depth map. By default the model works at a 518-pixel input: training resizes the shorter side to 518 and crops 518 by 518, and the reference inference script resizes to 518 while keeping the aspect ratio (Depth Anything V2 repository, 2024).
The four sizes shipped by the team are ViT-Small (24.8 million parameters), ViT-Base (97.5 million), ViT-Large (335.3 million), and ViT-Giant (1.3 billion). ViT-Small is the size that fits real-time on-device budgets; ViT-Large is the accuracy sweet spot for offline work; ViT-Giant is the teacher used to train the smaller students, and as of October 2026 its checkpoint has still not been released publicly. The paper reports inference more than ten times faster than Stable-Diffusion-based depth models of 2024, such as Marigold, at comparable or better accuracy (Yang et al., arXiv 2406.09414, 2024).
The training trick — what V2 added on top of V1 — is a three-step teacher-student pipeline. Step one trains the largest model (ViT-Giant) only on about 595,000 synthetic images from five datasets, with perfect ground-truth depth, because real-world depth sensors produce noisy labels and synthetic images do not. Step two uses that ViT-Giant teacher to generate pseudo-labels on 62 million unlabelled real images from eight public datasets. Step three trains the smaller student models on those pseudo-labels. The result: V2 produces much finer-grained and more robust depth than V1, especially on transparent and reflective surfaces where the old depth-sensor labels were systematically wrong.
There is a critical licensing detail. The ViT-Small checkpoint is released under Apache 2.0, which permits commercial use. The Base and Large checkpoints (and Giant, if it ships) are released under CC-BY-NC-4.0, which prohibits commercial use. If you are building a commercial product, you can only ship the Small variant out of the box. The Base/Large/Giant checkpoints are usable for research, evaluation, and pre-training your own model on a commercially-licensed dataset, but you cannot embed them in a paid product. This is the single most common compliance trap teams fall into when they prototype with Large (because it is more accurate), demo it to a stakeholder, and then discover they have to rebuild with Small before shipping. The same Small-is-commercial pattern repeats in Video Depth Anything, so read the licence on each checkpoint's model card, not on the repository.
Depth Anything V2 by default produces relative depth — depth values that respect ordering and relative scale but are not in metric units. For products that need actual distances in metres (a drone deciding whether an obstacle is two metres away or twenty), the team ships metric depth variants fine-tuned on two datasets: Hypersim for indoor scenes (max depth 20 metres) and Virtual KITTI 2 for outdoor scenes (max depth 80 metres). Pick the variant that matches your deployment environment — using an outdoor model indoors gives you depth values that float around in absurd ranges.
For video, vanilla Depth Anything V2 has a known limitation: running it frame-by-frame on a video produces temporal flickering — adjacent frames get depth values that jitter even when the underlying scene is static. Two follow-up models published in 2025 address this. Video Depth Anything (CVPR 2025 Highlight) replaces the per-frame DPT head with a spatial-temporal head that constrains the depth gradient between frames and handles arbitrarily long videos consistently. Its Small checkpoint (28.4 million parameters) is Apache 2.0; Base (113.1 million) and Large (381.8 million) are CC-BY-NC-4.0. During 2025 the team added metric-depth checkpoints and a training-free streaming mode for live input (Video Depth Anything repository, September 2025). FlashDepth (ICCV 2025) keeps the per-frame backbone but adds a recurrent module that aligns depth features across frames, and runs in real time at 2044 by 1148 pixels and 24 FPS (Chou et al., arXiv 2504.07093, 2025). Both are extensions of V2; choose Video Depth Anything if you need to process a stored clip with consistent depth, FlashDepth if you need a live stream with low latency.
Depth Anything 3 — What Changed In November 2025
On 14 November 2025 ByteDance Seed released Depth Anything 3 (arXiv 2511.10647). Where V2 predicts depth from one image, DA3 takes any number of views — one frame, a burst of photos or a video — and predicts depth together with camera pose (intrinsics and extrinsics) from a single plain transformer, using a depth-ray representation. Choose it when the product needs a consistent 3D reconstruction of a room or a scene, not just a depth map per frame. For one camera that only needs per-frame depth on a phone, V2 Small is still the lighter and better-documented option.
| Checkpoint | Parameters | Licence | Use it for |
|---|---|---|---|
| DA3-Small | 0.08B | Apache 2.0 | Any-view depth and pose on an edge budget |
| DA3-Base | 0.12B | Apache 2.0 | Same job, more accuracy |
| DA3-Large | 0.35B | CC BY-NC 4.0 | Research and evaluation only |
| DA3-Giant | 1.15B | CC BY-NC 4.0 | Highest accuracy, 3D Gaussian output; research only |
| DA3Metric-Large | 0.35B | Apache 2.0 | Monocular metric depth in a commercial product |
| DA3Mono-Large | 0.35B | Apache 2.0 | Monocular relative depth in a commercial product |
The licensing rule changes in one useful way: in DA3 the Metric-Large and Mono-Large models are Apache 2.0, so a commercial product can now ship a Large-class monocular depth model. In December 2025 the team added DA3-Streaming for long videos in under 12 GB of GPU memory, and it later published retrained 1.1 checkpoints of Large, Giant and Nested-Giant after fixing a training bug. If you evaluate those sizes, use the 1.1 versions (Depth Anything 3 repository, 2025).
SmolVLM2 — What It Is And How It Works
SmolVLM is Hugging Face's family of small open multimodal models — vision-language models small enough to actually run on a device the user is holding. The family started in November 2024 with the original SmolVLM (2.2B parameters); in January 2025 the team released the smaller 256M and 500M variants; in February 2025 they released SmolVLM2, which extended all three sizes to video. The technical report followed in April 2025 (Marafioti et al., arXiv 2504.05299). As of October 2026 SmolVLM2 is still the current generation: Hugging Face has not released a SmolVLM3. The 2.2B model is the strongest; the 500M is the on-device sweet spot; the 256M is for browser and severely constrained edge.
The architecture has three parts: a SigLIP vision encoder, a pixel-shuffle compression step, and a Llama-3 language decoder.
The vision encoder is SigLIP-B/16 (93 million parameters, 16-by-16 patches) for the 256M and 500M models and SigLIP-SO400M (428 million parameters, 14-by-14 patches) for the 2.2B model. The encoder turns each image into a grid of visual feature vectors — small tiles, each summarising a region of the image. The notable design choice is that the vision tower is small relative to the language tower. Bigger vision encoders quickly stop adding accuracy once the language model is also small.
The pixel-shuffle step is the trick that makes the whole thing efficient. A naive design feeds every visual feature vector into the language model as a token, and the token count blows up the attention cost. Pixel shuffle rearranges each r-by-r block of feature vectors into one vector with r² times more channels, so the visual token count drops by r², and the attention cost, which is quadratic in token count, drops by r⁴. SmolVLM compresses harder than its predecessor Idefics3, which used r = 2: the 2.2B model uses r = 3 (about nine times fewer visual tokens), and the 256M and 500M models use r = 4 (sixteen times fewer). The paper's finding is direct: smaller VLMs benefit from more aggressive compression (arXiv 2504.05299, 2025). This is what makes a 500M-parameter model with a vision input runnable on a phone.
The language decoder is the SmolLM-2 family of small Llama-3-architecture language models. It is the same architecture you would find in a small open LLM — transformer, RMSNorm, SwiGLU, rotary position embeddings — just at small parameter counts (135M for SmolVLM2-256M, 360M for the 500M, 1.7B for the 2.2B). Visual tokens and text tokens go in interleaved; the model outputs text, autoregressively.
For video, SmolVLM2 samples up to fifty evenly spaced frames per clip and treats them as an ordered sequence of images, prepending a textual marker like "Here are 50 frames sampled from a video" so the model can tell "the same object in different frames" from "different objects". The frame budget keeps the token count tractable even after pixel shuffle. On Video-MME, the standard video-understanding benchmark, the 2.2B model scores 52.1, the 500M 42.2 and the 256M 33.7 (Hugging Face model cards, 2025): the 256M and 500M are the smallest video-capable VLMs in wide use.
Resource numbers, because they decide what hardware you need (single-image memory at batch size 1 from the SmolVLM paper; video memory and Video-MME from the SmolVLM2 model cards, 2025):
| Model | Vision encoder + language model | Pixel shuffle | GPU RAM, one image | GPU RAM, video | Video-MME |
|---|---|---|---|---|---|
| SmolVLM2-256M | SigLIP-B/16 (93M) + SmolLM2-135M | r = 4 | 0.8 GB | 1.38 GB | 33.7 |
| SmolVLM2-500M | SigLIP-B/16 (93M) + SmolLM2-360M | r = 4 | 1.2 GB | 1.8 GB | 42.2 |
| SmolVLM2-2.2B | SigLIP-SO400M (428M) + SmolLM2-1.7B | r = 3 | 4.9 GB | 5.2 GB | 52.1 |
All SmolVLM2 checkpoints, datasets, and training recipes are released under Apache 2.0. No carve-outs, no NC restrictions. You can embed them in a commercial product without paying anyone. Hugging Face's own iPhone app, HuggingSnap, runs SmolVLM2-500M entirely on the device, and the 256M and 500M models have official ONNX checkpoints with WebGPU browser demos.
Small VLMs Compared: Is SmolVLM Still The Smallest In 2026?
Yes on size, no longer alone on capability. SmolVLM2-256M is still the smallest widely used instruction-following vision-language model as of October 2026, but 2026 brought small multimodal models from Alibaba and Google that trade a bigger footprint for stronger reasoning, longer context and, in Gemma 4's case, native audio. Pick the smallest model that answers your product question on your target device, then check the licence.
| Model | Size | Released | Licence | Input | Where it fits |
|---|---|---|---|---|---|
| SmolVLM2-256M | 256M | Feb 2025 | Apache 2.0 | Images, video | Browser (WebGPU), low-end phones, yes/no gating |
| LFM2-VL-450M (Liquid AI) | ~450M (350M LM + 86M SigLIP2) | 2025 | LFM Open License v1.0 (commercial use has conditions; read it) | Images up to 512×512 native | Constrained devices; adjustable image-token budget |
| SmolVLM2-500M | 500M | Feb 2025 | Apache 2.0 | Images, video | The on-device sweet spot; HuggingSnap on iPhone |
| Qwen3.5-0.8B (Alibaba) | 0.8B | Mar 2026 | Apache 2.0 | Images (video config in repo) | Stronger reasoning under 1B; Qualcomm-optimised build |
| SmolVLM2-2.2B | 2.2B | Feb 2025 | Apache 2.0 | Images, video | Laptops, Jetson, flagship phones |
| Gemma 4 E2B (Google) | ~2B effective | Apr 2026 | Apache 2.0 | Images, video, audio | Offline on phones, Raspberry Pi, Jetson Orin Nano; 140+ languages |
Sources: SmolVLM2 model cards (2025); LFM2-VL-450M model card; Qwen3.5-0.8B model card (March 2026); Google, "Gemma 4" announcement (2 April 2026).
Two names cause confusion in search. SmolVLA is not a smaller SmolVLM: it is Hugging Face's ~450M vision-language-action model for robot control (June 2025, Apache 2.0), built from SmolVLM2-500M plus an action expert. And research-licence models such as Apple's FastVLM (licence: apple-amlr) look attractive on benchmarks but are not cleared for a paid product — the same trap as Depth Anything V2 Large. For when a larger open model or a hosted VLM is the better answer, see open frontier VLMs: LLaVA and Qwen-VL and when a VLM should replace custom CV.
Where Each Model Belongs — A Decision Frame
The two models solve different problems, and the most common engineering mistake is to reach for the wrong one. A useful decision rule:
Use Depth Anything when you need a number per pixel. Examples: an AR app deciding whether to draw a virtual object in front of or behind a real one; a surveillance pipeline lifting a 2D detection into a 3D position; a robot deciding whether to move; a video conferencing app generating background blur that respects the actual physical depth of the scene; a video editing tool extracting a foreground subject for compositing. In all of these cases, the output is geometric and dense, and a VLM cannot produce it — a VLM speaks text, not depth maps.
Use SmolVLM when you need a sentence about the frame. Examples: a moderation pipeline asking "does this clip contain a person holding a weapon?"; an e-learning analytics tool asking "what is the student writing on the whiteboard right now?"; a surveillance dashboard asking "describe what just happened in this five-second clip"; an OTT search tool asking "what kind of scene is this — sports, drama, news?". In all of these cases, the output is text, the question changes from product to product, and you do not want to train a separate classifier for every question.
Use both when the product question is "what is this thing and where is it in the scene?" Stack a depth model and a VLM in parallel on the same frame; merge the outputs in your orchestration layer. The VLM tells you what is interesting; the depth model tells you how far away it is and how big.
A common pitfall: teams try to ask the VLM a geometric question — "How far away is the person in this frame?" — and then act surprised when the answer is wrong. SmolVLM and every other small VLM is unreliable for absolute geometric questions. It might be right by accident in one frame and wrong by an order of magnitude in the next. The model was not trained on metric depth labels; it was trained on captions and conversations. If you need numbers, use a model that outputs numbers. If you need words, use a model that outputs words.
| Question type | Right primitive | Output | Why |
|---|---|---|---|
| How far is every pixel from the camera? | Depth Anything V2 (metric variant) or DA3Metric-Large | Dense per-pixel depth map in metres | Geometric task; specialised model |
| Is there a person in this frame? | A small detector (YOLO-N) or SmolVLM | Bounding box or text | Detector is faster; VLM is more flexible |
| What is the person doing? | SmolVLM | Natural-language description | Open-ended; cannot pre-define a class list |
| Is the camera moving? | Optical flow (RAFT, Lucas-Kanade) | Per-pixel motion field | Geometric task |
| Should I alert the operator? | SmolVLM with a structured-output prompt | "alert: true/false, reason: ..." | Composes the other signals into a decision |
| Where exactly is the moving object in 3D? | Depth + detector + simple geometry | (x, y, z) per object | Combine geometry + identity |
The last row is the canonical small-on-device CV pipeline. A detector finds objects; depth gives them 3D position; a VLM (optionally) adds semantic context. Each primitive does the job it is good at.
A Concrete Stack — Depth + SmolVLM Together
Here is a worked example of a pipeline a real product might ship. Imagine an indoor security camera. The product question is: "alert me when someone walks into the kitchen at night carrying anything that looks like a tool." The naive way to answer this is to pipe every frame to a hosted frontier VLM and pay per call. The on-device way uses three primitives, in series, each tuned to its job.
Stage one — motion gate. Cheap background subtraction runs on every frame at 30 FPS and decides whether enough pixels have changed to warrant looking at this frame. Cost: a few milliseconds per frame. In a quiet indoor scene most frames are filtered out before any neural model runs; measure the rate on your own footage.
Stage two — depth + detection on the kept frames. On each kept frame, run a small person detector (a tiny YOLO-N) and the Depth Anything V2 metric-indoor (Hypersim) Small checkpoint in parallel. The detector outputs a bounding box; the depth model outputs a depth map. Combine them by reading the depth value at the centre of the bounding box — that gives you the person's distance from the camera. If the distance places the person inside the geometric volume you previously labelled "kitchen", proceed to stage three. Otherwise, drop the frame. Budget tens of milliseconds per kept frame on a Jetson Orin Nano class board, and measure it on your hardware before you fix the frame budget.
Stage three — SmolVLM2 semantic check. On the rare frames that survive stage two (someone is actually in the kitchen), sample a five-frame clip and feed it to SmolVLM2-500M with the prompt: "Does the person in this video clip appear to be holding any kind of tool or implement? Answer with 'yes' or 'no' and a one-sentence reason." The model runs locally on the same Jetson and returns a structured answer without a network call; plan for up to a second or two per clip. If it says yes, fire the alert.
The whole pipeline keeps up with a 30 FPS camera, because the VLM runs only on the rare clips that pass stage two, on an edge board in the few-hundred-dollar range (NVIDIA lists the Jetson Orin Nano Super developer kit at $249, December 2024), costs nothing per inference because no network call leaves the device, and respects the household's privacy because the video never touches a cloud server. The same pattern adapts to telemedicine ("is the patient still in frame and conscious?"), to e-learning ("is the student looking at the screen?"), to OTT moderation ("is this clip safe to play to a child user?"). The architecture is the same; only the prompts and the geometric thresholds change.
Production Failure Modes And Engineering Fixes
Three failure modes turn up reliably in real deployments. Plan for them on day one.
Failure 1 — flicker in the depth track. Running Depth Anything V2 frame-by-frame on a video produces depth values that wobble even on a static scene. For products where the downstream consumer is just looking at a per-frame number (a single distance check, a coarse occlusion test), the wobble is tolerable. For products that draw the depth map directly to the user (any AR application; any visualisation), the wobble looks terrible. Fix: switch to Video Depth Anything (CVPR 2025) for stored video or its 2025 streaming mode for live input, or to FlashDepth (ICCV 2025) for live 2K streams. Both are V2 extensions with a temporal-consistency module. Expect a higher per-frame compute cost than vanilla V2 and measure it on your target; Video Depth Anything Small (28.4 million parameters, Apache 2.0) is the commercially licensed starting point.
Failure 2 — wrong metric variant. The metric depth variants are fine-tuned on either Hypersim (indoor, 20 m) or VKITTI (outdoor, 80 m). Using the outdoor variant on indoor footage gives you depth values floating in absurd ranges; using the indoor variant on outdoor footage clamps everything past 20 m to the same value. Fix: pick the variant that matches your scene at config time, not at training time. A surveillance camera that sometimes points indoors and sometimes outdoors should ship both variants and route at runtime based on a one-time setup classification.
Failure 3 — SmolVLM hallucination on out-of-distribution inputs. SmolVLM2 was trained on a lot of data, but its 500M and 256M variants are still small models. When you ask them a question about content they did not see in training, the answers can be confidently wrong. Fix: constrain the output with structured prompts and a guard. Ask "yes or no" questions, not open-ended ones. Pair every SmolVLM call with a second pass that re-asks the same question on a different sampled frame and only acts when the two agree. For high-stakes decisions, route the survivors to a bigger model on the server — the on-device VLM is the cheap filter, not the final adjudicator.
Hardware And Deployment Reality
Where does each model actually run in production? A quick reference.
| Hardware | Depth Anything V2 Small | SmolVLM2-256M | SmolVLM2-500M | SmolVLM2-2.2B |
|---|---|---|---|---|
| Modern phone (iPhone 15 Pro / Pixel 9) | Real time after Core ML or LiteRT export | Yes | Yes — HuggingSnap runs it on iPhone | Marginal: about 5 GB for video |
| Browser via WebGPU | Yes, through ONNX | Yes — official WebGPU demo | Yes — official WebGPU demo | Not practical |
| Jetson Orin Nano (8 GB) | Yes, through TensorRT | Comfortable | Comfortable | Fits, little headroom |
| Jetson AGX Orin | Comfortable | Trivial | Trivial | Comfortable |
| Consumer GPU (RTX 4070, 12 GB) | Comfortable | Trivial | Trivial | Comfortable |
SmolVLM memory figures come from the SmolVLM paper and model cards; frame rates depend on resolution, precision and the rest of your pipeline, so measure on the target device before you commit to a frame budget.
The conversion pattern is well-established. Export the PyTorch checkpoint to ONNX, then either convert to TensorRT on NVIDIA, Core ML on Apple, or LiteRT (formerly TensorFlow Lite) or Qualcomm AI Hub on Android. Apple publishes a Core ML build of Depth Anything V2 Small (June 2024), and SmolVLM2 has MLX builds for Apple silicon and GGUF builds for llama.cpp. Quantising from FP32 to FP16 or INT8 halves or quarters weight memory; the speed-up depends on what the accelerator supports. For Depth Anything V2 specifically, the small variant exports cleanly to ONNX; the larger variants need a couple of operator workarounds at export time but the community has shared working scripts.
A practical 2026 rule for picking compute: the device runs the pipeline if it can hold the largest weight file you need in RAM at FP16 and still have memory for an OS, a video buffer, and other apps. For Depth Anything V2 Small that is about 50 MB of weights; for SmolVLM2-500M that is about 1 GB; the practical floor for combined use is a device with 4 GB of RAM and a hardware AI accelerator. Everything from an iPhone 13 onward and from a mid-range Android forward clears that bar.
Frequently Asked Questions
What is the SmolVLM architecture?
A SigLIP vision encoder (93M parameters for the 256M and 500M models, SigLIP-SO400M for the 2.2B), a pixel-shuffle step that cuts visual tokens 9 to 16 times, and a SmolLM2 language decoder of 135M, 360M or 1.7B parameters (arXiv 2504.05299, 2025).
How much memory does SmolVLM need?
About 0.8 GB of GPU memory for single-image inference with the 256M model, 1.2 GB with the 500M and 4.9 GB with the 2.2B; video inference needs 1.38 GB, 1.8 GB and 5.2 GB respectively (SmolVLM paper and model cards, 2025).
Can SmolVLM run on a phone?
Yes. Hugging Face's HuggingSnap app runs SmolVLM2-500M fully on an iPhone, and the 256M and 500M models run in a browser through WebGPU with official ONNX checkpoints.
Is Depth Anything V2 free for commercial use?
Only the Small checkpoint (24.8M parameters) is Apache 2.0; Base and Large are CC-BY-NC-4.0, and the Giant checkpoint has not been released as of October 2026.
Which Video Depth Anything model can I ship commercially?
Video Depth Anything Small (28.4M parameters) is Apache 2.0; the Base and Large checkpoints are CC-BY-NC-4.0 (Video Depth Anything repository, 2025).
Depth Anything V2 or Depth Anything 3?
Use V2 Small for fast per-frame depth from one camera on a phone; use Depth Anything 3 (November 2025) when you need consistent depth and camera poses across many views or a whole video, and ship only its Apache 2.0 checkpoints: Small, Base, Metric-Large or Mono-Large.
What is SmolVLA?
SmolVLA is Hugging Face's ~450M-parameter vision-language-action model for robotics (June 2025, Apache 2.0); it turns camera input and an instruction into robot actions and is built on SmolVLM2-500M, so it is not a general image-to-text VLM.
Free download: On-Device CV Deployment ChecklistOne printable page that maps an on-device vision pipeline's hardware (phone, Jetson, browser) to the right Depth Anything V2 and SmolVLM2 variant, with licence, memory and FPS expectations and a six-item failure-mode audit.Where Fora Soft Fits In
We have shipped on-device CV inside surveillance, e-learning, telemedicine, OTT, and conferencing products since the original MobileNet days, and the move to Depth Anything + small VLMs has changed which questions are scopeable inside which budgets. Three patterns we have used in 2026 client work: depth-aware background blur and replacement that respects real scene geometry instead of the flat 2D segmentation that older video conferencing apps still ship; on-device privacy redaction in e-learning replays, where a small VLM tags frames containing student faces or hand-written content and the redaction runs locally before anything reaches our infrastructure; and edge AI surveillance pipelines for retail and industrial sites, where the camera does the heavy lifting and the cloud only sees alerts. If you are scoping a feature that lives in one of these patterns, the answer in 2026 is often to build on Apache 2.0 checkpoints, provided the licence, memory and accuracy checks above pass on your target device. Our computer vision development services team ships these depth and small-VLM models on phones, laptops and Jetson boards, and our on-device AI mobile app development work handles the Core ML and Android runtime side.
The book
AI for Video: A Complete Guide to Intelligence in Video Products: From Models and Inference to Computer Vision, Speech, Search, and Generative Video
Once you have split a feature into measurement and understanding, the next questions are which deployment topology runs it, which model artifact you ship and how a VLM call compares with a specialist model on cost. Volume 6 of the Fora Soft Video Engineering Handbook works through exactly that: the AI feature lifecycle, edge versus cloud deployment, model artifact formats, and the computer-vision capabilities — detection, tracking, segmentation and pose — that sit next to depth and small VLMs in a real product. The seven volumes together walk the video stack from the pixel to production, so a team that works through them can design and ship an on-device vision pipeline end to end. Written by Nikolay Sapunov, CEO at Fora Soft.
Depth, a small VLM or both — what should run on your device?
Tell us the product question, the camera and the hardware you ship on, and we will tell you which Depth Anything and SmolVLM checkpoints are licensed for it, what fits in memory, where the motion gate goes and what a cloud call would cost instead. 250+ video products shipped since 2005.
See our case studies
Case study
Mindbox
An AI video management system with 50+ deployments whose ANPR module reads 500,000+ licence plates a day at about 95% accuracy — the detect-then-act chain this article builds at the edge.
Case study
NetcamStudio
IP camera surveillance software for homes and offices with motion and sound detection rules, praised for low CPU use — the cheap motion gate that decides which frames earn a model.
Case study
SuperPowerFX
A mobile video-effects app with 500K+ downloads that renders effects on the phone — the on-device video budget that depth-aware effects have to fit.
What to read next
Edge AI
Distillation and quantization for edge video AI
How to shrink a model like SmolVLM2 or Depth Anything to fit a phone or a Jetson.
On-device
MediaPipe selfie segmentation and WebGPU blur
The 2D segmentation baseline that depth-aware background blur improves on.
Foundations
CLIP and contrastive vision-language primer
Where the SigLIP encoder inside SmolVLM comes from.
Fine-tuning
Fine-tuning a VLM on your domain
When a small VLM needs your surveillance or telemedicine data to answer reliably.
References
-
Yang, L., Kang, B., Huang, Z., et al. "Depth Anything V2." NeurIPS 2024. arXiv:2406.09414. arxiv.org/abs/2406.09414. Accessed 2026-10-20. Model sizes, the synthetic-teacher and pseudo-label training pipeline, and the speed comparison with Stable-Diffusion-based depth models.
-
DepthAnything/Depth-Anything-V2. GitHub repository. github.com/DepthAnything/Depth-Anything-V2. Accessed 2026-10-20. Checkpoint licences (Small Apache 2.0; Base and Large CC-BY-NC-4.0), the 518-pixel default input and the metric-depth variants.
-
Chen, S., Guo, H., Zhu, S., et al. "Video Depth Anything: Consistent Depth Estimation for Super-Long Videos." CVPR 2025. arXiv:2501.12375. arxiv.org/abs/2501.12375. Accessed 2026-10-20. The spatial-temporal head that removes frame-to-frame flicker.
-
DepthAnything/Video-Depth-Anything. GitHub repository. github.com/DepthAnything/Video-Depth-Anything. Accessed 2026-10-20. Checkpoint sizes and licences, the 2025 metric models and the streaming mode.
-
Chou, G., Xian, W., Yang, G., et al. "FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolution." ICCV 2025. arXiv:2504.07093. arxiv.org/abs/2504.07093. Accessed 2026-10-20. Real-time depth at 2044 by 1148 and 24 FPS.
-
ByteDance-Seed/Depth-Anything-3. "Depth Anything 3: Recovering the Visual Space from Any Views." arXiv:2511.10647, November 2025; GitHub repository. github.com/ByteDance-Seed/Depth-Anything-3. Accessed 2026-10-20. Model sizes and licences, any-view depth and pose, DA3-Streaming and the 1.1 checkpoints.
-
Marafioti, A., Zohar, O., Farré, M., et al. "SmolVLM: Redefining Small and Efficient Multimodal Models." arXiv:2504.05299, April 2025. arxiv.org/abs/2504.05299. Accessed 2026-10-20. Architecture, pixel-shuffle ratios and single-image memory figures.
-
Hugging Face. "SmolVLM2: Bringing Video Understanding to Every Device." Blog, February 2025. huggingface.co/blog/smolvlm2. Accessed 2026-10-20. The three SmolVLM2 sizes and the HuggingSnap iPhone app running SmolVLM2-500M.
-
HuggingFaceTB/SmolVLM2-2.2B-Instruct. Model card, Hugging Face. huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct. Accessed 2026-10-20. Video-MME scores and video GPU memory for the SmolVLM2 family; Apache 2.0 licence.
-
Hugging Face. "SmolVLA: Efficient Vision-Language-Action Model." Blog, June 2025. huggingface.co/blog/smolvla. Accessed 2026-10-20. SmolVLA size, licence and the SmolVLM2-500M backbone.
-
LiquidAI/LFM2-VL-450M. Model card, Hugging Face. huggingface.co/LiquidAI/LFM2-VL-450M. Accessed 2026-10-20. Parameters, SigLIP2 encoder, native 512 by 512 input and the LFM Open License v1.0.
-
Qwen/Qwen3.5-0.8B. Model card, Hugging Face, March 2026. huggingface.co/Qwen/Qwen3.5-0.8B. Accessed 2026-10-20. Apache 2.0 licence and image-text input.
-
Google. "Gemma 4: Our most intelligent open models to date." 2 April 2026. blog.google/innovation-and-ai/technology/developers-tools/gemma-4/. Accessed 2026-10-20. E2B and E4B sizes, Apache 2.0 licence, video and audio input, offline on phones, Raspberry Pi and Jetson Orin Nano.
-
apple/FastVLM-0.5B. Model card, Hugging Face. huggingface.co/apple/FastVLM-0.5B. Accessed 2026-10-20. Licence: apple-amlr.
