Dating, Social, and UGC App Development — The AI Engineering Playbook

Published 2026-06-03 · 27 min read · By Nikolay Sapunov, CEO at Fora Soft

Why this matters

If you are a founder, product manager, or operations lead scoping a dating app, a social network, or any product where users post photos, video, or live streams, you will be told that "AI" is the matching algorithm and that safety is something the moderation team handles later. Both halves of that sentence are wrong, and both are expensive. The matching algorithm is the easy, well-understood part; the part that gets companies fined, delisted from the app stores, or sued is the safety system — and in 2026 that system carries legal duties that did not exist two years ago. This article is the plain-language map a non-technical decision-maker needs to scope the work, brief engineers, choose a development partner, and avoid the two mistakes that quietly kill these products: shipping a single content classifier and trusting its verdict, and launching to the public without the moderation and reporting machinery that the law now requires.

What "AI in a dating, social, or UGC app" actually means

When someone says "we want AI in our social app," they usually mean one of three very different things, and the first job of any serious development team is to find out which — because they share almost no code, no cost curve, and no risk profile.

The first kind is matching and ranking: software that decides who or what to put in front of each user. In a dating app this is the recommendation queue — who you see next. In a social feed it is the order of posts. On a UGC platform it is what gets discovered. Under the hood this is a recommendation system: software that learns a numeric fingerprint, called an embedding, for each user and each item, then finds the items whose fingerprint best matches yours. This is the "algorithm" everyone talks about, and it is the most mature and least risky of the three.

The second kind is trust and safety: software that keeps harmful content and harmful people out. This means detecting illegal images and video, nudity and violence, hate speech and harassment, spam and scams, fake profiles and bots, and — increasingly — AI-generated fakes of real people. It also means verifying that a user is a real, present human of the age they claim. This is the feature that does not make the demo but does decide whether the product survives contact with real users and real regulators. We will spend most of this article here.

The third kind is real-time and generative media: software that improves the video and audio users send and create. For a video-dating or live-social product this is the same family of camera and microphone features used in conferencing — background blur, beauty and augmented-reality effects, noise suppression, live captions. For creators it is editing assistance and generative content. This work is technically rich but well-trodden, and it borrows directly from the conferencing and streaming worlds.

Figure 1. The three jobs people call "AI" in a social product. They share almost no code, cost curve, or risk profile — so scope them as three systems, not one feature.

The reason this distinction matters on day one is risk and money. Matching is mature cloud computation that costs fractions of a cent per user and carries mostly reputational, not legal, risk. Real-time media runs on the user's device and costs almost nothing to operate. Trust and safety runs everywhere — on the device, in the cloud, and in a room full of human reviewers — costs a real per-item fee at scale, and is the one category where getting it wrong is now a regulated offence. Treating "the AI" as one feature is the first and most expensive mistake.

The two lines that decide your whole build

Before any code, two lines run through every decision in a social product. Most failed or fined projects crossed one of them without noticing.

The first is the moderation line. A content classifier does not know whether an image is harmful; it produces a score — its best guess that the image belongs to a category like "nudity" or "violence." That score is usually right and sometimes badly wrong, and crucially, no single threshold is right for every kind of harm. Cross this line — treat one model's score as the final verdict — and you ship a system that either floods users with false accusations or lets real harm through, and often both at once. The honest design accepts that the model is one voice in a layered process, not the judge.

The second is the platform line. There is a legal border between being treated as a neutral pipe that merely carries bits and being treated as a platform that hosts and recommends other people's content. You do not cross it by changing your technology. You cross it by reach and audience: once enough people use your service, and especially once minors can access it, a stack of affirmative legal duties switches on — proactively assessing risk, removing illegal content fast, verifying age, reporting child-abuse material, and publishing transparency reports. An app that says "we just connect people" is not exempt; the law looks at what your service does and who uses it, not at how you describe it.

Figure 2. The moderation line and the platform line. Engineering lives below the first; your legal status is set by which side of the second your reach and audience put you on.

Hold these two lines in mind as we go, because every recommendation below is really one of them in disguise: the model scores, the system and a human decide (respect the moderation line), and reach and minors trigger duties, deliberately design for them (respect the platform line).

Where the AI runs: cloud, device, or the human-review room

The most consequential architecture decision is where each piece of AI executes, because the place the computation happens decides latency, privacy, cost, and legal exposure all at once. The three feature groups pull in different directions, exactly as they do in any real-time video product. (We cover the general version of this trade-off in our guide to latency, deployment, and real-time vs batch.)

Matching and ranking want to run in the cloud. Deciding who to show next is not time-critical to the millisecond — a few hundred milliseconds is fine — and the models that do it well are far too large and data-hungry for a phone. This work splits naturally into two stages: a fast retrieval stage that narrows millions of candidates down to a few hundred using the embedding fingerprints, and a heavier ranking stage that orders those few hundred by predicted engagement. Both live on servers, close to the data.

Trust and safety runs everywhere, and that is the point. Fingerprint-matching of known-illegal content and the proactive machine-learning classifiers run in the cloud at upload time, where they can be heavy and accurate. Some checks — a live-video nudity filter in a dating call, or capturing a verification selfie — run on the device for speed and privacy. And the hardest cases run in the human-review room, because some judgements still need a person. A safety system that exists only in one of these places is incomplete.

Real-time media wants to run on the device. Background blur, beauty effects, and noise suppression in a live video call are real-time video, and round-tripping every frame to a server adds delay the user feels as lag, besides streaming intimate video off the device. The modern models for this are built to run on the phone, which is why this work belongs at the edge. (Our video-conferencing engineering playbook covers these live features in depth.)

Figure 3. Where the AI runs. Most real products are hybrids: ranking and heavy moderation in the cloud, live media on the device, the hard safety calls with human reviewers.

The 2026 default is therefore a hybrid, with one twist specific to social products: the human-review room is not an afterthought, it is part of the architecture. Budget it, staff it, and build the tooling for it from the start, because the law now assumes it exists.

The hard part: moderation is a funnel, not a filter

Here is the system that decides whether the product survives, and the one most teams underbuild. The instinct is to picture a filter: content goes in, a model says yes or no, clean content comes out. Real trust and safety looks nothing like that. It is a funnel with several layers, each catching what the layer before it missed, because no single method catches everything.

The first layer catches the known-illegal content, and it does not use a guessing model at all. It uses fingerprint-matching: every image and video is reduced to a short, stable code — a perceptual hash — that survives resizing or light editing, and that code is compared against databases of fingerprints of content already verified as illegal. Microsoft's PhotoDNA and Meta's PDQ are the standard image fingerprints; Thorn's Safer service reports it detected nearly 1.5 million images and videos of known child sexual abuse material in 2025 alone by matching against a database that grew to 6.3 million image hashes and 64 million video hashes. Fingerprint-matching is fast, cheap, and near-exact, so its hits can be acted on automatically — block the upload, preserve the evidence, and file a report.

The second layer catches new harmful content that no fingerprint exists for yet, and this is where the machine-learning classifiers live — models that score an image, a video frame, or a passage of text against categories like nudity, violence, hate, self-harm, or weapons. These models are good but never perfect, which is the entire reason the funnel has more layers below them. A classifier is a smoke detector, not a judge: it is tuned to be sensitive, and it hands its alarms to the next layer rather than passing final sentence.

The third layer is user reports, the cheapest and often most accurate signal you have, because humans recognise context — sarcasm, grooming, coordinated harassment — that models miss. The fourth layer is human review: trained people who look at what the upstream layers flagged and make the call the model could not. And a fifth layer, appeals, lets a wrongly-actioned user contest the decision — which is no longer optional politeness but a legal requirement in the European Union.

Figure 4. The moderation funnel. Each layer catches what the one above missed; severity decides which items get auto-actioned and which go to a human.

Pitfall — shipping a single classifier and trusting its verdict. The most common failure in social products is to wire one moderation model to an automatic ban-or-allow decision and call it done. Two things then happen. Innocent users get auto-banned because the model misfired on a beach photo or a breastfeeding image, and they have no way to appeal — which, in the EU, is itself a violation. Meanwhile real harm slips through because the model had never seen that exact attack before. The fix is not a better single model; it is the funnel — fingerprints for the known, classifiers for the new, reports and humans for the judgement, and a threshold tuned to each harm's severity.

The severity tuning is the part that separates a real system from a demo. You do not use one strictness setting for everything. For child sexual abuse material, the policy is near-zero-tolerance: maximise the chance of catching it, accept more false alarms, auto-block on a fingerprint hit, and report it — because in the United States, providers are legally required to report apparent child sexual abuse material to the National Center for Missing and Exploited Children. For spam, you can afford to be lighter-handed, because the cost of a missed spam message is far lower than the cost of a missed abuse image. The same architecture, tuned differently per harm, is the whole craft.

A worked example: the live-video sampling budget

Live video is where the funnel meets a hard cost wall, and the arithmetic is simple enough to do on a napkin — which is exactly why every team should do it before promising "we moderate every stream."

A live video runs at 30 frames per second. One hour of it is therefore a large number of images:

60 min × 60 s × 30 frames = 108,000 frames per hour

Now suppose you moderate every frame with a cloud image API at a typical rate of $1.00 per 1,000 images:

108,000 frames ÷ 1,000 × $1.00 = $108.00 per hour, per stream

A hundred and eight dollars an hour for a single stream is absurd the moment you have a thousand concurrent streams. So nobody moderates every frame. Instead you sample — check one frame every couple of seconds:

3,600 s ÷ 2 s = 1,800 frames per hour
1,800 frames ÷ 1,000 × $1.00 = $1.80 per hour, per stream

That is sixty times cheaper, and it is why frame sampling is the standard design for live moderation. The cost of sampling is a deliberate trade: a violation that appears can run for up to about two seconds before a sampled frame catches it. For most policies that is acceptable when paired with human verification — a common rule is to sample every one to two minutes and aim to stop an offending stream within three minutes. For a nudity filter in a one-to-one dating call you sample far faster. The point is that "we moderate live video" is never a yes/no capability; it is a sampling rate you choose against a cost and a service-level promise. Modern systems push the detection itself down to two to five seconds by running the sampled frames and the audio transcript through classifiers in parallel.

The same base-rate arithmetic governs the human-review room. Imagine a million uploads a day, of which a small fraction — say half a percent, or 5,000 — actually break a policy. A classifier with 90% recall catches 4,500 of them and misses 500, which is exactly why you still need user reports and re-scans. If that classifier flags items at 95% precision, then for every 4,500 true hits it also flags about 237 innocent items, and all of them — roughly 4,737 a day — land in front of human reviewers. At ten seconds per review that is over thirteen hours of work, close to two full moderator-shifts, for this one harm category alone. That number is why you auto-action the unambiguous tier, tune thresholds per harm, and treat human review as a standing cost rather than a surprise.

The platform line: when an app becomes a regulated platform

This is the section to read twice, because it is where a launch decision quietly creates legal exposure that engineering never signed off on. Three regimes matter most in 2026, and a fourth is coming.

In the European Union, the Digital Services Act governs any service that hosts and shares user content. It requires a working notice-and-action system so anyone can flag illegal content, a statement of reasons every time you remove something or suspend an account, an appeals process for users who disagree, and — for the largest platforms — annual risk assessments and transparency reports. The reports are now standardised: since the first harmonised reporting period, platforms publish machine-readable numbers on how much content they removed and how accurate their automated systems are. The penalty for non-compliance is up to 6% of global annual turnover, and enforcement is no longer theoretical — the Commission issued a €200 million fine to the marketplace Temu, and a second enforcement wave in 2026 extends the duties to a far broader set of companies than the giants.

In the United Kingdom, the Online Safety Act requires services likely to be accessed by children to run "highly effective" age checks and to proactively tackle illegal content. The regulator, Ofcom, is enforcing: by early 2026 it had opened investigations into more than ninety services and issued fines, including an £800,000 penalty against one platform for failing to put age checks in place. Its stated priorities include protecting children and protecting women and girls from intimate-image abuse — a direct signal to dating and social products.

In the United States, the TAKE IT DOWN Act, signed in May 2025, makes it a federal crime to publish non-consensual intimate imagery, including AI-generated deepfakes of identifiable real people. For platforms, the operative duty is a notice-and-removal clock: once you receive a valid request from a victim, you must remove the content — and make a good-faith effort to remove copies of it — within 48 hours. The Federal Trade Commission began enforcing this requirement in May 2026, so any product that hosts user images needs the removal workflow built and staffed, not planned.

The fourth regime is the EU AI Act's Article 50, which takes full effect in August 2026 and requires any system that produces synthetic images, video, or audio to mark those outputs in a machine-detectable way. If your product has a generative feature, or even just lets users post AI-made content, this lands on you. We cover the whole regulatory machine in detail in the regulatory-engineering article, and the disclosure-and-provenance side in the C2PA and AI-disclosure engineering guide.

One more boundary deserves a flag because it moves: the EU's proposed CSAM Regulation, nicknamed "Chat Control," would have required messaging services to scan private messages for child-abuse material. The voluntary version that let platforms scan lapsed in 2026 and the mandatory-scanning plan was dropped from the working draft, with negotiations continuing and the interim rules extended. Treat this as unsettled and watch it, rather than building to a version that may not survive.

Verifying a real human: age, liveness, and deepfakes

Two of the hardest 2026 problems sit at the front door: proving a user is the age they claim, and proving they are a real, present person rather than a stolen photo or an AI fake.

Age assurance has moved from a checkbox to a measured capability. The UK Online Safety Act demands checks that are "highly effective," which rules out the old "click to confirm you are 18" button. The two workable approaches are document verification (checking a government ID) and facial age estimation — a model that estimates age from a selfie without identifying the person. The independent benchmarks are good enough to lean on: leading facial-age-estimation systems correctly classify the large majority of teenagers as under-21, with a mean error of little more than a year for young adults, and the better implementations process the image and immediately discard it rather than storing a face. The engineering choice is to estimate first, cheaply and privately, and fall back to document checks only for the ambiguous band near the threshold.

Liveness and identity is the dating-app front line, and the industry moved fast. In late 2025 Match Group made Face Check mandatory for new Tinder users in a growing list of markets: a user records a short video selfie, and the system checks two things — that the face belongs to a live human present at that moment rather than a static photo or a recording, and that it matches the profile photos. The point is to stop accounts built on stolen or AI-generated faces. Match Group reported that markets already using the feature saw a 40% drop in "bad actor" reports, and said it would extend the check to its other apps through 2026. The face-detection and biometrics engineering behind this, and the consent rules that bind it, are covered in our face detection under the EU AI Act article — biometric data is special-category data, so liveness checks need explicit consent and careful handling.

The deeper threat is deepfakes and AI-generated profiles, and it is now a measurable harm rather than a hypothetical. Romance scams cost US consumers $1.16 billion in just the first nine months of 2025, and the long-game "pig butchering" investment frauds that often start on dating apps are estimated to have drained around $10 billion from Americans in a year. AI is the force multiplier: a single operation can run hundreds of fake profiles through chatbots that hold natural conversations for weeks, and a 2025 survey found that fewer than half of online daters could tell a real profile photo from an AI-generated one. Defending against this is a blend of the liveness check above, behavioural signals (a profile that messages a thousand people in an hour is not a person), and content provenance.

That last piece — provenance — is worth understanding because it is widely misunderstood. The C2PA Content Credentials standard attaches a signed record to a media file describing who made it, with what tool, and what edits were applied. By early 2026 over six thousand organisations had joined, including Google, Meta, OpenAI, and major camera makers. But C2PA does not detect a deepfake; it only proves provenance when a credential is present, and most AI images arriving on a social platform have no credential at all, because uploading often strips it. Provenance is a useful signal in a layered defence, not a deepfake detector — and treating it as one is its own version of the moderation-line mistake.

Three ways to build the safety stack

Once you know which harms you must catch and where the checks run, there are three ways to actually build the trust-and-safety machinery. They trade speed of delivery against control, exactly as they do in conferencing or surveillance.

The first route is to buy a moderation API. Vendors such as Hive, Amazon Rekognition, Microsoft Azure AI Content Safety, and Sightengine expose nudity, violence, hate, drugs, weapons, and self-harm detection behind a simple call, often with live-video support. You can have content scanning running in days. The trade-off is that you accept the vendor's categories and accuracy, and you pay per item — Rekognition, for example, prices image moderation around $1.00 per 1,000 images and video at about $0.10 per minute, while Sightengine starts near $29 a month for ten thousand operations. Crucially, general moderation APIs do not handle child-abuse-material matching; that requires a specialist.

The second route is to assemble your own stack on top of open models and shared databases — your own classifiers or open vision-language models for the general categories, perceptual-hash matching against PhotoDNA and PDQ for the known-illegal tier, a specialist service such as Thorn's Safer for child-safety detection and NCMEC reporting, and your own policy engine and human-review tooling on top. This takes weeks to months and means you own the policy logic, the thresholds, and the data flow. It is the right choice once you have enough scale that per-item API fees hurt, or once your harms are specific to your community.

The third route is to build and train your own models for harms the off-the-shelf systems handle poorly — a niche form of harassment specific to your platform, a community-specific policy, a language the vendors cover badly. This is months of work and only pays off when trust and safety is the product or a core differentiator. For most teams it is premature; the question of when a general vision-language model beats a custom classifier is the subject of our VLM-versus-custom-CV article.

Figure 5. Buy to ship in days, assemble to own the policy and the data flow, build only when trust and safety is the differentiator.

A comparison makes the trade-off concrete:

For most teams scoping a first public version, the honest path is to buy a general moderation API to launch, add a specialist child-safety service from day one regardless of route (this is non-negotiable, not a scaling concern), and assemble more of the stack as volume and policy complexity grow.

Where the matching algorithm actually fits

It is worth returning briefly to the part everyone assumes is the whole job, because scoping it correctly frees budget for the part that is not. Matching and ranking is a solved shape: the two-tower retrieval model that learns user and item embeddings, an approximate-nearest-neighbour search that pulls a few hundred candidates from millions in milliseconds, and a ranking model that orders them by predicted engagement. The open-sourced recommendation systems from large platforms all follow this pattern, and a competent team can stand up a credible version of it without inventing anything.

The mistake is not technical, it is editorial: pouring the AI budget into squeezing another percent of engagement out of the ranker while the safety funnel and the verification front door go underbuilt. In a dating, social, or UGC product, the ranking model determines how good the experience feels; the safety system determines whether the product is allowed to exist. Spend accordingly.

What a dating or social app actually costs to build

Budgets vary with scope, but 2026 market anchors help planning conversations. A minimum viable product — profiles, matching or a feed, messaging, basic reporting — generally runs in the low tens of thousands of dollars for a single platform. A competitive app with real-time video, a tuned recommendation system, identity verification, and a proper moderation stack runs from roughly $80,000 into the several hundreds of thousands, depending on how much of the safety stack you build versus buy. The search demand confirms this is a buyer's market: "dating app development company" alone draws on the order of a thousand US searches a month at low competition, which is why so many agencies court it.

Two cost lines are routinely underestimated. The first is moderation as an operating cost, not a build cost: every image and every minute of video carries a per-item inference fee if you buy, plus the standing salary cost of the human-review room, and both scale with success rather than shrinking. The second is compliance engineering: the notice-and-action system, the statement-of-reasons generator, the appeals flow, the age-assurance integration, and the transparency-reporting pipeline are real features with real timelines, and in 2026 they are not optional for a public launch in the EU or UK. A plan that books them as "legal will handle it later" has mis-scoped the build.

Where Fora Soft fits in

Fora Soft has built video software since 2005, across video conferencing, streaming and OTT, e-learning, telemedicine, and computer-vision surveillance — and a dating, social, or UGC product borrows from all of them. Live and one-to-one video is the same real-time WebRTC work we do for conferencing; user-generated video delivery is the same streaming and OTT work; and the moderation funnel is the same computer-vision detection engineering we apply in surveillance, where the rule has always been that the model flags and a human verifies. The discipline of handling sensitive personal data with explicit consent and careful retention is the same one we hold to in telemedicine. That cross-domain experience is what keeps a social build from treating safety as a feature bolted on at the end, instead of the layered system it has to be from day one.

What to read next

• Real-time content moderation in an SFU
• AI security camera + intelligent video analytics — engineering playbook
• Regulatory engineering — EU AI Act, Article 50, and biometrics

Talk to us / See our work / Download

• Talk to a video engineer — scope your matching system, moderation funnel, and verification front door with a team that has shipped real-time video since 2005. → /services/ai-software-development
• See our case studies — explore Fora Soft's work across conferencing, streaming, computer vision, and telemedicine. → /cases
• Download the Dating, Social & UGC Trust-and-Safety Engineering Decision Sheet — the three AI systems, the two lines, the moderation funnel, the live-video sampling math, the four regulatory regimes, and the build-vs-buy split on one page. → Download the decision sheet

References

1. Regulation (EU) 2022/2065 — Digital Services Act. European Commission. https://digital-strategy.ec.europa.eu/en/policies/digital-services-act — Tier 1 (statute). Defines hosting/platform duties: notice-and-action, statement of reasons, internal complaint handling and appeals, and (for VLOPs) annual risk assessments and transparency reports; penalties up to 6% of global annual turnover. The controlling EU source for the platform line.
2. European Commission — DSA transparency reporting and enforcement (harmonised reports Feb 2026; €200M Temu fine; X proceedings). https://digital-strategy.ec.europa.eu/en/policies/dsa-brings-transparency — Tier 1/Tier 4. Machine-readable transparency templates in force since 1 July 2025; first harmonised reports include content-removal counts and automated-system accuracy/error rates; €200M fine to Temu; second enforcement wave in 2026 broadens scope beyond VLOPs.
3. UK Online Safety Act 2023 + Ofcom enforcement (Feb 2026: 90+ investigations, £800,000 Kick fine). Ofcom. https://www.ofcom.org.uk/online-safety/illegal-and-harmful-content/roadmap-to-regulation — Tier 1/Tier 4. Requires "highly effective" age assurance for services likely accessed by children and proactive illegal-content duties; Ofcom priorities include protecting children and tackling intimate-image abuse against women and girls.
4. TAKE IT DOWN Act (Pub. L. 119-12, signed 19 May 2025). U.S. Congress / Congressional Research Service LSB11314. https://www.congress.gov/crs-product/LSB11314 — Tier 1 (statute). Criminalises non-consensual intimate imagery including AI deepfakes of identifiable people; requires platforms to remove validly reported NCII (and good-faith duplicate removal) within 48 hours; FTC enforcement of notice-and-removal began 19 May 2026.
5. EU AI Act (Regulation (EU) 2024/1689) — Article 50 transparency for synthetic content (full effect 2 Aug 2026). European Union. https://eur-lex.europa.eu/eli/reg/2024/1689/oj — Tier 1 (statute). Requires machine-readable marking of AI-generated or AI-manipulated image, audio, and video. Applies to generative features and AI-content hosting.
6. Thorn — Safer 2025 Impact Report (CSAM hashing & matching; classifiers for novel content). Thorn. https://www.thorn.org/blog/safer-impact-report-2026/ — Tier 4 (NGO/vendor). ~1.5M known-CSAM images/videos detected in 2025; SaferHash image hashes grew to 6.3M, video hashes to 64M; perceptual hashing (PhotoDNA, PDQ, SaferHash) finds known content, classifiers find novel content; reporting routes to NCMEC.
7. Match Group / Tinder — Face Check facial verification (US rollout, Oct 2025). Tinder Press Room. https://www.tinderpressroom.com/2025-10-22-Tinder-to-Expand-Facial-Verification-Feature-Across-the-U-S- — Tier 4 (vendor). Short video selfie checks liveness (live human vs static image/recording) and match to profile photos to block deepfaked-identity accounts; markets using it saw a reported 40% drop in bad-actor reports; extending to other Match Group apps through 2026.
8. U.S. Federal Trade Commission — romance-scam losses ($1.16B, Jan–Sep 2025); CFTC/FBI pig-butchering estimates (~$10B/yr). FTC Consumer Sentinel. https://www.ftc.gov/news-events/data-visualizations/data-spotlight — Tier 1 (regulator data). Quantifies the fraud the verification front door defends against; AI chatbots run hundreds of fake profiles, and a 2025 survey found fewer than half of online daters could distinguish a real profile photo from an AI-generated one.
9. Yoti — Facial Age Estimation white paper + NIST FATE evaluation. Yoti / NIST. https://www.yoti.com/blog/yoti-age-estimation-white-paper/ — Tier 4/Tier 1. Independent NIST benchmarking; high true-positive rate classifying 13–17-year-olds as under-21 and mean absolute error of ~1.2 years for young adults; privacy-preserving (image discarded after estimation). The basis for "estimate first, document-check the ambiguous band."
10. C2PA — Coalition for Content Provenance and Authenticity, Content Credentials specification (6,000+ members by Jan 2026). C2PA. https://c2pa.org/specifications/ — Tier 1/Tier 4 (open standard). Cryptographically signed provenance (creator, tool, edits); proves provenance only when a credential is present and does not detect deepfakes; credentials are frequently stripped on upload, so provenance is one signal in a layered defence, not a detector.
11. AWS Rekognition, Azure AI Content Safety, Hive, Sightengine — content-moderation API landscape & pricing (2026). Vendor documentation. https://aws.amazon.com/rekognition/pricing/ — Tier 4 (vendor). Rekognition ~$1.00/1,000 images and ~$0.10/min video; Azure AI Content Safety covers hate/sexual/violence/self-harm with configurable severity and custom categories; Sightengine from ~$29/mo; general APIs do not include CSAM matching. The "buy" build route.
12. Live-stream moderation by frame sampling (AWS IVS + Rekognition pattern; 2–5 s multimodal detection). AWS Machine Learning Blog. https://aws.amazon.com/blogs/machine-learning/moderate-your-amazon-ivs-live-stream-using-amazon-rekognition/ — Tier 4 (vendor). Sample frames rather than moderate every frame; sampling cadence is set by the Trust & Safety SLA; multimodal classifiers (frames + audio transcript) detect violations within 2–5 seconds; hybrid auto-stop plus human verification.

Standards-citation note: this is a vertical/regulatory playbook with no codec, container, color space, or quality-metric content, so the codec-spec obligation in the house rules is not applicable. The article still anchors on five official/regulatory primaries (DSA, UK Online Safety Act, TAKE IT DOWN Act, EU AI Act Article 50, and FTC data) plus the C2PA open standard, exceeding the three-official-source minimum.