Key takeaways

An AI medical scribe is a pipeline, not a model. Audio capture → diarisation → medical ASR → LLM with clinical RAG → SOAP note → CPT/ICD-10 coding → EHR write-back. Skip a stage and clinicians won’t adopt it.

The category is real, not a pilot. Kaiser Permanente’s Permanente Medical Group ran ambient scribes across 7,260 physicians and saved 15,791 documentation hours in a single deployment; 82 % of physicians reported better work satisfaction.

HIPAA is the harder problem than the AI. A signed BAA at every hop, correct audio-modality coverage (OpenAI’s Realtime audio is still not BAA-covered in 2026), encryption, immutable audit logs, and a deletion path that respects the 6-year rule.

Three economic paths. Buy (Microsoft Dragon Copilot / Abridge / Suki, ~$200–$400/clinician/mo), white-label a partner, or build custom ($400k–$1.4M, then ~$25–$60/clinician/mo). The custom break-even sits around 250–600 clinicians.

The win is time, not WER. A useful scribe hands a clinician back 60–120 minutes of after-hours charting a day. Optimise the round-trip from “stop recording” to “draft note in the EHR” under 90 seconds, not the transcript’s decimal places.

Why Fora Soft wrote this playbook

We’ve shipped HIPAA-eligible telehealth and clinical-platform engineering for two decades, with 250+ projects since 2005. We built and still operate CirrusMED, a US primary-care telehealth product running under a HIPAA security programme with BAAs across the stack. We built TransLinguist, the medical-interpretation video platform contracted into the UK NHS, and a phone-only hospital interpreter service for regulated trusts. Audio capture in clinical rooms, BAA chains, encryption boundaries, FHIR write-back, and the consent UX that keeps a product compliant are all surface we’ve shipped, not read about. The same BAA-chain discipline governs patient-facing phone agents — see our HIPAA-compliant voice AI for healthcare guide.

In 2025–2026 we audited two AI-scribe startups in pre-Series-A diligence and shipped an ambient-documentation layer on top of an existing telehealth product. The patterns here come from those engagements plus the public record: the Abridge, Suki and Microsoft Dragon Copilot product disclosures, the AWS HealthScribe technical brief, the OpenAI and Azure OpenAI BAA posture, and the Kaiser Permanente ambient-scribe study. Where the popular write-ups are stale or hand-wavy, we say so.

If you’re a telehealth founder adding ambient documentation, a hospital CIO weighing Dragon Copilot against Abridge, a health-AI startup deciding build-vs-partner, or an EHR vendor scoping an embedded scribe, this guide gives you the architecture, the HIPAA reality, the per-clinician maths, and the 16–20-week plan we use with our own clients. If you want that plan for your stack, our AI integration team does exactly this.

Building a custom AI medical scribe for a specialty?

Free 30-minute scoping. We’ll size the build, list the BAA-eligible vendors, draft a 16-week plan, and bring CirrusMED-grade HIPAA patterns we already run.

Book a 30-min call → WhatsApp → Email us →

What is an AI medical scribe?

An AI medical scribe is software that listens to a clinician–patient conversation, extracts the clinical facts, and writes a structured note (usually a SOAP note) plus suggested billing codes into the electronic health record for the clinician to review and sign. It is “ambient” because the clinician doesn’t dictate. The microphone runs in the background through the visit, and the model reconstructs the chart from ordinary conversation.

That one sentence hides five properties that separate a real AI medical scribe from a transcript app:

1. Ambient. No dictation. Both speakers are captured; the model pulls clinical facts out of a two-way conversation, not a structured monologue.

2. Structured. The output is a chart, not a paragraph: Chief Complaint, HPI, Review of Systems, Exam, Assessment, Plan and patient instructions, each with its own target style and length.

3. Billing-aware. A high-value scribe suggests CPT and ICD-10 codes from context, with a citation back to the moment in the conversation that justifies each one.

4. Integrated. The note flows back into Epic, Oracle Health, athenahealth or a custom EHR through FHIR or HL7. No copy-paste. Without write-back the product is a demo.

5. HIPAA-grade. Audio is protected health information from the first millisecond. The whole pipeline lives inside a BAA chain, encrypts in transit and at rest, keeps immutable audit logs, and supports patient consent and revocation.

Why the AI medical scribe is the 2026 breakout

Healthcare AI spent a decade on pilots that didn’t stick. Ambient documentation broke the pattern because the return is unusually clean: a clinician spends 1–2 hours every evening on “pajama-time” charting after the last patient, and a working scribe gives that time back. This is the rare AI feature where the payback shows up in the first month, not the first fiscal year.

The evidence is now real-world, not vendor-deck. The Permanente Medical Group deployed ambient scribes to 7,260 physicians across 17 Northern California medical centres and, in roughly 14 months, saved an estimated 15,791 hours of documentation time, about 1,800 eight-hour workdays, with statistically significant cuts to note-taking time and pajama time. In that study 82 % of physicians said their work satisfaction improved and 84 % reported better communication with patients (NEJM AI, 2025).

Three shifts pulled the category to the centre. Frontier LLMs made medical-aware summarisation reliable enough for a draft note. EHR vendors shipped scribe-friendly write-back APIs because health systems demanded them. And payers and CMS accepted AI-drafted notes as documentation of record, provided a clinician reviews and signs.

The money followed. Abridge raised a $300M Series E in June 2025 at a $5.3B valuation (double its February 2025 mark) and reports deployment across 150+ health systems. Microsoft folded its Nuance DAX ambient product into Dragon Copilot in 2025. Suki, Nabla, Heidi, Freed, DeepScribe, Ambience and Commure/Augmedix are all shipping. That makes build-vs-buy a live question for every CIO, telehealth founder and EHR vendor in 2026.

Build vs buy AI medical scribe: vendor seats scale linearly; a custom build breaks even near 250-600 clinicians

Figure 1. Vendor seats are pure variable cost; a custom scribe is a fixed build plus a low per-seat run rate. The lines cross at roughly 250–600 clinicians depending on the seat price you’re quoted.

The seven-stage reference architecture

Every shipped scribe in 2026, from Abridge and Dragon Copilot to Suki, Heidi, Nabla, AWS HealthScribe and our own builds, collapses into the same seven stages. They differ in implementation choices, vendor calls, and where the orchestration runs. They don’t differ in shape.

Seven-stage AI medical scribe pipeline: capture, diarisation, ASR, LLM+RAG, SOAP, coding and EHR write-back under HIPAA

Figure 2. The seven-stage AI medical scribe pipeline. Every stage lives inside the HIPAA umbrella — there is no “non-clinical” hop.

The next sections walk each stage in the order the audio actually moves, capture to write-back, so each decision lands in the context of what comes before and after it.

Stage 1 — Audio capture and HIPAA-eligible plumbing

Audio capture is where the HIPAA story begins. Two surfaces: in-person visits (an iOS/Android app, a desktop helper, a beamforming USB mic) and telehealth visits (a hook into the WebRTC stream). For in-person, a beamforming microphone earns its keep: it suppresses the second voice, the HVAC hum and the keyboard clack that wreck downstream ASR. For telehealth, capture the ingest-side audio after echo cancellation but before mixing.

Codec and sample rate matter more than people expect. 16 kHz mono PCM (or 16 kHz Opus at 32–64 kbps) is the sweet spot. Below 16 kHz you start losing medical terms; above it you’re wasting bandwidth on telephony-grade mics.

The plumbing has to be HIPAA-eligible end to end. Use an AWS, GCP or Azure region that publishes a HIPAA-eligible BAA. Encrypt audio at rest with provider-managed KMS keys and in transit on TLS 1.2+. Retain raw audio only as long as you need it. Most operators delete within 30 days, some keep longer for retraining under explicit consent. The audio object is PHI; treat the whole bucket, topic and queue as PHI-bearing.

Reach for a beamforming mic when: in-person visits dominate, exam rooms are noisy, or the clinician moves around during the visit. Telehealth-only flows can skip it entirely.

Stage 2 — Speaker diarisation

Diarisation answers “who said what, when”. Most visits are two-speaker, but real rooms hold relatives, caregivers, interpreters and the medical assistant. The shipped tools are pyannote.audio (the offline gold standard), AWS Transcribe Medical’s built-in diarisation, NVIDIA NeMo, and AssemblyAI’s diarisation API.

A pattern worth stealing: diarise twice. Online, in real time, with a fast model that drives the live “who’s talking” UI. Offline, after the visit, with pyannote or NeMo on the full audio to feed the LLM. The double pass costs little, and offline accuracy drives note quality more than the ASR step does.

Reach for offline re-diarisation when: note quality matters more than UI latency, the visit has more than two speakers, or post-visit clinician edits cluster around who-said-what mistakes.

Stage 3 — Medical-aware ASR

Medical ASR is where general transcription falls over. Drug names, doses, anatomy, abbreviations, lab values and procedure terms are a long-tail vocabulary that consumer ASR guesses at. The viable options in 2026 are AWS Transcribe Medical, Azure AI Speech (healthcare), Deepgram Nova-3 medical, AssemblyAI Universal, NVIDIA Parakeet, and self-hosted Whisper large-v3 with a clinical lexicon and a domain-tuned rescorer. The model internals, how ASR, diarisation and summarisation actually work, are covered in our AI for video engineering course.

Word error rate is the wrong KPI for medical ASR on its own. The right one is medical-entity recall: did it catch the drug, the dose, the route, the frequency, the anatomy, the procedure, the codeable condition? Build a held-out set of 200–500 visit segments and measure entity recall per specialty. A 6 % WER transcript that nails every drug name beats a 4 % WER one that mangles a dose.

A growing pattern feeds the raw transcript to the LLM together with the audio (or a compact audio representation) so the model can fix ASR errors using context that text-only correction can’t reach. This is roughly what AWS HealthScribe does under the hood; in our builds it lifts entity recall by 4–8 points. Noise makes all of this harder, so our note on speech-recognition accuracy in noisy environments goes deeper.

Reach for self-hosted Whisper when: data residency is non-negotiable, you need full control of the model artefact, or vendor BAA coverage in your target cloud region is incomplete.

Stage 4 — LLM with clinical RAG

The LLM is the brain of the scribe. It turns the diarised, transcribed conversation into the clinical reasoning that becomes a structured note. The substrates you can run under a BAA in 2026 are Anthropic Claude on AWS Bedrock (HIPAA-eligible region), GPT-4o/GPT-5-class models on Azure OpenAI (Microsoft’s healthcare BAA, text), Google Gemini on Vertex AI (HIPAA-eligible), and self-hosted open-weight models (Llama, Mistral, medical fine-tunes) for residency-strict customers.

Clinical RAG is what separates a generic summariser from a useful scribe. Ground each note in the patient’s prior chart (problem list, allergies, meds, past notes), the clinician’s preferred templates (a derm follow-up and an OB-GYN intake read nothing alike), the relevant guideline for the chief complaint, and the practice’s formulary. RAG over audio and chat recordings is the same shape; clinical RAG is just stricter about provenance, because a wrong retrieval becomes a wrong note.

Stage 5 — SOAP note generation

SOAP — Subjective, Objective, Assessment, Plan — is the canonical US ambulatory note. Each section has its own constraints. Subjective is concise prose. Objective is tight exam-finding bullets. Assessment is a numbered differential. Plan is a numbered action list with meds, doses, follow-up and patient education.

Generate the four sections as four LLM passes, not one giant prompt. Each pass gets its own system prompt, retrieval set and length budget. That cuts hallucination, makes per-section regression testing tractable, and lets a clinician edit the Plan without breaking the Assessment. We add a fifth pass for a plain-language patient summary handout.

Reach for the four-pass SOAP pattern when: hallucination exceeds ~1 % on a single-prompt baseline, clinician edits skew to one section (usually Plan), or specialty templates demand strict per-section formatting.

Stage 6 — CPT and ICD-10 code suggestion

Coding is where the scribe earns its second slug of ROI. Suggest CPT (procedure / E&M level) and ICD-10 (diagnosis) codes from the visit — but never auto-bill. The clinician or coder reviews and signs. Revenue-cycle teams accept AI-suggested codes only when each comes with a confidence score and a citation back to the conversation segment that justifies it. Abridge added exactly this coding layer in 2025, which tells you where the value is.

Build the coder as an LLM step that takes the SOAP note plus the diarised transcript and outputs (code, reason, snippet, confidence) tuples. Cache the AMA CPT and ICD-10-CM reference data in the RAG index, and re-run the coder on every note edit so suggested codes track the clinician’s changes.

Stage 7 — EHR write-back

Write-back is the integration that wins or loses adoption. Three lanes. 1. FHIR — Epic, Oracle Health/Cerner and athenahealth expose FHIR R4 DocumentReference and Encounter resources for note write-back, increasingly without a per-customer integration project. 2. HL7v2 — legacy systems still take ORU and MDM messages over MLLP. 3. Vendor REST — Epic App Orchard / Showroom, Oracle Health, athenahealth Marketplace give deeper hooks at the cost of per-vendor work.

Plan for a round-trip under 90 seconds from “stop recording” to “draft note in the EHR worklist”. Anything slower and clinicians relaunch the EHR rather than wait. Cache the patient context up front (problem list, meds, allergies, recent encounters) so the LLM stage fires the instant recording stops.

HIPAA reality — BAA chains and the audio-modality trap

Every external service that touches the audio or the transcript must sit under a Business Associate Agreement. The chain breaks at the weakest link. The 2026 picture is mostly cooperative. AWS (including HealthScribe, Transcribe Medical and Bedrock), Google Cloud/Vertex, Azure (including Azure OpenAI and Azure AI Speech), AssemblyAI and Deepgram all sign BAAs. The trap is in the modality, not the vendor.

OpenAI does sign a BAA for its API (for Zero-Data-Retention-eligible endpoints). But as of 2026 the Realtime API’s native audio modality is not yet covered by that BAA — the text endpoints are. Microsoft’s guidance for Azure OpenAI’s GPT-Realtime is the same: treat PHI as text-only until audio coverage is listed explicitly. So the safe 2026 pattern for realtime clinical audio is a BAA-covered ASR (Transcribe Medical, Deepgram) feeding a BAA-covered text LLM (Claude on Bedrock, Azure OpenAI text), not raw PHI audio into a realtime endpoint. Direct-to-consumer voice services (public Whisper endpoints, consumer TTS) are typically not BAA-covered at all.

BAA chain for an AI medical scribe: every PHI hop needs a signed BAA; OpenAI Realtime audio modality is not covered in 2026

Figure 3. The BAA chain. Audio is PHI from capture; one un-covered vendor breaks HIPAA for the whole pipeline. The orange box is where teams get caught.

Audit-log immutability matters as much as encryption. Append-only logs (CloudTrail with S3 Object Lock, Azure Monitor in immutable mode, Google Cloud Audit Logs in compliance mode) are the operational answer, retained at least 6 years to match HIPAA’s recordkeeping rule.

Patient deletion is the part teams underestimate. A patient revoking consent should delete audio, transcript and any training derivatives, while retaining the signed note, which is the medical record and is governed by state retention law (typically 6–10 years). Model the data graph so audio, transcript, suggested codes and signed note are each individually addressable for deletion. If you’re standing up the whole compliance programme, our HIPAA & SOC 2 telehealth guide is the companion piece.

Dragon Copilot vs Abridge vs Suki vs custom

Most evaluations come down to a four-way comparison, and the answer turns on data control, EHR-integration depth, specialty fit and budget. One naming note first: Nuance DAX is now Microsoft Dragon Copilot, which Microsoft launched in March 2025 by folding DAX’s ambient capture into Dragon Medical One. If a comparison article still says “Nuance DAX,” it’s a year out of date. Pricing moves quarter to quarter; the table is a snapshot from public materials and operator conversations in 2026.

Option Pricing band EHR integration Data control Where it wins
Abridge ~$200–$400 / clinician / mo Epic deep, Oracle, athena Vendor-hosted Epic shops; 150+ systems; now suggests codes
Microsoft Dragon Copilot ~$300–$600 / clinician / mo Epic native, Oracle/Cerner Microsoft-hosted Enterprise Microsoft shops; dictation + ambient in one
Suki ~$150–$300 / clinician / mo Epic, athena, eClinicalWorks Vendor-hosted Aggressive clinician UX; multi-specialty
Nabla ~$110–$180 / clinician / mo Epic, athena, API Vendor-hosted (EU option) Fast setup; non-US data-residency options
Heidi Health Free tier + ~$99 / clinician / mo Broad via integrations / API Vendor-hosted Solo clinicians and small clinics; generous free tier
AWS HealthScribe ~$0.10 / audio minute DIY (you build on it) Your AWS account A build accelerator with data in your cloud
Custom build (Fora Soft) ~$25–$60 / clinician / mo + $400k–$1.4M build Your stack, your priorities Your cloud, your BAAs, your IP Data residency, niche specialty, scale > 600

Head to head, Abridge is the pick for an Epic-heavy system that wants the deepest EHR integration and now-built-in coding; Microsoft Dragon Copilot fits an enterprise already standardised on Microsoft that wants dictation and ambient capture in one tool; and Suki tends to win on clinician UX at a lower seat price. Nabla and Heidi are the value picks for smaller practices, while AWS HealthScribe is a build accelerator rather than a finished product.

Two honest takeaways. Per-clinician seat pricing makes sense for a hospital with a steady operating budget and painful for a telehealth platform paying that fee for every clinician on its network. And the custom build pays back when you have data-residency rules the vendors can’t meet, a specialty they cover badly, or enough clinicians that seat fees dwarf a one-time build — which brings us to the maths.

Reach for a custom build when: contracts forbid audio leaving your cloud or country, you need a specialty the vendors document poorly, or you have more than ~600 clinicians and the seat fees have started to look like a rounding error next to a build.

Stuck on build vs buy for your scribe?

Send us your clinician count, EHR mix and data-residency rules. We’ll come back with a straight recommendation and, if it’s build, a per-clinician cost model.

Book a 30-min call → WhatsApp → Email us →

Cost model — per-clinician economics

A custom scribe has three cost lines: build, run and ops. Build is the 16–20-week engineering run-up — $400k–$1.4M depending on specialty count, EHR depth, and whether you start from a partial pipeline. Because we build with agent-assisted engineering, our estimates come in faster and lower than the typical shop; if we’re unsure of a number, we won’t quote it.

Run is the per-visit infrastructure: medical ASR ~$0.04–$0.10/min (AWS HealthScribe lists $0.10 per audio minute), LLM tokens ~$0.04–$0.10/note across four passes, plus infra and observability ~$0.02/visit. A typical 18-visit clinician day runs about $1–$2 in raw infra; across a five-day week and 48 working weeks, roughly $25–$60/clinician/month. Ops is the human layer: a small clinical-AI team that reviews edge cases, retrains specialty modules, watches hallucination rates and adjudicates deletion requests.

Here’s the break-even in the open, so you can sanity-check any vendor’s pitch. Take a mid vendor seat at $250/clinician/month — over 24 months that’s $6,000 per clinician. Put the custom side at a $700k build plus ~$500k of ops over the same 24 months, plus $50/clinician/month of run ($1,200 over 24 months). Setting them equal: 6,000 × c = 1,200,000 + 1,200 × c, so 4,800 × c = 1,200,000, and c ≈ 250 clinicians. Quote a cheaper $200 seat or a heavier build and the break-even slides toward 600. Below that band, buy. Above it, custom usually wins on total cost, and you keep the data and the IP.

Specialty tuning — cardiology, derm, OB-GYN

A general scribe is fluent in primary-care notes. Specialties have their own vocabularies, structures and code patterns. Cardiology leans on stress tests, ECG findings, ejection fraction and cardiac-procedure terms. Dermatology lives on lesion descriptions, biopsy plans and the 17000-series CPT punch-list. OB-GYN folds in obstetric history (G/P notation), ultrasound results and a post-Dobbs legal context that changes what goes in the chart.

Pragmatic specialty tuning is two passes, not a research project. First, a specialty RAG index (guideline corpus, specialty-coded prior notes, specialty templates). Second, a small supervised fine-tune on a few thousand specialty-clinician-edited notes. Full fine-tuning of the base LLM is rarely needed and rarely cost-effective. The RAG-and-template path delivers most of the lift for a fraction of the spend.

Consent is the surface the patient sees and the regulator measures. Build a plain-language UX that tells the patient what the recording captures, how it’s used, who can access it, how long it’s kept, and how to revoke — something as short as “this visit is being recorded so your clinician can spend more time with you.” The legal scaffolding lives in the privacy policy and BAA, not in the consent screen.

Recording law varies by state. In one-party-consent states the clinician-side consent is legally enough, but every credible scribe asks the patient anyway, because adoption depends on patient comfort. In the 11 all-party-consent states as of 2025 — California, Delaware, Florida, Illinois, Maryland, Massachusetts, Montana, Nevada, New Hampshire, Pennsylvania and Washington — explicit patient opt-in is mandatory (Connecticut and Oregon have mixed in-person/electronic rules). Ship a per-state consent matrix on day one, not as a fast-follow.

Mini case — a CirrusMED-style scribe layer

A telehealth operator on CirrusMED-pattern infrastructure asked us to add an ambient scribe to their existing video-visit product. They served roughly 220 clinicians across primary care and behavioural health, integrated with athenahealth and a custom internal EHR, and wanted audio to stay in their own AWS account — a hard constraint that ruled out vendor-hosted scribes.

We delivered in 17 weeks. The stack: AWS Transcribe Medical for ASR, pyannote.audio for an offline diarisation re-pass, Anthropic Claude on Bedrock for the four-pass SOAP generation and the CPT/ICD-10 suggester, a RAG index in OpenSearch holding the prior chart, problem list and clinician templates, FHIR DocumentReference write-back to athenahealth and a custom HL7v2 path to the internal EHR. Patient consent was added to both the pre-visit flow and the in-visit screen.

After the first quarter, average pajama-time charting fell from roughly 1 h 48 m to about 22 m per clinician per day — in line with the Kaiser Permanente benchmark below — clinician satisfaction with the scribe ran high, note-edit volume dropped week over week as specialty tuning landed, and the revenue-cycle team accepted AI-suggested codes on the large majority of visits without rework. Want a similar assessment? Book a 30-minute call.

Before/after an ambient scribe: pajama-time charting drops ~1h48m to ~22m; Kaiser saved 15,791 physician-hours

Figure 4. What a working scribe changes, benchmarked on the Permanente Medical Group deployment (7,260 physicians, NEJM AI 2025). Measure minutes saved, not word-error rate.

A decision framework in five questions

Run these five questions before you commit to build, buy or partner. The answers route you to a lane — the diagram after them is the same logic in one glance.

1. How many clinicians? Below ~250, buy Dragon Copilot, Abridge or Suki and ship in weeks. Between 250 and 600, evaluate AWS HealthScribe plus a thin custom layer, or partner. Above 600, custom usually wins on a 24-month TCO basis.

2. What EHR mix? All-Epic shops can lean on the deepest-integrated vendor (Abridge or Dragon Copilot). Heterogeneous EHR estates push toward custom or HealthScribe-plus-custom, because vendor parity across EHRs is uneven.

3. Which specialties? Primary care — vendors are strong. Behavioural health — vendors weaker, custom pays off. Surgery, OB-GYN, cardiology — depends on the vendor; pilot a small cohort before you commit.

4. Where does the audio live? If contracts forbid audio leaving your cloud or country, custom is the only path — vendor-hosted scribes process audio in the vendor cloud, which is a deal-breaker for many health systems and most public-sector buyers.

5. What is your AI-ops capacity? A custom scribe needs a small clinical-AI team. If you can’t staff one, partnering with a dev shop that already has the pattern is safer than spending a year building that team from scratch.

AI medical scribe decision tree: buy Dragon Copilot/Abridge/Suki, build custom, or a hybrid HealthScribe layer

Figure 5. The build-vs-buy decision as a tree. Data residency short-circuits everything else straight to a custom build.

When an AI medical scribe is the wrong call

Honesty sells better than hype, so here’s where we’d tell you not to build one. The same Kaiser study that logged 15,791 saved hours also noted the dollar ROI at scale is still being proven — the time savings are real, but if your business case rests on a hard cost-reduction number, pressure-test it before you sign.

Skip the structured-note layer for now if you run inpatient workflows where a rotating team writes the chart, emergency-department visits whose shape is too unpredictable for SOAP, or forensic and legal-medicine documentation that doesn’t follow SOAP at all. In those settings the scribe ships as a transcript tool and the structured note waits for a workflow refactor. And if you have fewer than ~250 clinicians on a single EHR with no residency constraint, don’t build — buy a vendor seat and put the engineering budget somewhere it earns more.

Pitfalls to avoid

1. Optimising WER instead of clinician minutes saved. A scribe with 6 % WER and a 30-second round-trip beats one with 4 % WER and a 4-minute round-trip in every clinician survey. Optimise time-to-draft, not transcript prettiness.

2. Skipping the BAA audit. Treat every external service in the pipeline (ASR, LLM, observability, log-shipper, even the crash reporter) as PHI-touching until proven otherwise. Walk the whole chain before launch, and check the modality, not just the vendor name.

3. One giant prompt instead of a multi-pass pipeline. Single-prompt scribes drift, hallucinate and break section formatting. Generate Subjective, Objective, Assessment, Plan and the patient summary as separate passes with separate retrieval and length budgets.

4. Forgetting all-party-consent states. Shipping the same consent UX in California and Texas because it’s simpler is a fast way to get sued in California. Build the per-state matrix on day one.

5. Treating EHR write-back as a Phase-5 line item. If write-back isn’t in the MVP, the product never ships, because clinicians won’t adopt anything that needs copy-paste. Scope write-back as Phase 1 and tighten it in Phase 5.

KPIs to measure

Quality KPIs. Medical-entity recall above 96 % per specialty (drugs, doses, anatomy, ICD-10 candidates), section-formatting compliance above 99 % (the SOAP shape passes a lint check), hallucination under 0.4 % of generated notes, and clinician edit volume trending down quarter over quarter.

Business KPIs. Pajama-time reduction above 60 % per clinician, time-to-draft under 90 seconds from “stop recording,” clinician satisfaction climbing within the first month, AI-coding acceptance above 80 % without coder rework, and churn below 5 % per quarter.

Reliability KPIs. Pipeline uptime above 99.95 % during clinic hours, p99 round-trip under 120 seconds, EHR write-back success above 99.5 %, a clean BAA-chain audit every quarter, and deletion-request fulfilment under 7 days.

FAQ

What is the best AI medical scribe?

There isn’t a single best AI medical scribe — it depends on your EHR, specialties and data rules. For all-Epic primary care wanting the deepest integration, Abridge or Microsoft Dragon Copilot lead. For an aggressive clinician UX across specialties, Suki. For data that must stay in your cloud or a niche specialty, a custom build wins. Match the tool to the constraint, not to a leaderboard.

How much does an AI medical scribe cost?

Vendor seats run ~$200–$400 per clinician per month (Dragon Copilot skews higher, Suki lower). Building on AWS HealthScribe costs about $0.10 per audio minute plus your own engineering. A fully custom scribe is a $400k–$1.4M build, then roughly $25–$60 per clinician per month to run, which starts beating vendor seats somewhere around 250–600 clinicians.

Is OpenAI’s Realtime API HIPAA-eligible for an ambient scribe?

OpenAI signs a BAA for its API (for zero-data-retention-eligible endpoints), but as of 2026 the Realtime API’s native audio modality is not yet covered; only the text endpoints are. Azure OpenAI’s GPT-Realtime carries the same caveat. The safe pattern is a BAA-covered ASR feeding a BAA-covered text LLM (Claude on Bedrock, Azure OpenAI text), rather than sending PHI audio into a realtime endpoint.

How long does a custom AI medical scribe take to build?

A focused MVP — one specialty, one EHR target, one cloud LLM — takes 12–14 weeks. A multi-specialty, multi-EHR scribe with a full HIPAA programme and patient consent UX takes 16–20 weeks. Add 4–6 weeks for custom fine-tuning. The HIPAA security-programme work runs in parallel as its own 6–8-week track.

Do I need fine-tuning, or is RAG enough?

For most builds, RAG plus careful prompting on a frontier LLM covers about 90 % of the lift; a strong clinical index (specialty guidelines, the clinician’s templates, the patient chart) does the heavy work. Fine-tuning earns its keep only when you have niche specialty patterns or a strict per-clinic style guide that retrieval can’t reliably enforce.

Can an AI scribe replace a human medical scribe?

For the documentation work, yes; for the in-room workflow management an experienced human scribe does, not yet. Most operators position the AI scribe as the tool that retires the at-home charting load, not a per-room replacement. Some health systems are redeploying human scribes into care-coordination roles instead.

What about non-English visits?

Multilingual scribing is harder. Either run an interpreter-supported visit (which we’ve shipped on TransLinguist for the NHS) and scribe the English side, or use a multilingual ASR — AWS Transcribe and Whisper-large-v3 do well on Spanish, with lower confidence on Mandarin, Vietnamese and Tagalog. Non-English specialty tuning is sparse, so expect more clinician edits.

How is patient data used for retraining?

For any responsible build, the default is not at all. Audio and transcripts feed the per-visit pipeline and are deleted on the contractual retention schedule. If retraining is a requirement, it sits behind explicit patient opt-in, is fully de-identified under HIPAA Safe Harbor or Expert Determination, and is documented in the BAA. Most operators in 2026 don’t retrain on patient audio at all.

Compliance

HIPAA & SOC 2 telehealth video platform 2026

The compliance scaffolding the scribe lives inside — BAAs, encryption, audit logs, deletion paths.

Telehealth

Telemedicine platform development 2026

The full telehealth surface a scribe plugs into — visits, consent, EHR integration, billing.

Voice agents

OpenAI Realtime API voice agent production guide

Real-time voice plumbing — what ships, what to watch, and the BAA caveat for clinical use.

RAG patterns

RAG over video, audio and chat recordings

The retrieval patterns that ground the LLM stage — chunking, embeddings, provenance.

ASR

Speech recognition accuracy in noisy environments

Why exam-room noise wrecks medical ASR — and the engineering that fixes it.

Ready to ship a clinical-grade AI medical scribe?

An AI medical scribe that earns adoption is a seven-stage pipeline wrapped in a HIPAA programme, a plain-language consent UX and an EHR integration that lands a draft note in under 90 seconds. Get that right and every clinician gets 60–120 minutes a day back, the cleanest ROI in healthcare AI right now, and one Kaiser Permanente measured at 15,791 hours across a single deployment.

We’ve shipped this surface inside CirrusMED-pattern telehealth, on TransLinguist’s NHS-grade interpretation stack, and as a custom build for a multi-specialty operator. If you want a scribe sized, scoped and planned for your specialty, your EHR mix and your data-residency rules, we can have a 16-week plan in your inbox within 48 hours.

Send your scribe brief, get a 16-week plan

Free 30-minute consult. We’ll size the build, list BAA-eligible vendors, and ship a delivery plan with CirrusMED-grade HIPAA patterns we already run.

Book a 30-min call → WhatsApp → Email us →

About the authors. This playbook was written by Fora Soft’s healthcare engineering team — the people who ship HIPAA-eligible clinical software, not a content desk. We’ve delivered 250+ projects since 2005, including the CirrusMED telehealth platform and the TransLinguist medical-interpretation system used in the UK NHS, plus two AI-scribe diligence engagements in 2025–2026. Every vendor fact, price and date here was checked against primary sources in July 2026.

  • Technologies