By Dr. Sina Bari
The FDA and the Future of Generative AI in Clinical Care: Why Ambient Scribes Can’t Fly Under the Radar Anymore
By Dr. Sina Bari
Lately, I’ve been spending more time talking to providers, hospital CIOs, and startup founders about a topic that’s about to get very real for anyone building generative AI tools for clinical settings: regulation.
Specifically, I’ve been getting asked the same question in different forms:
“Do we need FDA clearance for our ambient AI scribe?”
My short answer: probably, yes. But as with everything in medicine, the long answer is more nuanced—and more important to understand.
Ambient Scribes Aren’t Just Note-Taking Tools
At first glance, a generative AI that listens to a clinical conversation and drafts a SOAP note might seem like a fancy dictation tool. After all, human scribes aren’t regulated. Neither are EHR text boxes nor voice recorders. But that analogy doesn’t hold up under scrutiny.
Ambient AI scribes don’t just transcribe—they interpret. They summarize, prioritize, and sometimes invent. And because these notes often become part of the legal medical record, any error or omission can directly affect patient care. If a hallucinated symptom or misdocumented dose gets into the note and is acted on, that’s not a clerical error. That’s a clinical one.
So it’s not surprising that the FDA has started to view these tools as Software as a Medical Device (SaMD)—and therefore subject to regulation.
How the FDA Decides What to Regulate
Under current law, the FDA doesn’t regulate all software used in healthcare. They focus on tools intended to diagnose, treat, or prevent disease. That means administrative tools like billing software or appointment reminders are outside the FDA’s scope.
But once software crosses into territory where it influences clinical decision-making—or becomes part of the clinical record—it enters device territory. And this is where ambient scribes often land, especially when they summarize the clinical encounter and generate structured documentation used by other clinicians downstream.
The FDA also has a helpful 4-part test from its Clinical Decision Support (CDS) guidance to determine whether software can be exempted from regulation. Most ambient scribes fail at least one part of that test, particularly the requirement that a clinician must be able to review the basis of the AI’s output independently. Let’s be real: no one’s going back and listening to the whole patient conversation just to cross-check a note meant to save time.
No, There’s Not a Predicate Yet
Many people assume they can go the 510(k) route—demonstrating their device is “substantially equivalent” to something already cleared. The problem? The FDA has cleared no ambient AI scribe using a large language model. That means the first one through the door will likely need to go the De Novo pathway, designed for novel, moderate-risk devices.
Once that happens, others might be able to piggyback using 510(k) submissions. But until then, if you’re building one of these tools, you should be preparing for De Novo classification—meaning validation studies, clinical performance benchmarks, risk mitigation strategies, and all the paperwork that comes with it.
Why This Matters for the Whole Ecosystem
There are over 100 AI scribe tools on the market right now. Not one has FDA clearance. That’s a regulatory time bomb. When one tool gets cleared (and it will happen—some vendors are already in conversations with the agency), every health system CIO will start asking why the others haven’t.
It’s not just about compliance—it’s about trust. Would you rather roll out a tool that’s been through the FDA’s safety and efficacy filter, or one that hasn’t? Providers are already skeptical of AI. Giving them tools with a regulatory seal of approval changes the conversation.
From my vantage point, running a medical AI division that supports annotation for ambient scribe training data, I’ve seen just how variable these models can be—and how small hallucinations can have big downstream impacts. We’ve already started working with clients on performance benchmarking, dataset validation, and clinical QA pipelines to prepare for the scrutiny that comes with a regulated device.
What to Do If You’re Building in This Space
If you’re developing an ambient scribe or similar clinical generative AI, here’s what I’d recommend based on the latest FDA guidance:
- Engage early: Set up a pre-submission meeting with the FDA. Get their perspective on your intended use, risk profile, and likely classification. Build your PCCP: The FDA is now accepting Predetermined Change Control Plans that let you pre-authorize certain types of model updates. Take advantage of that.
- Collect validation data now: Benchmark your AI-generated notes against human scribe notes across specialties, patient populations, and encounter types.
- Design for clinician oversight: FDA will want to see that your product doesn’t turn providers into passive recipients of AI output. Make sure there’s visibility into what the model “heard” and why it generated what it did.
- Prepare for post-market surveillance: Even after clearance, you’ll need to monitor for errors, document adverse events, and potentially update your model under FDA-approved protocols.
Final Thoughts
The idea that we can deploy LLMs into live clinical workflows without any oversight was always too good to be true. Generative AI is powerful, but power in medicine demands accountability. And that’s what the FDA is gearing up to enforce.
Ambient scribes aren’t exempt just because they’re cool. If anything, their potential impact makes them more deserving of thoughtful regulation. The good news? The FDA is listening. Their latest guidance shows a real attempt to modernize how software gets reviewed, especially in the age of adaptive, learning systems.
The onus is on us—builders, clinicians, data scientists—to meet that bar. Because when done right, these tools can truly reduce burnout and improve care. But only if we’re honest about their risks, and rigorous in how we manage them.
Dr. Sina Bari is the Senior Director of Medical AI at iMerit, where he leads a team specializing in clinical NLP, diagnostic imaging, and regulatory-compliant data pipelines for AI in healthcare. A Stanford-trained plastic surgeon, he writes about the intersection of medicine and technology.