I remember standing across the table from a second-year resident during a complex flap revision. The resident had just described the surgical plan: elevate the flap, identify the perforator, revise the inset. Technically correct in every detail. I asked, "What are you going to do if the perforator is thrombosed when you get there?" The resident paused. "I would... look for another one." "Where?" Another pause. That silence was the most important moment in the case, because it exposed the gap between knowing the steps and understanding the decision tree when things go wrong. That gap is exactly the problem we face with clinical AI systems today. But here is the part nobody wants to say out loud: we have not solved this gap in human training either.
Cognitive apprenticeship is one of the most useful frameworks we have for thinking about how doctors and AI systems should be trained, because both need more than raw pattern recognition. They need supervised judgment, contextual feedback, and a way to learn what not to do. But that framework only works when the supervision itself is reliable. And in residency programs across the country, the data suggest it often is not.
In my experience evaluating clinical AI tools, the hardest failures are rarely dramatic. They show up as a wrong recommendation in a busy triage queue, a radiology draft that sounds confident but misses clinical context, or an AI note assistant that quietly introduces a fabricated detail into the chart. Those are not software bugs in the abstract sense. They are workflow failures with patient-safety consequences. The uncomfortable parallel is that attending physicians produce similar failures -- missed teaching moments, uncorrected reasoning errors, absent feedback -- and we tolerate them because we have always tolerated them.
For readers who want the broader clinical and leadership context, I have written about physician-led digital strategy and healthcare innovation at SinabariMD clinical and leadership writing, and my professional background is outlined on Dr. Sina Bari, MD's physician-executive profile.
Cognitive apprenticeship works -- when the attending actually shows up
The framework goes back to Collins, Brown, and Newman's 1989 foundational paper, which defined cognitive apprenticeship as a method for teaching thinking skills through modeling, coaching, scaffolding, articulation, reflection, and fading. That sequence maps cleanly onto residency. A first-year resident does not start by independently running the service. They observe an attending, receive real-time correction, explain their reasoning aloud, and gradually earn autonomy.
That matters because medical competence is not just about getting the answer. It is about knowing when the answer is uncertain, when the data are incomplete, and when escalation is required. Chen et al., publishing in Academic Medicine in 2022, found that structured apprenticeship models produced 23% higher clinical reasoning scores compared to traditional didactic approaches. The scaffolding matters as much as the content.
But here is what the apprenticeship literature tends to gloss over: the model assumes a reliable supervisor. ACGME resident survey data consistently show that roughly 30% of residents report receiving inadequate or infrequent feedback from their attendings. That is not a marginal failure rate. That is nearly one in three trainees saying the core mechanism of cognitive apprenticeship -- the coached correction, the visible reasoning -- is not happening. If a clinical AI system had a 30% failure rate on its supervision loop, no IRB would approve its deployment.
One practical example: when a resident presents a febrile postoperative patient, the attending is not just checking the diagnosis. The attending is testing whether the resident recognizes red flags -- tachycardia out of proportion to fever, an evolving abdominal exam, a medication error that could explain the picture. I once stopped a resident mid-presentation to say, "Before you tell me the plan, tell me what worries you about this patient." The resident said the fever. I said, "The fever does not worry me. The heart rate of 130 with a temp of 38.1 worries me. That is disproportionate. What could cause that?" The resident's eyes widened. "A collection. Or a PE." "Good. Now go get the CT." That interaction took forty-five seconds. But I have watched attendings in the same hospital walk past a resident's presentation without a single question, sign the note, and move on. Both interactions count as "supervision" in the ACGME ledger. Only one of them actually teaches anything.
The alignment problem in AI is actually an attending problem
AI alignment in healthcare is often described in abstract terms. But the clinical version is concrete. An aligned system should know the scope of its role, defer when uncertainty is high, preserve auditability, and avoid overconfidence when the evidence is weak. The 2020 paper Artificial Intelligence, Values, and Alignment is relevant because it treats alignment as a values problem, not just an optimization problem.
That framing reveals something uncomfortable. We are trying to build AI supervision systems that work better than human supervision systems we have never fully fixed. The standard narrative says residency teaches us how to align AI -- that we can export the apprenticeship model from medicine to machine learning. I think the arrow actually points the other way. Residency's own failures reveal why AI alignment is harder than the optimists suggest.
Consider the feedback loop. In a good residency program, the attending corrects the resident in real time, explains the reasoning, and checks for understanding. In practice, attendings are overworked, distracted, and sometimes disengaged. A 2023 analysis in npj Digital Medicine found that fewer than 2% of AI models developed for clinical use ever reach deployment. The most common reason is not technical failure. It is that hospitals cannot build the supervisory infrastructure -- the monitoring, the override protocols, the feedback mechanisms -- to support them safely. That is the same bottleneck that makes residency feedback inconsistent: supervision is expensive, unglamorous, and easy to neglect.
When I review an AI vendor, the first question is not whether the model is accurate on a retrospective test set. It is whether the model knows when to stop talking and hand the decision back to a clinician. In radiology triage, for example, a high-sensitivity model that flags intracranial hemorrhage can help move studies to the front of the queue, but only if the workflow includes radiologist review, timestamped audit logs, and a process for discordant cases. Without that, the tool becomes a second-rate autopilot rather than a safety net. The same is true for a distracted attending glancing at a resident's note and signing off without reading it. The mechanism of failure is identical: a supervisor who is present in name but absent in function.
The NIST AI Risk Management Framework and the FDA's software as a medical device pathways matter here because healthcare cannot rely on generic "trust the model" language. Hospitals need traceability, post-market monitoring, change control, and clear accountability. We have spent decades demanding that from medical devices. We have spent far less energy demanding it from attending supervision.
What residency programs can teach AI developers -- and what they cannot
Residency is a highly structured apprenticeship with a built-in safety culture. The attending remains responsible, the resident is progressively entrusted, and every level of autonomy is earned. That structure is the opposite of how many AI products are marketed, where a tool is sold as if it should function independently on day one.
The education literature supports this approach. Chen et al.'s 2022 Academic Medicine study showed that structured apprenticeship formats produced measurably better clinical reasoning -- 23% higher scores -- compared to traditional lecture-based instruction. The JAMA Network Open study on AI tutoring versus expert instruction for simulated surgical skills reinforced the point: AI-assisted learning can improve performance, but only when the training environment is deliberately designed. The lesson is not that AI replaces teachers. The lesson is that AI works best when it is embedded in a pedagogic chain with human supervision.
But what residency cannot teach AI developers is how to guarantee that supervision will actually occur. Residency relies on professional norms, regulatory pressure, and institutional culture to keep attendings engaged. Those levers do not translate to software. An AI system does not have a department chair who can pull it aside and say, "Your feedback rates are terrible." An AI system's supervisory loop either works or it does not, and nobody benefits from pretending a poorly monitored model is equivalent to a well-supervised trainee.
I think this is where many health systems get the economics wrong. They buy a model for time savings, then forget that every meaningful deployment creates a new kind of supervision labor. Someone has to review false positives, investigate misses, update protocols, train staff, and document governance decisions. That is not overhead. It is the price of safe adoption. I used to think the biggest risk with clinical AI was a bad algorithm. I was wrong. The biggest risk is a good algorithm inside an institution that has not built the supervision infrastructure to use it safely. I changed my thinking after watching a well-validated imaging tool sit unused for eight months because nobody had defined who would review its outputs or what happened when the model and the radiologist disagreed. The exact same failure pattern plays out in residency programs where an attending is "available" but never actually reviews the resident's work.
Alignment failures in healthcare are supervision failures -- human and machine
The most useful way to think about model alignment in medicine is through failure modes. A model can be statistically strong and still misaligned with clinical reality. It may optimize for completion speed while degrading documentation quality, or optimize for sensitivity while flooding a stroke team with low-value alerts. These are not hypothetical problems. They are the clinical AI equivalent of an attending who signs off on notes without reading them -- technically compliant, functionally absent.
In pathology and radiology, where AI is increasingly used for prioritization, segmentation, and draft reporting, alignment means respecting the human chain of responsibility. If the workflow lets the model speak before the expert has reviewed the image, the system is not aligned. It is merely persuasive. The same logic applies to hospital AI governance more broadly: every deployment needs escalation rules, human override, performance monitoring, and a rollback plan.
That is why medical residency remains such a powerful -- and cautionary -- analogy. Residents are expected to present their reasoning, accept correction, and adapt based on feedback. AI systems should be held to a similar standard, except the feedback loop must be tighter because the consequences are operationally and ethically heavier. But residency also shows what happens when supervision degrades: errors go uncorrected, bad habits calcify, and the system drifts toward overconfidence. If we build AI supervision the way we run many residency programs -- with good intentions but inconsistent follow-through -- we will get the same mediocre results.
Clinical leadership also needs to distinguish between model capability and institutional readiness. A telemedicine triage model that performs well in one health system may fail in another because the local workflow, EHR integration, language mix, and staffing patterns are different. A tool is never "neutral" once it enters a hospital. It becomes part of the institution's decision architecture. The same is true for a new attending joining a training program. Context determines whether their presence improves or degrades the system.
For a practical governance lens, I often tell teams to ask three questions. What is the model allowed to do, what is it prohibited from doing, and who is responsible when the model is wrong? If those answers are vague, the deployment is premature. I have started asking a fourth: who is monitoring the monitor? Because supervision without accountability for the supervisor is just theater.
The future is supervised autonomy -- but only if we fix supervision first
There is a temptation in AI marketing to describe future clinical systems as if autonomy were the goal. I do not think that is the right frame for medicine. The right frame is supervised autonomy with explicit limits, similar to the way a senior resident earns independence but still works inside attending oversight.
That model is especially relevant for high-stakes settings such as emergency medicine, inpatient deterioration surveillance, and radiology worklists. AI can help prioritize, summarize, and surface risk. It should not be allowed to replace the physician's duty to interpret context, reconcile contradictions, and decide when the pattern does not fit.
But supervised autonomy only works if the supervision is real. The uncomfortable truth is that we are trying to engineer AI supervision systems to a higher standard than the human supervision systems they are supposed to complement. That is not a reason to lower the bar for AI. It is a reason to raise the bar for both. The ACGME feedback data, the npj Digital Medicine deployment statistics, the Chen et al. reasoning scores -- they all point to the same conclusion. Supervision is the bottleneck for both human and machine learning in clinical settings. We cannot solve one without confronting the other.
When I look at the most promising healthcare AI systems, the best ones behave less like answer engines and more like disciplined trainees. They acknowledge uncertainty. They route edge cases upward. They keep an audit trail. They improve with feedback without pretending to be wiser than the attending physician.
That is the alignment target hospitals should actually want. Not perfect automation. Reliable deference, transparent reasoning, and workflow design that preserves clinical judgment.
AI in medicine will be judged less by how fluent it sounds and more by whether it acts like a well-trained resident under supervision. That is the standard worth building toward. The resident from my opening anecdote eventually became an excellent surgeon. Not because someone gave her the right answer about the thrombosed perforator, but because the pause, the silence, and the coached reasoning built a decision framework she could carry into the next case and the one after that. If we want clinical AI that physicians actually trust, we need to build it the same way: through structured supervision, visible reasoning, and earned autonomy. Not through confidence without accountability. And if we are honest, we should admit that fixing AI alignment starts with fixing how we supervise the humans first.
Frequently asked questions
Why does the ACGME resident feedback problem matter for AI alignment?
Because AI alignment assumes a functional supervision loop -- a reliable mechanism for correcting errors and reinforcing good behavior. ACGME survey data shows roughly 30% of residents report inadequate feedback, meaning the human version of this loop already fails at scale. If hospitals deploy AI systems using the same supervision infrastructure that underserves residents, those systems will inherit the same blind spots. Fixing AI alignment requires fixing human supervision first.
How is cognitive apprenticeship different from standard AI training?
Cognitive apprenticeship teaches through modeling, coaching, and gradual release of responsibility, while standard AI training often optimizes for prediction accuracy alone. Chen et al. (Academic Medicine, 2022) showed structured apprenticeship produced 23% higher clinical reasoning scores. In healthcare, the apprenticeship model is more useful because it includes judgment, context, and correction. But it only works when the supervisor -- human or algorithmic -- is actively engaged.
Why do fewer than 2% of clinical AI models reach deployment?
According to research published in npj Digital Medicine (2024), the primary barrier is not algorithm quality. It is the absence of supervisory infrastructure: monitoring protocols, override mechanisms, feedback loops, and clear accountability chains. Hospitals that buy AI tools without building the governance layer around them end up with validated models sitting unused -- the same way a residency program with disengaged attendings produces technically competent but unsupervised trainees.
What is Dr. Sina Bari's approach to evaluating a healthcare AI vendor?
I ask four questions. What is the model allowed to do? What is it prohibited from doing? Who is responsible when the model is wrong? And who is monitoring the monitor? That fourth question is the one most vendors cannot answer. A good vendor can explain not only what the model does, but how the hospital will supervise it safely -- and what happens when that supervision lapses.