Skip to content
Reza Motaghi
Lab Notes4 min read

The error to fear is the treatment nothing justifies

By Reza Motaghi

The AI plan line that worried me did not invent a finding. It proposed a definitive restoration for a child, with nothing in the findings to justify it.

In October I had a local AI model draft treatment-plan lines from my findings. One oral and maxillofacial radiologist, my own findings, one case: a child in the mixed-dentition years. The read carried one finding, an estimate of dental age. The tooth chart already showed crowns on two upper baby molars. The model wrote a definitive restorative line for those crowns, and gave the age estimate as its reason.

One line of code caught it. The treatment class on that line is not on the list of classes allowed for that finding. The list is mine: allowed treatments per finding, drafted from my rulebooks and confirmed row by row by two specialists.

What would have justified it? The paediatric guideline is plain about crowns on a baby tooth.1 Extensive or multi-surface decay, a tooth after pulp treatment, a developmental defect, or a child at high risk of decay. An age estimate is none of those. The line had a category and a confident verb and no cause.

Here is the shape of it. Everyone watches for the invented finding, the lesion that is not there. In dental AI the error to fear is the treatment nothing justifies. It is the error that removes tooth structure, and it cannot be undone. The literature already points there. In a randomized trial in the Journal of Dentistry, dentists with AI help found more early lesions.2 Both their non-invasive and their invasive treatment decisions rose. A cost model built on that trial found the same tooth retention and the same cost with and without the AI.3 The extra lesions were treated rather than watched. Detection is not the end of the decision. The step after detection is where the harm lives. That is not a claim that AI overtreats by nature: a 2025 trial on periapical lesions found the opposite, fewer false positives and more conservative decisions4. It is a claim about what happens after a finding, and in routine care that step already leans toward the drill. A Finnish records study called the restoration rate at early lesion depth "unexpectedly high"5.

Language models lean the same way. Given ten thousand emergency visits, two of them were "overly cautious", recommending more than the physicians did, at the cost of specificity6. A careful prompt helps and does not finish the job. Under a safety prompt, six models still elaborated on a fabricated detail in forty-four percent of cases, down from sixty-six.7 The prompt persuades. It does not refuse.

The guard is not a smarter model. It is a list a specialist confirmed, and a check that cannot be talked out of it. Shown the list, a frontier model's off-list lines fell to none. The code catches what the list does not persuade.

Then I caught my own error. Off the list is not the same as wrong. Biology has exceptions a rulebook does not hold, and a list enforced as a wall makes the model as rigid as the list. So the check flags, and a specialist sorts the flags. The first batch of flags is still mine to sort.

This is not a radiology rule. Wherever a model proposes an action, the error to fear is the one nothing justifies. The refund nobody asked for, the escalation with no trigger, the prescription without a diagnosis. Write what is allowed per cause, with someone who confirms it, and put the check in code.

Two caveats I owe the reader. One case, one model, one line. The same line came back on every re-run and on the cases that followed. Enough to build the guard, not enough to call it a rate. And the claim is about dentistry and the restorative decision. In general medicine the severe errors that studies count are mostly omissions, things not done. So I keep this to the field where the harm is a tooth that does not grow back.

What I would do differently: write the exception door into the list on day one, not after the wall was up.

References

Footnotes

  1. American Academy of Pediatric Dentistry, "Pediatric Restorative Dentistry", The Reference Manual of Pediatric Dentistry, 2025 (latest revision 2022), https://www.aapd.org/globalassets/media/policies_guidelines/bp_restorativedent25.pdf. ↩

  2. S. Mertens et al., "Artificial intelligence for caries detection: Randomized trial", Journal of Dentistry 115:103849, 2021, https://doi.org/10.1016/j.jdent.2021.103849. ↩

  3. F. Schwendicke et al., "Cost-effectiveness of AI for caries detection: randomized trial", Journal of Dentistry 119:104080, 2022, https://doi.org/10.1016/j.jdent.2022.104080. ↩

  4. U. Pul et al., "Impact of artificial intelligence assistance on diagnosing periapical radiolucencies: A randomized controlled trial", Journal of Dentistry 160:105868, 2025, https://doi.org/10.1016/j.jdent.2025.105868. ↩

  5. M. H. Muhonen et al., "Bitewing Radiographs in Proximal Caries Diagnostics and Restorative Treatment of Adults: A Retrospective Study of Electronic Health Records", Caries Research, 2025, https://doi.org/10.1159/000549383. ↩

  6. C. Y. K. Williams et al., "Evaluating the use of large language models to provide clinical recommendations in the Emergency Department", Nature Communications 15:8236, 2024, https://doi.org/10.1038/s41467-024-52415-1. ↩

  7. M. Omar et al., "Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support", Communications Medicine 5:330, 2025, https://doi.org/10.1038/s43856-025-01021-3. ↩

About the author

Reza Motaghi is an oral and maxillofacial radiologist and Chief Innovation Officer who reads every day and builds, evaluates, and trains the imaging AI for it. He built CBCTScope, the first CBCT viewer with native AI-agent control, and writes about what a model did on a real case and what it did not.

More about RezaSubstackLinkedInGitHub

The newsletter

Get the essays as they’re written.

Imaging AI, evaluation, and building alone, from a radiologist who reads every day and tests before he trusts. A few essays a month, nothing else.

Join for FreeFree · Unsubscribe anytime

Related

  1. What the best general model read on a specialist's ruler

    The strongest general model read three findings in ten on my cases and invented three in four. One accuracy number hides that. Three questions to ask.

  2. Agreement with past cases is not a check

    A model turned my rulebook into a decision table that agreed with 99 of 100 past lines. A reviewer briefed to disagree found 19 errors, then 20 more.

  3. Saying less is not seeing more

    The newest general model invented half as much on 128 of my own radiographs and missed more bone loss and caries. A single safety number cannot tell a safer reader from a quieter one. The two-direction test, in three steps.