Skip to content
Reza Motaghi
Lab Notes4 min read

Inventory before interpretation

By Reza Motaghi

The model reported a root canal in a tooth that was not in the mouth. Then it kept writing.

Blind, on a set of my own panoramic radiographs, I tested the strongest general AI model I could reach. One oral and maxillofacial radiologist, the reads I had signed as the answer key. Every statement the model made was compared to those reads by a judge that could not see who had written it.

Every expert read starts with an inventory of what is there. In a radiograph: which teeth are present, which are gone, which are still in the bone. Disease is read on top of that list. Nobody praises the list. It is the step a fluent system skips first, because nobody thinks to grade it.

Before the model said anything about disease, its list was wrong. It called a tooth missing that my read had present, or still unerupted in the bone, again and again across the set, in most cases. It called a tooth present that was not there many times the other way. And in some cases it went on. A filling, a root canal, a carious lesion, a bone defect, described in well-formed prose on a tooth that was not there to carry it.

A finding on a tooth that does not exist is not a finding. It cannot be checked, treated or followed up. A tooth called missing that is in fact present hides everything on it. Whatever the model said after that was built on a map with the wrong streets.

The field has a name for this, object hallucination, and has measured it since 2023 by asking a model whether a thing is in the picture. On panoramic radiographs a 2025 study found a general model doing exactly this: present teeth labelled missing, most of them third molars. The cure is older than the name. Ask for the inventory first, as its own answer. Compare it to your own count, item by item. If the count is wrong, stop there. Do not grade the findings and do not admire the prose. A read that fails the count has failed before the interesting part began.

This is not a radiology rule. An auditor counts the accounts before judging the books. A pilot walks around the aircraft before reading the instruments. A reviewer of an AI summary checks that every clause it cites exists before weighing what it says about them. Every expert read starts with a boring list. The boring list is what fluent systems get wrong first, because it is the one part nobody tests.

One caveat I owe the reader. These are the judge's flags on an early answer key, before my own hand adjudication and before that key was revised. The count will move. The order of failure will not: the model had the map wrong before it described the territory.

The check

The inventory check, before you trust an AI read of anything:

  1. Ask what is there before what is wrong with it. The parts list first: teeth, accounts, clauses, files. Its own answer, before any finding.
  2. Compare that list to your own, item by item. Count, do not sample. A fluent report hides a wrong count.
  3. If the count is wrong, stop. A finding on a thing that is not there is not a finding. Do not grade the rest, do not admire the prose.

Sources

  • Li Y, Du Y, Zhou K, Wang J, Zhao WX, Wen JR. Evaluating Object Hallucination in Large Vision-Language Models. EMNLP 2023. arXiv:2305.10355. The name, and the poll that measures it by asking whether an object is in the image.

  • Liu Z, Ai QYH, Yeung AWK, Tanaka R, Nalley A, Hung KF. Performance of a Vision-Language Model in Detecting Common Dental Conditions on Panoramic Radiographs Using Different Tooth Numbering Systems. Diagnostics, 2025. Fifty panoramic radiographs. Most of the present teeth the model called missing were third molars.

  • Asadi M, O'Sullivan JW, Cao F, Nedaee T, Rajabalifardi K, Li FF, Adeli E, Ashley E. MIRAGE: The Illusion of Visual Understanding. arXiv:2603.21687, March 2026. Frontier models produce detailed clinical findings for images never provided.

  • Ghazali AB. Interpreting panoramic radiographs for beginners. BDJ Student, 2020. Anatomy and dentition are categorised before the diagnosis is formulated.

  • The test itself: a set of my own cases, blind, every statement judged against the reads I had signed. The judge's nominations, before adjudication, are held in the private record.

About the author

Reza Motaghi is an oral and maxillofacial radiologist and Chief Innovation Officer who reads every day and builds, evaluates, and trains the imaging AI for it. He built CBCTScope, the first CBCT reading viewer with native AI-agent control, where the agent moves the camera and the human reads, and writes about what a model did on a real case and what it did not.

More about RezaSubstackLinkedInGitHub

The newsletter

Get the essays as they’re written.

Imaging AI, evaluation, and building alone, from a radiologist who reads every day and tests before he trusts. A few essays a month, nothing else.

Join for FreeFree · Unsubscribe anytime

Related

  1. What the best general model read on a specialist's ruler

    The strongest general model read three findings in ten on my cases and invented three in four. One accuracy number hides that. Three questions to ask.

  2. Saying less is not seeing more

    The newest general model invented half as much on 128 of my own radiographs and missed more bone loss and caries. A single safety number cannot tell a safer reader from a quieter one. The two-direction test, in three steps.

  3. It counts the teeth and misses the disease

    Blind on 128 of my own radiographs, the strongest general model I could reach counted every implant exactly and never once called the jaw joint. The parts list is solved, the call is not. The check, in three steps.