Three training runs, one lesson

A model can learn your report without ever learning your image. Mine did, three times, and every dial said success.

I trained a vision-language model, an AI that looks at an image and writes text about it, on my own panoramic radiograph reads. One board-certified oral and maxillofacial radiologist, every case read blind and taken to consensus with a second reader before it became a training example. Three training runs between 22 and 30 August. The grading was written and sealed before the first run: cases the model would never train on, scored finding by finding against my signed reads, the judge blinded to which model wrote what.

It learned the report completely. The section order, the phrasing, the way I hedge, the line I write when a tooth is missing. From the outside the output was mine.

Then the grade. At its best checkpoint the model never once wrote bone loss. It wrote lesions and retained roots often, in the wrong places. It found caries fewer than one time in three while writing caries lines by the hundreds. Cases it had trained on scored no better than cases it had never seen, which is the proof that the image never entered the learning.

Then I measured what its eye can see. A vision model reads an image in small square tiles. On a panoramic at this pixel size, one tile is about three millimetres. The crest of the alveolar bone, the wall of a canal, a foramen: each is smaller than one tile. Two of the findings it never learned live below the tile. No number of my reads would have taught it those. The physics beat the data. For scale: the largest panoramic reader published so far trained on images from more than a hundred thousand patients (Nature Biomedical Engineering, 2026).

Here is what I would keep from the three runs. Write the grading before the training and seal it. Decide in advance what each failure would look like and what you would do about it. Before the third run I wrote down four failure signatures and the move for each, so when the grade came in the move was already made: stop writing examples for a class the tile cannot resolve, and stop trusting any dial that is not a blind grade against your own reads.

The loss curve fell every time. The output was fluent every time. Neither can tell you whether the model looked.

The check

The three-line version, for any model you fine-tune on expert work:

Before training, write the failure signatures and the move for each. If you cannot name what failure looks like, you cannot recognise it.
Grade finding by finding against your own signed cases, on cases the model never saw, with the judge blinded to the source. One accuracy number hides everything that matters.
Compute the model's smallest unit of attention in millimetres and list every finding smaller than it. Those are not data problems, and no dataset fixes them.

Sources

The two 2026 dental readers: citations to be confirmed before publish (see open check).