Why AI summaries turn possible into definite

Two in five of the important words in a radiology report sit inside a sentence that says not, or possibly. A study of 6,000 reports counted it.¹ About 15 percent of all the words live inside a negated or speculative statement. Among the diagnostic terms that matter, the share is about 39 percent. A report is not a list of findings with a few hedges attached. It is a document about certainty, with findings inside.

That is why a summary can keep every finding and still change the report. It only has to delete one word.

I read medical images every day, and every report I sign is graded in plain words. What is there. What is probably there. What is possible on this image and needs another. I train a vision model on my own reads, and I wrote its answer key myself. One joint I was certain showed degenerative change stayed possible in the label, because the image did not support certain. A hedge chosen for a reason is a finding about the evidence. It is the first thing a rewrite deletes, and the rules below exist to stop it.

What the certainty budget is

The certainty budget is the number of graded signals a rewrite can carry before it starts rounding them off. Every summary, translation and simplification has one. Below the budget, possible stays possible. Above it, the hedges collapse toward whatever the sentence sounds like without them, which is almost always more certain than the original.

The budget is not the same as accuracy. A rewrite can keep every finding right and still spend its certainty budget, because the finding and the hedge are two different pieces of information. The models tested so far have small budgets. A single hedge usually survives. Two or three in one passage usually do not.

The joint I left possible

The receipt is one label. Writing training examples from my own reads, I came to a joint I was sure showed degenerative change. Twenty years of pattern memory said so. The image did not. It supported possible, and no more. So the label said possible, and the certainty I felt went into a separate file that no model will ever train on.

That hedge was not caution. It was the most precise statement the evidence allowed, and it carried a decision: get another view before you call it. A rewrite that turns it into definite has not made the report cleaner. It has deleted the decision, and handed the next reader a certainty nobody earned. That is the same line I hold when normal is not a verdict. A read is about this image, and the words say exactly how far it reaches.

Where the flattening comes from

The cleanest measurement is a benchmark built for exactly this failure. Researchers took 1,200 clinical documents with thousands of certainty cues marked across five levels, and asked three off-the-shelf models to preserve them.² The models kept the original level under half the time. The best of them read a single cue right about four times in five. With five cues in one passage, it fell to a coin flip. The failure is not blanket erasure. It is what happens under load, when several graded signals compete in one sentence.

One caveat the benchmark's own authors give, and the piece should practise what it preaches. The ground truth was built by a rule-based pipeline that matched manual review about four times in five. Read the number as a strong signal about a real mechanism, not as a settled measurement.

The field reports the same thing from the desk. Clinicians who edit ambient AI drafts of their own consultation notes described the tool turning "possibly this" into a flat statement.³ It wrote an unconfirmed diagnosis as confirmed unless they blocked it. Several said they insert hedges back by hand, as protection. A frontier model asked about colorectal guidelines matched their content nine times in ten.⁴ It matched its own stated confidence to the strength of the evidence only three times in four. On weak recommendations it fell to one in three. The authors named the pattern certainty inflation, and found that naming the evidence strength in the question raised calibration to nineteen in twenty.

The harm has been measured and, so far, fenced. When 500 imaging reports were rewritten for patients at seventeen reading levels, accuracy held down to an eleventh-grade level.⁵ At seventh grade, one in five reports carried an error that could change management, including a serious finding that read as benign. A separate randomised study found patients rated simplified reports far clearer and equally trustworthy, while the rewrite introduced causal claims the original never made.⁶ The line researchers have drawn is real. So is the drift on the wrong side of it.

One contrast sharpens the claim rather than softening it. In a benchmark of confidence under missing information, most models tested were severely overconfident.⁷ One recent frontier model showed a small gap in the other direction. It still invented an answer four times out of five when no correct answer existed. So the honest sentence is not that language models cannot hold graded certainty. It is that most deployed ones do not, because nobody has demanded it of them yet.

Every field has a hedge

Auditing wrote the rule down decades ago. American auditing standards ban the phrase "subject to" from a qualified opinion.⁸ The standard calls it not clear or forceful enough, and requires the word "except" instead. An audit qualification is a hedge that must survive being read past, and the standard is built for the reader who skims.

Weather forecasting learned it from the public. A study of Florida's 2004 hurricane season drew on nearly a thousand public comments.⁹ It found the forecast cone itself invited people to lock onto the centre line as the answer. The spread was the forecast. Readers took the line. In finance, an analyst estimate stored as its average has thrown away the spread.¹⁰ The spread is the one number that says how contested the estimate is.

In every case the shape is the same. The graded signal was the information. The rewrite kept the object and lost the grade.

Not every hedge is a finding

The field's own quality literature is blunt about this, and the rules have to be too. Three in four referring physicians want a ranked list of diagnoses with explicit certainty attached, not "cannot be entirely excluded" on everything.¹¹ Radiologists trained to write more definitively cut their certainty phrases from about two per report to about one.¹² The hedge-heavy reports were the ones that triggered more follow-up scans. A radiologist wrote in 2015 that reflexive hedging defeats the purpose of ordering the imaging in the first place.¹³ That was a decade before any model touched a report.

So a hedge is not automatically worth protecting. A hedge chosen for a reason, on a joint the image could not settle, is a finding. A hedge used as cover, on everything, is noise. And noise is the first thing a rewrite flattens, which takes the reasoned hedge down with it. The discipline is to know which of the two you wrote.

Four rules before a machine rewrites your hedges

The rules run on any document that carries graded certainty into a system that rewrites it. One of them I changed after reading the field's own case against hedging.

1. Say what each hedge word commits to. Possible, probable, cannot exclude, consistent with. Write, once, what each one promises the next reader and what it asks them to do. A word whose commitment nobody can state will not survive a rewrite, because the rewrite has nothing to preserve. The profession is now trying to standardise the same rule for itself. The callable floor applies it to labels: a call is only a call if you can say what the input supports. Off-ramp: a field with one hedge word and one meaning needs no list. Most fields have five and pretend they have one.

2. A rewrite may lower certainty, never raise it. Any summary, translation or simplification that turns possible into definite is wrong, even when the finding underneath is right, because it has invented a decision. Lowering certainty is a smaller error, and it is at least visible. The seventh-grade rewrites that read a serious finding as benign are the measured cost of ignoring this.⁵ Off-ramp: a rewrite for a reader who cannot act on the grade may drop the grade, if it says so. A patient-facing summary that says to ask the doctor is that case. It may never upgrade it.

3. Feed it hedged text and count what survives. Take ten sentences from your own work with graded certainty in them. Run them through the tool you rely on. Count how many kept their level. For the last three, stack two hedges in one sentence, because that is where the benchmark says the budget runs out. The count is the tool's certainty budget. It is lower than its accuracy. It is the number to know before the tool touches a report that will be read by three readers. Off-ramp: a tool whose output nobody acts on needs no test. If it feeds a decision, it does.

4. Keep the hedge that carries a reason, cut the one that carries cover. My first version of this rule said keep every hedge. The field changed it. A possible on a joint the image could not settle is a finding, and the rules above protect it. "Cannot entirely exclude" on every line is cover, it costs the reader follow-up scans, and it is exactly the noise a rewrite flattens first. Cutting it yourself, before the machine does, is how the reasoned hedge survives. Off-ramp: in a report that will be read for its legal weight, keep the cover hedge where the standard requires it. Mark it as such. Auditing does.

What the rules are not

They are not a defence of vague writing. The evidence that hedge-heavy reports cause follow-up scans is as solid as the evidence that rewrites flatten hedges. The rules protect the grade, not the fog.

They are not a claim that models cannot learn this. One recent model showed the opposite direction of error, which means the discipline is possible and nobody has asked for it. Asking is the point of rule three.

They are not about patients. The rewrite that matters here is the one between a specialist and the next specialist, or the machine that reads the report for a decision. The patient-facing summary has its own rules, and its own line at the eleventh grade.

What I sign for

I read medical images every day and I write the words that grade what I saw. Every hedge in a report I sign carries a reason. When I wrote training examples from my own reads, the hedges went into the labels exactly as the image supported them. A model trained on that learns the grade. A rewrite that deletes the grade has deleted a finding, and it did it quietly, in a sentence that reads better than mine.

Run the ten-sentence test on the summarizer you rely on. The count will tell you its certainty budget, and you will know it before the tool knows it. If you build or evaluate models that read images, the contact page reaches me directly. A radiologist who writes the grade can sit on your side of the table.

References

Footnotes

1. Weng, Liu, Chen. Deep learning approach for negation and speculation detection for automated important finding flagging and extraction in radiology report. JMIR Medical Informatics, 2023. 6,000 reports; 15.01 percent of words and 39.45 percent of important diagnostic keywords inside negated or speculative statements. https://pmc.ncbi.nlm.nih.gov/articles/PMC10170361/ ↩

2. Du H, Lu Z, Qu J. Possible or definite? A benchmark for evaluating diagnostic uncertainty preservation in clinical text. arXiv, 16 June 2026. 1,200 clinical documents, 9,184 uncertainty annotations across five levels, three off-the-shelf models; original level preserved correctly under half the time; best single-cue accuracy 78.4 percent, 48 percent with five stacked cues; ground truth from a rule-based pipeline matching manual review 82 percent of the time, kappa 0.61. https://arxiv.org/abs/2606.18471 ↩

3. Guo, Hu, Yang, Chow, Tam, Perret, Pandita, Zheng. Clinicians' rationale for editing ambient AI-drafted clinical notes: persistent challenges and implications for improvement. Journal of the American Medical Informatics Association, 2026, volume 33, issue 7. https://pmc.ncbi.nlm.nih.gov/articles/PMC13317938/ ↩

4. Maida, Facciorusso, Fuccio, Tham, Vitello, Calabrese. Risk of guideline overselling and certainty inflation in GPT-5 responses: a statement-based analysis of four ESGE colorectal guidelines. Endoscopy International Open, June 2026. Content adherence 91.5 percent; calibration to evidence strength 74.6 percent overall, 33.3 percent for weak recommendations; 94.9 percent when the evidence strength was named in the prompt. https://pmc.ncbi.nlm.nih.gov/articles/PMC13289777/ ↩

5. Lee, Song et al. The ethics of simplification: balancing patient autonomy, comprehension, and accuracy in AI-generated radiology reports. BMC Medical Ethics, October 2025. 500 CT and MRI reports at 17 reading levels; 20 of 100 seventh-grade simplifications carried management-changing errors; accuracy stable to the eleventh grade. https://pmc.ncbi.nlm.nih.gov/articles/PMC12523008/ ↩ ↩²

6. Horr et al. Perceived communication quality of a standardized neuroradiology report following AI-based simplification: a randomized, blinded study. Insights into Imaging, August 2026. Clarity, trust and usability rated higher for the simplified version; unsupported causal attributions introduced, a secondary observation. https://pmc.ncbi.nlm.nih.gov/articles/PMC13522326/ ↩

7. Tahermazandarani, Mahmood, Islam, Sheng. When confidence fails: overconfidence in LLMs under uncertainty and missing clinical information. arXiv, August 2026. Open-weight models with unsafe confident error rates of 0.83 and 0.76; one frontier model with a small, correctly directed calibration gap, still answering four times in five when no correct answer existed. https://arxiv.org/abs/2608.09080 ↩

8. Public Company Accounting Oversight Board. AS 3105, Departures from unqualified opinions and other reporting circumstances. "Subject to" is barred from a qualified opinion as not clear or forceful enough; "except" is required. https://pcaobus.org/oversight/standards/auditing-standards/details/AS3105 ↩

9. Broad, Leiserowitz, Weinkle, Steketee. Misinterpretations of the "cone of uncertainty" in Florida during the 2004 hurricane season. Bulletin of the American Meteorological Society, 2007, volume 88, pages 651 to 668. 962 public comments plus interviews. https://journals.ametsoc.org/view/journals/bams/88/5/bams-88-5-651.xml ↩

10. Adithyan N. How to analyze analyst estimate ranges with Python. freeCodeCamp, June 2026. A practitioner's account of the spread lost when only the consensus average is stored. https://www.freecodecamp.org/news/how-to-analyze-analyst-estimate-ranges-with-python ↩

11. Rios Ignácio, de Souza, D'Ippolito, Garcia. Radiology report: what is the opinion of the referring physician? Radiologia Brasileira, 2018, volume 51, issue 5. 75.7 percent preferred a ranked differential with explicit certainty. https://pmc.ncbi.nlm.nih.gov/articles/PMC6198844/ ↩

12. Zafar et al. Improved diagnostic confidence imparted by radiologists in radiology reports after educational interventions on reporting styles. Cureus, February 2024. Certainty phrases per report from about two to about one over three cycles; uncertain reports linked to more follow-up CT. https://pmc.ncbi.nlm.nih.gov/articles/PMC10924976/ ↩

13. Jha S. It's time for radiologists to stop hedging. KevinMD, June 2015, originally in the ACR Bulletin. https://kevinmd.com/2015/06/its-time-for-radiologists-to-stop-hedging.html ↩