Skip to content
One-Person Startup10 min read

How to overrule an AI flag and leave a record

By Reza Motaghi

A small sheet pinned over a signed page with the pin catching the light, a black and white image about the addendum that carries its own date
On this page

On the night of 1 July 2002, two aircraft over southern Germany received opposite instructions. The air traffic controller told one crew to descend. The collision-avoidance system in their cockpit told them to climb. The crew followed the human, as their manual allowed, and the two aircraft collided. Seventy-one people died. The investigation found the collision would not have happened had both crews followed the machine.1 It traced the crew's choice to manuals that described the system as a backup.

The fix was not a lecture about trusting machines. It was a written rule about which instruction wins, and it was written before the next night it would matter. That is the whole subject of this article. Once a machine and a professional differ, the thing a later reviewer examines is not the finding. It is the record of the moment the professional's judgment moved or held.

I read medical images every day. When I test a model blind, against a two-radiologist reference, my read is written first, and only then do I look at the model's. When I wrote the answer key my own model learns from, one joint I was certain showed degenerative change stayed possible in the label, because the image did not support certain. The image did not support the call. The reason is in the file beside it, dated. That disagreement, between what I believed and what I could show, is in the record on purpose.

What the dated disagreement is

The dated disagreement is a record, made at the time, of four things. What you read. What the machine said. Why your answer stands. What would have changed it. It is written before anyone knows who was right.

Nobody writes it for the day they were right. On that day nobody asks. It exists for the replay. That is the day someone who was not in the room reads the machine's flag next to your answer and has to decide what the difference means. A record made at the time answers that question. A memory offered later is testimony.

The joint that stayed possible

The receipt is one label. When I built the dataset my model learns from, the contract said a finding goes into the label only when the image supports it. On one case I was certain a joint showed degenerative change. I had read thousands like it. The image, at that resolution and that angle, did not show enough to meet the bar the contract set, so the label stayed possible, not certain.

Beside it went the reason, and beside that the date. Reading it back, the sentence is the useful part. It says what I believed, what the evidence allowed, and which one the label followed. A model trained on that label learns the evidence, not my conviction. A reviewer replaying the case learns that the gap between the two was seen and decided, not missed. A blind read written before the second opinion is the same discipline with the order reversed.

What the replay does

The best evidence comes from a research programme that puts hypothetical cases in front of lay readers and asks who was at fault. It is proxy evidence, and I will say exactly how much it carries.

In the first experiment, published in 2025, readers judged a radiologist who had missed a brain bleed.2 With no AI in the story, 56 percent found against him. When an AI tool had flagged the bleed and the radiologist had missed it, 73 percent did. When those readers were also told the tool's own miss rate, the share fell to 34 percent. A cancer version of the same experiment moved less, in the same direction, so the size of the effect depends on the story. Its direction held.

The follow-up, posted in 2026 and not yet peer reviewed, is the sharper study.3 It put 2,347 readers across 22 workflow conditions. The most punished behaviour in the whole design was a reader who had the case right, saw the machine call it normal, and changed the answer. Nothing in the record said why. The authors named it the erroneous-change penalty. Two things lowered the perceived liability: a second reader, and documenting the initial interpretation before looking at the AI output. Whether the radiologist personally knew the tool's error rate did not move the readers much. Whether they were told it did.

Read that carefully, because it is not the finding people expect. Disagreeing with the machine was not what cost the reader. Silently agreeing with a wrong machine after getting it right was. And the protective act was not being right. It was having written the read down first.

How much the evidence carries

Both studies are online vignette experiments with paid respondents. There was no deliberation, no courtroom, and no unanimity to reach. Jury science has lived with this limit for decades. Real jury rooms are closed to researchers almost everywhere, so proxies are the only instrument the field has.4 The numbers will date. The mechanism is older than the numbers, and it is the part worth keeping.

There is also, as of early 2026, no decided court case on an AI-assisted medical read. A survey of the legal literature found none.5 It argued that courts will most likely extend the existing reasonable-practitioner test rather than write new doctrine. A clinician-liability review reached the same absence a year earlier.6 The tension between being liable for using a tool and liable for not using it is real in theory. It is untested in any ruling. What courts have always done is judge a professional's read with the benefit of hindsight. A radiology miss case from 1991 was already arguing about exactly that.7

Geography matters too. In Germany a claim against a physician runs through the civil code before a judge.8 The European AI Act adds documentation and oversight duties on top. It creates no liability rule of its own. The directive that would have harmonised AI liability across the Union was withdrawn in February 2025.9 Nothing in this article is legal advice. The moves below are documentation, and documentation is the part a professional controls in every jurisdiction.

Both directions fail, and the record survives both

Overruling the machine was the wrong call over Germany in 2002. Deferring to it was the wrong call on two American highways in 2024.10 Drivers on a partly automated system did not brake, and neither did the car. The investigators cited overreliance on the automation. So the question is never whether to trust the machine. It is whether the moment you decided is written down.

In imaging the regulator has settled part of this already. Software that analyses a medical image to produce a diagnostic recommendation remains a regulated device.11 The professional's review of its output is part of the design. The flag is an instrument reading. Your read is the judgment. A second check only adds safety when it is wired to disagree, and the record is where the wiring shows.

Four moves before you overrule a machine

The moves cost a sentence each. One of them I changed after reading the second study.

1. Write your own read first. My first version of this move said write the disagreement down. The 2026 study sharpened it. Documenting the initial interpretation before seeing the machine's output was one of two things that protected the reader. So the read comes first, wherever the workflow allows it. A judgment that exists on paper before the flag cannot be accused of having drifted toward it or away from it. Off-ramp: some workflows show the flag before you can read. Then write your read as if the flag were absent, and say in the record that the flag was visible.

2. Write the disagreement at the time, with the reason. Flag seen, read stands, because. Then the date. The reason is the load-bearing word, because a reason written at the time is a decision and a reason offered later is a reconstruction. The report now has three readers, and the third one arrives after the fact. Off-ramp: when you agree with the flag, write that too. In the 2026 study, agreement was never the problem. Silent movement was.

3. Name what would have changed your mind. One sentence. A better image, a prior study, a second reader. It proves the override was a decision rather than a reflex, and it tells the next reader exactly where to look. It also protects you from yourself: an override you cannot state a condition for is a habit. Off-ramp: none. If no condition exists, that is the sentence to write.

4. Know the machine's miss rate for that kind of call, and write it down. The single number that moved the mock jurors most was the tool's own error rate. In the second study, telling the readers that rate lowered liability. The radiologist merely knowing it did not. So the rate belongs in the record, not in your head. Off-ramp: a tool whose error rate for your kind of case is not published has answered a different question. The answer key behind its score is itself a settled argument. Ask for the rate before the next renewal.

The dated disagreement, four moves: write your own read first, write the disagreement with the reason, name what would have changed your mind, record the machine's miss rate

What the moves are not

They are not legal advice, and they are not a defence. No court has tested whether a documented disagreement changes an outcome, because no such case has been decided. The moves are sound practice on their own terms. If they ever help in a replay, that will be a consequence, not the reason.

They are not a case for overruling. The best evidence punishes silent capitulation, not disagreement. It says nothing about a documented override that turns out wrong, and the aviation record shows what one of those costs. Write the record either way.

They are not about the other AI litigation now in the news. The live cases that name an algorithm in a medical decision are about an insurer automating coverage denials at scale, with little individual review.12 That is a different mechanism from one professional's dated judgment on one case, and blending the two would misstate what is actually before a court.

What I write

I read images every day, and I write my read before I look at the model's. The one label I held at possible against my own conviction carries its reason and its date. The replay does not grade the finding. It grades the moment my judgment moved or held. I would rather it read a sentence I wrote at the time than a memory I offer later.

Write the next override down while it is still a decision. If you build or evaluate tools that flag a professional's work, the contact page reaches me directly. I have sat on the flagged side of the desk.

References

Footnotes

  1. 2002 Überlingen mid-air collision, 1 July 2002. The German Federal Bureau of Aircraft Accident Investigation found the collision would not have occurred had both crews followed their TCAS instructions, and traced the crew's choice to manuals describing TCAS as a backup to air traffic control. https://en.wikipedia.org/wiki/2002_%C3%9Cberlingen_mid-air_collision ↩

  2. Baird, Bernstein et al. NEJM AI, published 22 May 2025, figures as reported by AuntMinnie. Brain-bleed vignette: 56.3 percent with no AI, 72.9 percent when AI flagged the finding the radiologist missed, 34.0 percent when the AI's false-omission rate was also disclosed, 48.8 percent when the AI disagreed and its false-discovery rate was disclosed. Cancer vignette: 65.2, 78.7, 56.4 and 73.1 percent. Online lay respondents, 652 to 682 per arm. https://www.auntminnie.com/imaging-informatics/artificial-intelligence/article/15747540/if-ai-finds-an-abnormality-that-a-radiologist-misses-whos-at-fault ↩

  3. Song, Bernstein, Sheppard, Bruno, Baird. Brown Radiology, Psychology and Law Lab, medRxiv preprint posted 22 May 2026, not peer reviewed. 2,347 participants recruited through a paid research panel, 22 workflow conditions. Perceived liability reduced by double reading (p<0.0001), documenting the initial interpretation before AI review (p=0.0125), disclosing the AI's false-discovery rate (p=0.0038) or false-omission rate (p=0.0035); highest under the erroneous-change penalty. https://www.medrxiv.org/content/10.64898/2026.05.20.26353717v1.full.pdf ↩

  4. Ross. Mock juries, real trials: how to solve (some) problems with jury science. Journal of Law and Society 51(3), 2024. https://researchonline.lse.ac.uk/id/eprint/122243/3/Ross_mock-juries-real-trials--published.pdf ↩

  5. Kazakov. The new standard of care? AI and the future of medical malpractice law. Suffolk University Journal of Health and Biomedical Law, 25 January 2026. https://sites.suffolk.edu/jhbl/2026/01/25/the-new-standard-of-care-ai-and-the-future-of-medical-malpractice-law/ ↩

  6. Lee, Naftalovich, Ali, Chaudhry, Tewfik. Liability exposure of clinicians in artificial intelligence-driven healthcare. Biomedical Instrumentation and Technology 58(2), 2024. https://pmc.ncbi.nlm.nih.gov/articles/PMC10987009/ ↩

  7. Goldberg-Hoss. Hindsight bias in radiology miss cases. Chemnick Moen Greenstreet, 1 April 2013, discussing Ketchum v. Overlake Hospital Medical Center, 60 Wn. App. 406 (1991). https://cmglaw.com/articles/hindsight-bias-in-radiology-miss-cases-a-get-out-of-jail-free-card-for-negligent-radiologists/ ↩

  8. Deutsches Ärzteblatt. Künstliche Intelligenz: Was die EU-KI-Verordnung für die ärztliche Haftung bedeutet. Issue 1/2025. Liability under BGB 823 and 630; the AI Act adds oversight and documentation duties without a liability rule of its own. https://www.aerzteblatt.de/archiv/kuenstliche-intelligenz-was-die-eu-ki-verordnung-fuer-die-aerztliche-haftung-bedeutet-2b1f34b8-9f76-4c44-86af-919de73bf028 ↩

  9. IAPP. European Commission withdraws AI Liability Directive from consideration. 12 February 2025. https://iapp.org/news/a/european-commission-withdraws-ai-liability-directive-from-consideration ↩

  10. National Transportation Safety Board. Press release, 31 March 2026: driver overreliance on a partial-automation system contributed to two fatal 2024 crashes in which neither the driver nor the system braked. https://www.ntsb.gov/news/press-releases/Pages/NR20260331.aspx ↩

  11. American College of Radiology. FDA updates guidance on clinical decision support. 14 January 2026. Software analysing medical images to generate a diagnostic recommendation stays within device regulation. https://www.acr.org/News-and-Publications/2026/fda-updates-guidance-on-clinical-decision-support ↩

  12. Georgetown Health Care Litigation Tracker. Estate of Gene B. Lokken et al. v. UnitedHealth Group Inc. et al., current as of 15 September 2026: a Medicare Advantage class action over an algorithm used to end post-acute care coverage; motion to dismiss partly granted, unresolved. https://litigationtracker.law.georgetown.edu/litigation/estate-of-gene-b-lokken-the-et-al-v-unitedhealth-group-inc-et-al/ ↩