Skip to content
Reza Motaghi
One-Person Startup10 min read

Why an AI second reader is not twice as safe

By Reza Motaghi

Two mirrors facing each other with one candle between them, a black and white image about two readers who only reflect one another
On this page

For fifteen years American insurers paid for a second reader that did not help. Computer-aided detection software marked suspicious spots on mammograms, and Medicare reimbursed it from 2000. By the time anyone measured it properly, it sat on more than nine in ten screening mammograms in the country. Then a study of 625,625 mammograms found the software improved no measure of accuracy.1 Among radiologists who read both with and without it, sensitivity was lower with the marks on the screen.

A second check that adds nothing is the good case. The bad case is a second check that makes the pair worse than the better reader alone. Across 106 experiments of people working with AI, that was the average result on decision tasks.2

I read medical images every day, and I evaluate frontier vision models blind, against a two-radiologist reference. My own read is on paper before I open the model's. That is not distrust. It is the one condition that makes a second opinion worth anything. It is also the first thing a workflow loses when nobody writes it down.

What the second check is

The second check is a wiring decision, not a headcount. Two readers are safer than the better one alone only under conditions that can be named, whether the pair is two humans, a human and a model, or a reviewer and a bot. The reads are independent. Each is better than a coin on the case at hand. Somebody pays for the disagreements. Remove any one of the three and the pair can score below its stronger member while looking, on paper, like twice the scrutiny.

The mathematics is older than the profession. In 1785 the Marquis de Condorcet showed that a jury of independent judges, each more likely right than wrong, grows more accurate as it grows larger.3 The same theorem runs backward for judges worse than a coin, and it says nothing at all for judges who are correlated. Two readers who share a screen are correlated. They are one reader with two signatures.

The read that had already seen the answer

The receipt is a habit, not a case. Evaluating a vision model on my own reads, I compare its report to mine finding by finding. The comparison is only worth making if my read existed before I saw the model's. Once I have read a fluent report that names a lesion, I cannot un-see it on the image. My read would drift toward the model's, and the agreement I measured afterward would be the model's agreement with itself.

So the rule is mechanical. I write first. Then I open the model. Whoever looks first does the reading, which is the whole of the cold read. This article adds the arithmetic on the other side. What happens to the pair when that order is kept, and what it costs when it is not.

Where the arithmetic comes from

The meta-analysis is the place to start, because it is the strongest evidence against the pair and it carries its own correction. Across 106 studies and 370 effect sizes, human plus AI combinations scored below the better of the two alone on decision tasks.2 But the direction flipped on who was already stronger. When the human alone beat the AI alone, adding the AI helped. When the AI alone was better, adding the human hurt. Most of the tasks were generic, fake reviews and satellite images, so the number is a warning about wiring, not a verdict on any profession.

Screening mammography then ran the experiment at scale, and the results split by design. In two British centres, AI replaced the second human reader for 45,602 women.4 Sensitivity and specificity held, and reading work fell by roughly half. The disagreements did not fall.

Arbitration is the third opinion on every case the two readers split on. At one of the two centres it rose 142 percent. Across thirteen European studies of the same wiring, it rose by three to four times.5

The safety did not vanish. It moved to the arbitration room and sent a bill.

Two other wirings paid no such bill. In the same synthesis, triage designs let the AI decide which cases need a human at all. They cut initial reads by half or more with detection held.5 In Germany, an AI safety net sat on top of standard double reading for 463,094 women. It fired only when both humans had called a case normal and the AI disagreed. Cancer detection rose 17.6 percent and recall did not.6

In Sweden, a randomised trial put one reader plus AI against two humans for 105,934 women. Interval cancers fell 12 percent, and specificity was identical in both arms.7 Reading work fell by nearly half.

Same label, AI second reader. Three wirings, three bills.

One more study explains why the order matters, not only the count. In a French cohort, second readers were sometimes told which flags came from AI. When told, they corrected the AI's errors in 19 percent of cases, against 14 percent when the source was hidden.8 Disclosure changed the read. A 2026 preprint then tested the cleanest wiring of all. Elicit the human and the AI independently, and call in a second human only when they disagree. It beat the standard AI-advises-first workflow on every one of ten datasets.9 It is a preprint, tested offline, so treat it as a strong bet rather than a proven protocol.

Every field has a second reader

The wiring question is not a radiology question. A code-review bot flags style reliably and misses injection and deserialisation flaws, so the study that tested it concluded the human audit stays mandatory.10 Barings Bank had a front office and a back office on paper. One trader ran both, an internal audit had said so, and the bank was gone within a year of nobody acting on it.11 After a co-pilot alone in the cockpit crashed an airliner in 2015, European regulators required two crew in the cockpit at all times.12 A second person is a safety measure against one unsupervised actor. It is not a second read of anything.

Content moderation runs the mirror design at the largest scale there is. One platform reported that its automated systems actioned 94 percent of violating posts with no human review.13 Researchers now build models for deciding which cases to escalate to a person, because routing too much or too little to humans wastes the only scarce review capacity there is. That is the triage wiring, and it works or fails on the same arithmetic as a screening programme.

Four decisions before you add a second check

The decisions come before the first case is read, because the wiring cannot be changed afterward without re-running the pilot. One of them I changed after reading the evidence against my first version.

1. Name the failure the second check is for. Misses or inventions, not both. A check tuned to catch what the first reader missed raises the number of things asserted, and the inventions collect in the arbitration room. The screening trials that worked chose one: the safety net was built for misses only, and fired only on a case both humans had cleared. I keep the same discipline when I grade a model's findings, counting the inventions before the hits. Off-ramp: a triage gate is for misses by definition, so name the miss rate it is allowed and skip the rest.

2. Keep the reads independent in time. The second reader writes before seeing the first, or after both first reads are closed, but never with the other on screen. My first version of this decision said every second check must be blind. The German safety net is not blind, it fires after both humans, and it is the wiring with the cleanest result. So the rule became: keep them independent, in whichever order the design chooses, and write down which order you chose. Off-ramp: none. A second reader that has seen the first is a second signature, and the French disclosure study shows even knowing the source moves the read.

3. Measure the pair, never each alone. A pair can score below the better of the two. So a vendor's accuracy number for the model is not a safety claim about the workflow, and neither is your own number for the reader. Only the pair's number is. The Swedish trial measured interval cancers in each arm, which is the pair's outcome, not the model's. Off-ramp: if the pilot is too small to measure the pair's outcome, measure the pair's disagreements. That count exists from the first week.

4. Budget the arbitration before go-live. Someone senior owns the disagreements, and their count is on the dashboard from day one, because that is where the moved safety lands. The British centres held their accuracy and paid with a 142 percent rise in third opinions. That is a fair trade if it was planned and a failure if it was discovered. Off-ramp: a triage wiring has no arbitration, so budget the audit of the cases the AI cleared instead. Somebody still has to read a sample of what nobody read.

The second check, four decisions: name the failure it is for, keep the reads independent in time, measure the pair never each alone, budget the arbitration before go-live

What the decisions are not

They are not an argument against AI second readers. Two of the three wirings beat standard double reading on real outcomes across more than half a million women. One of them did it in a randomised trial. The decisions exist because the third wiring, the one that sounds most like a second reader, is the one that sends the bill.

They are not a claim that arbitration is waste. A third opinion on a real disagreement is where cancers get caught. In the British study, some AI recalls that arbitration overruled turned out to be cancers found later. The bill is real in both directions. Budget it, do not hide it.

They are not a rule about humans over machines. The meta-analysis says the pair helps when the human is the stronger party and hurts when the machine is. The decisions do not care which is which. They care that the two reads never share a screen and that somebody owns the gap between them.

What I sign for

I read medical images every day and I evaluate frontier vision models blind, against a two-radiologist reference, my read on paper before theirs is open. Twenty years of double reading taught the field that the second reader helps only when independent of the first. Testing models the same way taught me the machine is no different. The tone of its report is not a channel worth listening to.

A pair is safer than the better reader alone when it is wired to be. That is a design decision, made before the first case. It is the same decision I made in my own CBCT viewer, where the agent moves the camera and the human reads. Before the next review layer ships, ask who owns the disagreements. If you build or evaluate models that read images and want a radiologist who builds on your side of the table, the contact page reaches me directly.

References

Footnotes

  1. Lehman CD et al. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Internal Medicine, 2015. 625,625 mammograms, 271 radiologists, 66 facilities; sensitivity 85.3 percent with the software against 87.3 without among radiologists who read both ways. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2443369 ↩

  2. Vaccaro M, Almaatouq A, Malone T. When combinations of humans and AI are useful: a systematic review and meta-analysis. Nature Human Behaviour, 2024. 106 studies, 370 effect sizes; losses concentrated in decision tasks; combinations helped when the human alone outperformed the AI alone (g plus 0.46) and hurt when the AI did (g minus 0.54). https://pmc.ncbi.nlm.nih.gov/articles/PMC11659167/ ↩ ↩2

  3. Condorcet's jury theorem, 1785, as summarised in the encyclopedia entry; the original text was not consulted. https://en.wikipedia.org/wiki/Condorcet%27s_jury_theorem ↩

  4. Warren LM, Venton A, Young KC et al. Impact of using artificial intelligence as a second reader in breast screening including arbitration. Nature Cancer, 2026. 45,602 women, two NHS centres; sensitivity 49.2 versus 48.0 percent, specificity 96.8 versus 96.5; arbitration 891 to 2,158 cases at one site and 2,558 to 3,125 at the other; some arbitration-overruled AI recalls were later confirmed as cancers. https://pmc.ncbi.nlm.nih.gov/articles/PMC13035470/ ↩

  5. Sossavi, Roy, Molière. Artificial intelligence in breast cancer screening: a systematic review and meta-analysis of integration strategies. European Journal of Radiology Open, 2026. Thirteen studies, 1.03 million screens, 2017 to 2024; triage cut initial reads 44 to 70 percent and recalls 11 percent at parity; independent parallel reading raised arbitration 270 to more than 300 percent. https://pmc.ncbi.nlm.nih.gov/articles/PMC12818982/ ↩ ↩2

  6. The PRAIM study, Nature Medicine, 2025. 463,094 women in German screening, an AI safety-net alert on top of double reading; cancer detection 6.7 versus 5.7 per 1,000; recall 37.4 versus 38.3 per 1,000. Read via CancerNetwork and Medical Xpress. https://www.cancernetwork.com/view/ai-supported-mammography-screening-leads-to-increased-breast-cancer-detection-rate ↩

  7. The MASAI randomised trial, The Lancet, 2026. 105,934 women; interval cancers 1.55 versus 1.76 per 1,000 (a noninferiority trial, the reduction a secondary finding); sensitivity 80.5 versus 73.8 percent; specificity 98.5 in both arms; reading workload down 44 percent. Read via The ASCO Post and the PMC commentary. https://pmc.ncbi.nlm.nih.gov/articles/PMC13036691/ ↩

  8. Sossavi et al. Human-AI interaction in a cancer-enriched double-reading breast screening cohort. Cancer Imaging, 2026. Second readers corrected AI-origin errors in 19.1 percent of cases when told the source, 13.6 when it was hidden. https://pmc.ncbi.nlm.nih.gov/articles/PMC12910764/ ↩

  9. Berger, Analytis, Satopää, Kurvers. Beyond AI advice: independent aggregation boosts human-AI accuracy. arXiv preprint, version 2, April 2026, not peer reviewed, tested on offline datasets. https://arxiv.org/abs/2603.29866 ↩

  10. Amro, Alalfi. GitHub's Copilot code review: can AI spot security flaws before you commit? arXiv preprint, September 2025. https://arxiv.org/abs/2509.13650 ↩

  11. Reserve Bank of Australia Bulletin. Implications of the Barings collapse for bank supervisors. November 1995. https://www.rba.gov.au/publications/bulletin/1995/nov/1.html ↩

  12. European Union Aviation Safety Agency. EASA recommends minimum two crew in the cockpit. 27 March 2015. https://www.easa.europa.eu/en/newsroom-and-events/news/easa-recommends-minimum-two-crew-cockpit ↩

  13. EUobserver. TikTok used AI in nearly 100 percent of violating-content moderation in Europe. 2 March 2026, on the platform's own transparency report (93.8 percent automated, a self-reported figure). Bachar et al. LLM performance predictors: learning when to escalate in hybrid human-AI moderation systems. arXiv, January 2026. https://euobserver.com/205083/tiktok-used-ai-in-nearly-100-of-violating-content-moderation-in-europe and https://arxiv.org/abs/2601.07006 ↩