Skip to content
One-Person Startup10 min read

How to catch a wrong AI answer without knowing the right one

By Reza Motaghi

Two identical compasses side by side with their needles pointing different ways, a black and white image about an instrument that contradicts itself
On this page

In 1947 the US Public Health Service gave the same chest films to five expert readers. Two to three months later it sent the same films round again. Nobody knew which patients were truly ill, and the paper says so.1 One reader had missed, on his first pass, 22 percent of the films he called positive on his second. The count of positives called from the same films ran from 50 for one reader to 100 for another. With no answer key, the authors made agreement the standard, and disagreement the finding.

That is the position of anyone who must rely on an output they cannot check. A hospital piloting an AI reader has no expert hours to label a thousand cases. A buyer has a vendor's accuracy figure and no way to test it. This article is about the test you can run with no labels at all, and about what it can and cannot tell you.

Today I build the training data from my own reads, train the model on it, test it on cases it has never seen, and ship tools I read with. Before any AI model I test gets near the cases I keep sealed for the real exam, it reads each image twice, and once mirrored. On the mirrored read the tooth numbering has to swap sides, because the truth did. The habit came from one afternoon of my own. Same image, same weights, same prompt, same seed, and a clinically opposite answer to a run made weeks earlier. Only the inference software had changed. Neither answer looked wrong on its own.

What the consistency test is

The consistency test is a set of checks that need no answer key. You change the input in a way that cannot change the truth, and you watch whether the answer changes. Read the same image twice at the same settings. Mirror it, so that left becomes right. Reorder the options, or the slices, in a way that leaves the evidence intact. Then count.

The logic is old and simple. Two answers to one question cannot both be right. A reader that gives two is wrong at least once, and you do not need to know which time. A mirrored image carries the same finding on the other side, so the answer has to move with it. A reader whose answer stays put was not reading the image.

The test is cheap because it produces contradictions, not verdicts. It takes an afternoon, no labels, and the cases you already have. It is also the test that measurement science runs first. Metrology holds an instrument to agree with itself before it asks whether the instrument is true, and it keeps two words apart.2 Precision is agreement between repeat readings. Trueness is closeness to the reference value.

Bland and Altman made the same point for clinical measurement in 1986.3 The true value is usually unknown, so you judge a new method against an old one. A method that cannot repeat itself cannot agree with anything.

An accuracy score hides a reader that contradicts itself

The strongest imaging evidence comes from brain MRI. In a preprint from August 2026, four vision-language models diagnosed 90 patients with tumours confirmed under the microscope.4 Then they diagnosed the same images again, with only the order of the two candidate answers swapped in the prompt. Two of the AI models had similar accuracy on one sequence. One changed its diagnosis in 59 percent of patients after the swap, the other in 14 percent.

Nothing on the accuracy table told those two apart. The comparison comes from the paper's own tables rather than from a sentence the authors wrote, and the gap is far wider than the study's margin of error. The paper adds a caution of its own. Many flips turned a wrong first answer into a right one, so the first answer is not the privileged one. A flip into "unsure" is a hedge, not a contradiction.

The pattern is not confined to images. In a test of seven text models across meaning-preserving rewrites of the same problems, the highest-scoring of the seven ranked sixth on stability.5 Scale did not predict robustness in either direction. On tabular clinical data, three prediction models with strong headline scores broke common-sense rules in 27 to 87 percent of pilot tests.6 A planted sign error in one input barely moved the headline score.

Fluri, Paleka and Tramèr wrote the formal version for AI models whose answers nobody can check, published at IEEE SaTML in 2024.7 Their chess example: a top engine valued a small share of mirrored boards differently. In the worst cases the winning chance moved by up to 69 percent. Their bail example: a language model asked to decide bail became more likely to grant it when someone added a felony to the record. Finding either fault needed no ground truth. The authors state plainly that passing the checks is no guarantee.

Software named the method in 1998

Software got here first, because software had the same problem. In 1982 Elaine Weyuker challenged the assumption that someone can always decide whether a program's output is correct.8 She called such programs non-testable. In 1998 a technical report from Hong Kong gave the answer a name, metamorphic testing.9 Its first worked example was a binary search that reports "not found". Nobody can confirm the absence without scanning the whole database. So the method plants a key known to be present and asks whether the program finds it. That is the software twin of a radiologist's negative read.

The report says of its own checks what imaging should say of these. A checker that rejects an output has proved it wrong. A checker that accepts one has proved nothing. Fluri and colleagues call their consistency checks an instance of the same method. Compiler builders found 147 confirmed bugs in eleven months the same way, with no reference answer for any of them.10

The consistency test, four checks

These are the four checks I run before an expert hour is spent. Each carries its evidence and the case where it does not apply. One of the four I changed after reading the evidence against it, and I wrote the change into the check.

1. The same input twice. Same settings, same day, and compare the decisions, not the wording. Two different answers is one wrong answer, and you do not need to know which. This check tests the serving stack as much as the AI model. A thousand identical requests to one open model at temperature zero came back as 80 different completions, because server load changed the arithmetic under them.11 In that example the split was a late difference in wording, not a different decision. A letter in a radiology journal this year argued that scoring an AI model on a single response hides the spread.12 Off-ramp: a server built so that the answer does not depend on the other requests in its batch returns zero disagreements by construction.13 Record the runtime beside the result, and the check still tells you something.

2. The mirrored input. Flip the image so that left becomes right. The finding is now on the other side, so the numbering has to swap. This is the one check of the four where the answer must move. On flipped and rotated abdominal CT slices, four vision-language models scored at chance on left and right when told to answer from the image.14 The strongest of them matched textbook anatomy about three times in four. It was answering from the average patient rather than the pixels, the same habit as an AI model filling a missing fact with the average. Off-ramp: run it on cases where the finding sits on one side, because a finding with no side gives the check nothing to swap. Strip burnt-in side markers and text first, because they mirror with the image and give the flip away.

3. The reordered input. Shuffle the options in the prompt, or the order of the slices, in a way that leaves the evidence intact. The truth did not move, so the answer must not. The brain MRI preprint above is the evidence. The pair of AI models with similar accuracy and four-fold different flip rates is the reason to run it on every candidate. Off-ramp: the reorder must be one the task is truly indifferent to. Reversing a stack of slices breaks the continuity a human reader also uses, and the paper's authors call it a stronger stress test. A flip on that check is a warning, not a verdict.

4. Log the disagreement rate before any accuracy claim. Write the counts on the first page of the pilot report, above any accuracy figure the vendor supplied. A high rate is a fault found, and the pilot can stop there. This is the check I changed. As I first wrote it, a low rate read as good news. The evidence says it is only permission to spend the expert's time. Across nine chest-radiograph systems audited this year, three ignored the image entirely. A text-only model came within 5.7 points of the best multimodal one.15 A second study found that low flip rates under rephrasing can mean the image was never used.16 A reader that never looked will not contradict itself when you shuffle. An AI model that learned the report and not the image passes every check that only shuffles the input, and saying less is not seeing more. So a low rate is good news only if the mirror check passed. That is the one check an image-blind reader cannot pass on a lopsided case. Off-ramp: if the mirror check could not be run, the low rate stays unread, and the pilot report says so.

The consistency test, four checks. One, the same input twice at the same settings, two different answers is one wrong answer. Two, the mirrored input, left must become right or the reader never looked. Three, the reordered input, the truth did not move so the answer must not. Four, log the disagreement rate before any accuracy claim, a low rate counts only if the mirror check passed.

What the checks are not

They are not a validation. Consistency is necessary and not sufficient, and every source above says so in its own words. Fluri and colleagues state that passing is no guarantee. The 1998 report says a checker that accepts an output has proved nothing. Metrology keeps precision apart from trueness for the same reason. A scale can read the same wrong weight forever.

They are not a substitute for graded truth. The expert is the dataset, and the expert's afternoon is what these checks protect, not what they replace. They tell you which readers have not yet earned it.

They are not a verdict on the first answer. In the brain MRI study, many flips turned a wrong answer into a right one. A flip proves that one of the two was wrong. It does not say which, and it does not say the first one was right.

The figures are not the point. Every 2026 number here comes from a preprint, and one of the studies changed its counts between versions. The mechanism will read the same in three years. The figures will not, which is why each one carries its date at the foot of this page.

What I sign for

I read medical images every day. Every AI model I test reads each of them twice and once mirrored before it earns an afternoon of my time. A test set answers honestly once, so the checks that cost nothing run first, and the sealed cases wait.

Run the four checks on the reader under your pilot this week. If the disagreement rate is high, you have your answer before the validation study does. If you build for medical images and want a reader who runs these checks on his own cases first, the contact page reaches me directly.

References

Footnotes

  1. Yerushalmy J. Statistical Problems in Assessing Methods of Medical Diagnosis, with Special Reference to X-Ray Techniques. Public Health Reports 62(40):1432-1449, 3 October 1947. Five expert readers read the same chest films twice, two to three months apart. One reader had missed on the first reading 22 percent of the films he called positive on the second, and positives called from 1,256 films ranged from 50 to 100 across readers. https://jhanley.biostat.mcgill.ca/anniversaries/ByTopic/Yerushalmy1947.pdf ↩

  2. Joint Committee for Guides in Metrology. International Vocabulary of Metrology, Basic and General Concepts and Associated Terms (VIM), JCGM 200:2012, entries 2.14, 2.15, 2.20 and 2.21. Defines precision, trueness and repeatability conditions. https://www.bipm.org/documents/20126/2071204/JCGM_200_2012.pdf ↩

  3. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986, i:307-310. When true values are unknown, the repeatabilities of two methods limit the agreement possible between them. https://www-users.york.ac.uk/~mb55/meas/ba.pdf ↩

  4. Khoramfar A, Dousti MJ, Mohamadian A, Faili H. Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations. arXiv preprint 2608.04885 v1, 5 August 2026. Four models, 90 patients with histopathology-confirmed labels, temperature 0. Under answer-option reordering two models with overlapping accuracy intervals flipped in 59.1 and 14.4 percent of cases on one sequence and 67.8 and 16.7 on another, with non-overlapping intervals (Tables 1, 2, 6 and 7, the comparison drawn from the printed cells). The paper's headline maxima all belong to one medically fine-tuned open model. Table 4 reports that between 29 and 83 percent of flips per configuration turned a wrong answer into a right one, and that some flips ended at "unsure". https://arxiv.org/abs/2608.04885 ↩

  5. de Zarzà I, de Curtò J, Cabot J, Manzoni P, Calafate CT. Semantic Invariance in Agentic AI. arXiv preprint 2603.13173 v2, 16 March 2026. Seven models, eight meaning-preserving rewrites of 19 text reasoning problems. The model with the highest score was sixth of seven on stability (50.7 percent invariant responses), a smaller model was the most stable (79.6 percent). Scale did not predict robustness. https://arxiv.org/abs/2603.13173 ↩

  6. Wu JJW, E F, Chen B. Metamorphic Testing for Clinical ML Models: A Framework Proposal and Pilot Study. arXiv preprint 2607.22984 v1, 25 July 2026. Three models with AUROC 0.849 to 0.900 violated clinical common-sense relations in 27 to 87 percent of pilot tests (61 test patients, tabular data). An injected sign-negation error raised violation rates by 31 to 67 percentage points while AUROC changed by at most 0.007. https://arxiv.org/abs/2607.22984 ↩

  7. Fluri L, Paleka D, Tramèr F. Evaluating Superhuman Models with Consistency Checks. IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2024 (arXiv 2306.09983, 16 June 2023). Position mirroring changed a top chess engine's evaluation by more than 0.05 in 0.4 percent of 400,000 positions, with the worst cases moving the winning chances by up to 69 percent. A language model deciding bail could become more likely to grant it after the record was made worse. The authors state that failing a check means something is wrong and that passing is no guarantee. https://doi.org/10.1109/satml59370.2024.00017 ↩

  8. Weyuker EJ. On Testing Non-Testable Programs. The Computer Journal 25(4):465-470, 1982. Examines the assumption that an oracle can decide whether output is correct and concludes it is often unrealistic. https://doi.org/10.1093/comjnl/25.4.465 ↩

  9. Chen TY, Cheung SC, Yiu SM. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report HKUST-CS98-01, Hong Kong University of Science and Technology, 1998 (arXiv 2002.12543, posted 28 February 2020). Derives new test cases from passing ones and works where no oracle exists. The binary search example and the line that a checker which accepts an output does not guarantee correctness are in the introduction and first example. https://arxiv.org/abs/2002.12543 ↩

  10. Le V, Afshari M, Su Z. Compiler Validation via Equivalence Modulo Inputs. PLDI 2014, 9 to 11 June 2014, Edinburgh. 147 confirmed, unique bug reports for GCC and LLVM in eleven months, found without a reference answer. https://doi.org/10.1145/2594291.2594334 ↩

  11. He H and Thinking Machines Lab. Defeating Nondeterminism in LLM Inference. Thinking Machines Lab blog, 10 September 2025. 1,000 identical temperature-0 requests to one open model gave 80 unique completions, the first divergence at token 103, traced to batch size varying with server load. Batch-invariant kernels made all 1,000 identical. https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ ↩

  12. Güzel HE. Letter to the Editor: Rethinking reference accuracy in large language models for radiology. Diagnostic and Interventional Radiology 32(4), 1 July 2026. Argues that one response per query hides the spread of outputs and that identical prompts can differ across sessions. https://dirjournal.org/articles/doi/dir.2026.264022 ↩

  13. vLLM project. Batch Invariance (developer preview documentation). Page footer 27 September 2026. A mode that makes output independent of batch size and request order, at a stated performance cost. https://docs.vllm.ai/en/latest/features/batch_invariance/ ↩

  14. Wolf D, Hillenhagen H, Taskin B, Bäuerle A, Beer M, Götz M, Ropinski T. Your other Left: Vision-Language Models Fail to Identify Relative Positions in Medical Images. MICCAI 2025 (arXiv 2508.00549, 1 August 2025). On 4,878 flipped and rotated abdominal CT slices, four models were at chance against the image on left-right questions (0.49 to 0.51). The strongest matched standard anatomy in 0.757 of answers, a finding the authors say holds for two of the four models. https://arxiv.org/abs/2508.00549 ↩

  15. Lotfinia M, Ziegelmayer S, Adams L, Truhn D, Maier A, Tayebi Arasteh S. Vision-language models for chest radiography do not always need the image. arXiv preprint 2606.17710 v2, 19 June 2026. Three of nine systems ignored the image, one was unstable and five used it selectively. A text-only model came within 5.7 accuracy points of the best multimodal model. https://arxiv.org/abs/2606.17710 ↩

  16. Sadanandan B, Behzadan V. PSF-Med: Measuring and Explaining Paraphrase Sensitivity in Medical Vision Language Models. arXiv preprint 2602.21428 v2, 12 April 2026 (v1 24 February 2026 reported different counts). Flip rates of 3 to 37 percent across nine models. Text-only baselines show that a low flip rate does not imply the image was used. https://arxiv.org/abs/2602.21428 ↩