Skip to content
Reza Motaghi
Lab Notes3 min read

A search engine can make a medical AI worse

By Reza Motaghi

A search engine made the medical AI's answers less accurate. That is the published result I read before building one.

In October I planned a search agent for my radiology AI, for the rare cases that sit outside the textbooks. The model I am building reads my own cases, and the rare ones are where it has least to go on. Before building anything, I had the published evidence opened in full.

Here is what it says. In a 2025 preprint, medical residents and fellows rated a strong general model's long answers less factual when a standard medical retrieval system was bolted on, a drop of about six points.1 Only 22 percent of the top sixteen passages it pulled were judged relevant. The same paper found a cure, and the cure was not more search. A relevance filter trained on expert labels, plus a rewritten query, lifted a small model's multiple-choice accuracy by up to twelve points. A peer-reviewed study the same year found the same shape. Plain retrieval lowered a strong model's average accuracy from 86.0 to 83.1, and only filtering the evidence and guiding the query turned the loss into a gain2. Its authors put it plainly: without filtering, retrieval does not always guarantee improved performance. And in Nature Communications, even a model with web search left about 30 percent of its individual statements unsupported by the sources it cited3. A separate agent that rewrote or removed those statements fixed most of them, more than a third by deleting them.

One study cuts the other way, and I keep it. In 2026 a large comparison found that for smaller models retrieval often hurt regardless of where the text came from, expert or lay4. So the lesson is not "use expert sources". It is "filter what reaches the model", and the specialist writes the filter.

Here is the shape of it. A model given a search tool reads what the engine returns, and the engine returns what is popular. For the common question that is fine. For the rare question, popular is wrong, and the model reads the wrong page with the same confidence it reads the right one.

So the search engine is a commodity. The reading list is the expertise. Which sources count, which patients deserve a search, and when a search is finished are a specialist's judgments, not settings. I can write them. The engine cannot.

This is not a radiology rule. Any model with a search tool answers the rare question from the popular page, in law, in engineering, in finance. The exception is the one where someone who knows the field decided what it may read.

One caveat I owe the reader. Nothing is built. Our own comparison, guarded search against open search on hard cases, has not run. This note is the decision before the build, with the evidence it rests on.

What I would do differently: I read the abstracts first and nearly cited them. Two numbers changed when the papers were opened in full. Open the paper.

References

Footnotes

  1. H. Kim et al., "Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights", arXiv preprint, 2025, https://arxiv.org/abs/2511.06738. ↩

  2. J. Sohn et al., "Rationale-Guided Retrieval Augmented Generation for Medical Question Answering", Proceedings of NAACL 2025 (long papers), 12739-12753, 2025, https://doi.org/10.18653/v1/2025.naacl-long.635. ↩

  3. K. Wu et al., "An automated framework for assessing how well LLMs cite relevant medical references", Nature Communications 16:3615, 2025, https://doi.org/10.1038/s41467-025-58551-6. ↩

  4. E. Nourbakhsh et al., "When Retrieval Doesn't Help: A Large-Scale Study of Biomedical RAG", arXiv preprint (BioNLP Workshop at ACL 2026), 2026, https://arxiv.org/abs/2606.04127. ↩

About the author

DDS, MSc · Board-certified specialist in oral and maxillofacial radiology · Independent researcher

0009-0005-3194-1879

Reza Motaghi is a board-certified specialist in oral and maxillofacial radiology who reads medical images every day, 100,000+ reports signed, and runs enterprise AI in production as Chief Innovation Officer. He builds the training data from his own reads, made together with fellow radiologists under one written standard, trains models on it, and tests them on cases they have never seen.

He published Radigrade, the registered benchmark for AI-written radiograph reports: invented findings counted first, graded by an AI judge calibrated against his own verdicts, general models in public and vendors’ systems privately on request. He also built CBCTScope, the first CBCT reading viewer with native AI-agent control.

Research recordAbout RezaContactLinkedIn (opens in a new tab)

The newsletter

Get the essays as they’re written.

Imaging AI, evaluation, and building alone, from a radiologist who reads every day and tests before he trusts. A few essays a month, nothing else.

Join for FreeFree · Unsubscribe anytime

Related

  1. Inventory before interpretation

    Blind on my own radiographs, the strongest general model reported a root canal in a tooth that was not there, then kept writing. Count before you interpret.

  2. The error to fear is the treatment nothing justifies

    An AI plan line proposed a restoration for a child with no finding to justify it. One line of code caught it against a list two specialists had confirmed.

  3. What the best general model read on a specialist's ruler

    The strongest general model read three findings in ten on my cases and invented three in four. One accuracy number hides that. Three questions to ask.