AI
Retrieval beats loading everything: RAG outperforms long-context for EHR clinical tasks
A new study shows that targeted information retrieval remains more efficient than feeding entire patient histories into AI models, even as language models grow stronger.
Researchers tested whether retrieval-augmented generation (RAG)—a technique that fetches relevant information on demand—could serve as a leaner alternative to giving AI models entire electronic health records to analyze at once. The answer, across several real-world clinical tasks, is yes.
The three clinical challenges#
The team evaluated three representative EHR tasks that hospitals face regularly:
- Imaging procedure extraction: identifying the type of imaging (CT, MRI, X-ray), the date it was performed, and which body part was scanned
- Antibiotic timeline reconstruction: piecing together when patients were given which antibiotics during their stay
- Diagnosis identification: determining the primary diagnoses documented for a hospitalization
All three tasks were tested using real inpatient notes from a US academic health system, with three different AI models: GPT-5.4-mini, Mistral Medium 3, and DeepSeek V3.1.
RAG wins on token efficiency#
The key finding: RAG delivered superior results while using far fewer tokens. For imaging extraction, targeted retrieval beat recent-note baselines and even matched or exceeded long-context performance, using fewer than 8,000 tokens. Performance gains ranged from 0.17 to 9.83 points on the F1 metric depending on the model.
Antibiotic timeline tasks showed similar patterns. Retrieving fewer than 8,000 relevant tokens achieved performance levels that required much larger context windows when the models simply read the most recent clinical notes.
Where the approach hit limits#
Diagnosis generation proved the hardest task—performance remained largely flat regardless of retrieval strategy or model choice. The researchers attributed this partly to variability in how clinicians document diagnoses and partly to limitations in how the evaluation was structured.
They also noted that missing information in clinical notes—often because patients had transferred between hospitals—acted as a ceiling on performance across all methods.
Why it matters#
As large language models grow capable of ingesting longer documents, the assumption might be that more context is always better. This study suggests that's not the case in clinical settings. RAG remains a computationally efficient and effective approach that can serve patients and healthcare systems well, avoiding unnecessary token consumption while maintaining or improving accuracy on key documentation tasks.
