AI
New framework teaches AI radiology models when to express doubt about findings
Researchers introduce ConRad, a machine-learning technique that makes medical AI systems more trustworthy by having them attach realistic confidence scores to their diagnostic reports.
Deploying large vision-language models (LVLMs) in clinical radiology requires more than just accuracy—it demands that AI systems signal when outputs warrant careful review, helping radiologists catch hallucinated findings before they reach patient care. A common strategy is to have models verbalize their confidence, but current systems tend to overstate certainty in ways that could mislead clinicians.
A new framework called ConRad (Confidence Calibration for Radiology Reports) addresses this gap by fine-tuning medical AI to produce realistic, clinically aligned confidence estimates alongside radiology reports.
How ConRad works#
The system uses reinforcement learning to teach models to express calibrated confidence in two formats: a single score for the entire report, or sentence-level scores that attach confidence to individual clinical claims. The training relies on the GRPO algorithm paired with reward functions based on logarithmic scoring rules—a mathematically principled approach that penalizes overconfidence and underconfidence equally, ensuring the model learns to be truthfully self-aware rather than defaulting to high certainty.
Clinical alignment and real-world performance#
In experiments, ConRad substantially improved how well the model's stated confidence matched actual accuracy. When tested with clinicians, the framework's report-level confidence scores aligned well with how radiologists themselves judged the quality of the AI output. By flagging entire reports or low-confidence statements for targeted review, ConRad offers a practical pathway to safer clinical deployment of AI-assisted report generation.
Why it matters#
As AI systems take on larger roles in diagnostic workflows, the ability to communicate uncertainty becomes as important as accuracy itself. ConRad demonstrates that with the right training approach, models can learn to express doubt in ways that resonate with human experts and support selective, risk-aware verification workflows.
