AI
Researchers develop multi-turn diagnostic AI for iterative clinical reasoning
A new dataset and fine-tuning approach helps medical LLMs make decisions like real doctors do—by ordering tests and updating diagnoses step by step.
Most medical AI systems are trained and tested on idealized scenarios where all patient information arrives at once. Researchers from arXiv cs.CL argue this misses a core challenge of real clinical practice: physicians order tests, gather results over time, and refine their diagnosis iteratively.
The Gap in Current Medical AI#
Today's large language models typically struggle in this active diagnostic setting. The researchers identified three recurring failure modes:
- Ungrounded test ordering — models suggest tests without sound reasoning
- Unreliable diagnostic updates — models fail to adjust their hypothesis based on new evidence
- Degraded multi-turn coherence — models lose track of their reasoning across multiple back-and-forth exchanges
The root cause, the team suggests, is that most medical training data teaches models to infer conclusions from complete information, not to navigate partial evidence and update beliefs dynamically.
MedAction: A New Dataset and Pipeline#
To bridge this gap, researchers developed MedAction, a system that synthesizes high-quality multi-turn diagnostic scenarios. The approach uses a tree-structured pipeline that simulates LLM-environment interaction, drawing from 2,896 medical cases to generate 32,681 diverse diagnostic trajectories.
The pipeline filters trajectories using two knowledge-graph-grounded metrics:
- Disease Trajectory Consistency (DTC) — measures whether the model's hypothesis gradually converges on the correct diagnosis
- Reasoning-Action Consistency (RAC) — ensures that belief updates are actually driven by evidence gathered during the dialogue
Results#
An 8-billion-parameter model fine-tuned on the MedAction-32K dataset achieved state-of-the-art performance among open-source medical AI systems on both standard benchmarks and a new curated test set. The work demonstrates that structured multi-turn diagnostic trajectories can meaningfully improve how medical LLMs reason under uncertainty.
The dataset and approach are part of a broader effort to align AI diagnostic behavior with actual clinical workflows.

