Researchers Build Multilingual Medical AI to Answer Questions Beyond English

Researchers have developed a new approach to help large language models answer medical questions reliably across multiple languages. The work addresses a significant gap: most medical AI systems today are built for English speakers and often rely on distilled knowledge from general-purpose models rather than medical expertise.

The Dataset and Methodology#

The team created 500,000 reasoning traces spanning English, Italian, and Spanish by combining retrieval-augmented generation with medical information from Wikipedia. Rather than having LLMs generate answers on their own, they designed a pipeline that pulls relevant medical knowledge first, then generates step-by-step reasoning to solve questions from established medical QA benchmarks. The researchers also translated existing medical question datasets—MedQA and MedMCQA—into the two Romance languages, expanding the available training material.

Performance Gains#

Testing across both in-domain benchmarks (questions similar to training data) and out-of-domain scenarios showed consistent improvements. The reasoning traces boosted performance whether applied through few-shot prompting or full supervised fine-tuning. The approach achieved state-of-the-art results for 8-billion-parameter LLMs, suggesting that grounding in explicit medical knowledge can rival larger, less transparent systems.

Why It Matters#

Medical AI must inspire confidence in clinical settings, and transparency is crucial. By making reasoning traces explicit and anchoring them to recognized medical sources, the system becomes easier to audit and trust. The multilingual dimension is equally important—it means non-English speakers can access comparable medical decision-support tools. The team is releasing the full suite of resources, including the reasoning traces, translated datasets, extracted medical Wikipedia content, and fine-tuned models, to accelerate development of trustworthy clinical tools worldwide.