Study uncovers major language gaps in healthcare AI reliability

A new study shows that large language models give significantly more reliable medical information when responding to questions in English compared to other languages, even when the user explicitly asks in German, Turkish, Chinese, or Italian. This disparity raises important questions about equitable healthcare access in an increasingly multilingual world.

The problem: unequal information across languages#

The research examined how well healthcare knowledge translates across languages in AI systems. The investigators built a multilingual dataset from Wikipedia covering five languages—English, German, Turkish, Mandarin Chinese, and Italian—and then tested how accurately commercial LLMs retrieved and presented health information in each. They found that despite being asked in non-English languages, the models consistently pulled from and aligned with English-language sources rather than offering locally relevant medical knowledge from their other language databases.

What the disparities reveal#

The findings point to deeper structural inequalities. Healthcare coverage itself differs dramatically between Wikipedia versions: English entries tend to be more comprehensive than those in other languages. When LLMs were tested, they compounded this problem by anchoring their answers to English knowledge even when better local sources existed. This means speakers of other languages may receive less accurate, less culturally adapted health information from these systems—a particular concern for populations that may already face healthcare access challenges.

A potential fix: using local context#

The researchers tested a method to improve fairness: supplying LLMs with relevant excerpts from non-English Wikipedia sources during their inference process. This simple intervention—a technique called Retrieval-Augmented Generation (RAG)—redirected the models toward culturally and linguistically appropriate information. When given explicit access to non-English healthcare sources, the systems shifted their answers toward that content, demonstrating that the bias wasn't inevitable but rather a consequence of training and retrieval architecture.

Why this matters for healthcare AI#

As healthcare systems increasingly adopt AI-powered tools for patient education, diagnosis support, and health information, the language-based disparities identified in this work have real consequences. Ensuring that multilingual populations receive equally reliable information is fundamental to equitable healthcare delivery. The research offers a practical pathway: integrating locally relevant knowledge sources into AI retrieval systems can meaningfully reduce these gaps.