LLMs Reveal Striking Gender and Specialty Gaps in How Patients Rate Physicians Nationally

Researchers have deployed large language models to extract a comprehensive picture of how patients perceive their doctors, analyzing millions of online reviews to map physician traits across the entire U.S. clinical workforce. The work raises important questions about bias in AI-mediated healthcare decisions.

The Study#

A team analyzed reviews for over 226,000 physicians from a dataset of one million U.S. doctors, parsing 4.1 million patient comments to score clinicians on ten dimensions: five aligned with the Big-Five personality model (openness, conscientiousness, agreeableness, extraversion, neuroticism) and five patient-specific traits. The researchers validated their LLM-based extraction pipeline by comparing model outputs with human expert ratings, confirming that the AI assessments aligned well with human judgment. Trait scores also showed strong correlations with overall review ratings while maintaining meaningful independent variance.

Key Findings#

Two striking national-scale patterns emerged from the data:

• Gender disparity: Male physicians scored higher across all measured traits, with the widest gap appearing in perceived clinical competence. This disparity persisted even after controlling for other factors.

• Specialty effects: Interpersonal qualities and clinical perception vary significantly by medical specialty. Surgical fields showed stronger interpersonal trait scores, while psychiatry ranked lowest on these dimensions—a gap the researchers attribute to differences in encounter context and patient expectations across specialties.

Physician Archetypes#

Cluster analysis identified four distinct physician profiles:

• Uniform High (33.8%): Consistently high trait scores across all dimensions
• Uniform Low (22.6%): Consistently low scores
• Two additional clusters representing mixed trait patterns

Why It Matters#

As patients increasingly rely on LLMs to evaluate physician reviews and guide provider selection, this research exposes both the capabilities and potential pitfalls of AI-driven healthcare decision support. The gender and specialty biases embedded in patient reviews—and subsequently amplified by LLM processing—highlight fairness concerns that demand attention. While the findings require clinical validation, they point to how algorithmic intermediaries may reshape patient choice and reinforce existing healthcare inequities.