How Average Scores Hide AI Model Weaknesses in Popular Benchmarks

Researchers have identified a fundamental problem with how language models are evaluated: aggregate benchmark scores often fail to reflect how uniformly capable these systems actually are. An unevenly distributed performance across subdomains can make a model look broadly competent when it actually excels in narrow areas while struggling elsewhere.

The Core Problem#

Language models are typically assessed using benchmark scores that average results across many different tasks. However, these summary numbers can be deceptive. If a model performs excellently on half of a benchmark's subdomains while failing on the other half, its average score might still suggest reliable, well-rounded performance. This mismatch between what aggregate numbers claim and actual capability distribution has major implications for understanding model strengths and limitations.

Introducing Benchmark Harmony#

To tackle this issue, researchers developed a metric called benchmark Harmony, which uses entropy to measure how uniformly model performance spreads across a benchmark's subdomains. They evaluated Harmony across 19 language model benchmarks spanning five different model families. Results revealed substantial variation in how representative aggregate scores actually are.

• Low-Harmony benchmarks produce aggregate scores that overstate broad competence
• High-Harmony benchmarks provide more faithful summaries of what models can actually do

What Rebalancing Revealed#

When researchers artificially rebalanced benchmarks by removing overrepresented subdomains to increase Harmony, the effects varied dramatically. Stable benchmarks like BoolQ showed little change in aggregate accuracy, while performance-critical benchmarks like PubMedQA—which assesses capabilities in medical domains—exhibited substantial and often statistically significant shifts.

The Bottom Line#

The research demonstrates that relying solely on aggregate accuracy can fundamentally misrepresent whether a language model possesses genuinely broad competence. To address this, the team recommends reporting benchmark Harmony alongside traditional aggregate scores as a diagnostic tool. This additional metric would help researchers and practitioners better understand when benchmark results reflect true, well-rounded capability versus inflated averages hiding concentrated strengths and weaknesses.