New Evaluation Metric Exposes Language Models' Tendency to Guess Incorrectly

Standard ways of evaluating language models reward confident predictions, creating an incentive structure that punishes models for expressing doubt — even when that doubt is warranted. Researchers have proposed a new metric that fundamentally rethinks how we score model responses, one that treats guessing and genuine uncertainty very differently.

The Problem with Traditional Scoring#

Current evaluation approaches assign grades based solely on whether a model's single best guess is right or wrong. Under this system, a model with high uncertainty faces a perverse incentive: it's better off making a bold, unfounded claim than admitting "I don't know." Even penalty-based methods that raise the bar for guessing fail to capture something crucial — they ignore how a model distributes its remaining confidence across other possible answers. A model hedging between two equally wrong options differs fundamentally from one hedging between a wrong answer and "I don't know," yet traditional metrics treat both the same way.

A New Approach to Uncertainty#

The researchers introduce a distributional uncertainty scoring system that examines a model's entire probability distribution across all available options. This approach naturally distinguishes between two forms of hedging: harmful overconfidence in incorrect answers versus genuine uncertainty expressed through abstention or "I don't know" responses. The metric produces interpretable scores in a familiar range and creates the right incentives — rewarding models that acknowledge what they don't know rather than penalizing them for caution.

Testing Across Benchmarks and Models#

The team adapted 12 existing evaluation benchmarks to work with their new metric and tested six language models across them. The results were sobering: on roughly half of the benchmarks tested, every single model scored negatively. This signals widespread hallucination tendencies — models confidently committing to incorrect answers rather than expressing genuine uncertainty. The theoretical analysis and worked examples demonstrate that this evaluation framework offers a more nuanced and alignment-friendly alternative to existing paradigms.

Through this research, the authors make the case that how we evaluate models shapes how models behave. By better accounting for uncertainty, evaluation metrics can guide language models toward more honest and reliable outputs.