Calibration alone misses the mark: New framework reveals blind spots in how LLMs estimate confidence

Researchers have identified a significant gap in how we evaluate whether large language models truly understand their own reliability. While existing methods focus on whether confidence levels align with correctness, they overlook a critical insight: models should maintain steady confidence when prompts are reworded or answers are restated identically—and shift confidence when meaning actually changes.

The confidence estimation problem#

When users interact with language models, they depend on confidence scores to judge trustworthiness and make decisions. But current evaluation approaches examine only one dimension: whether high confidence correlates with correct answers. This narrow lens misses broader questions about consistency and meaningful variation.

A three-part framework for better evaluation#

The new framework measures three complementary properties:

• Robustness: Confidence should remain stable when prompts are semantically equivalent but linguistically varied
• Stability: Confidence should stay consistent across different ways of expressing the same answer
• Sensitivity: Confidence should shift noticeably when answers differ in actual meaning

These three metrics reveal that they operate largely independently from traditional confidence estimation measures—suggesting the field has been measuring different aspects of the problem than intended.

Where current methods fall short#

Testing common confidence estimation approaches exposed a striking pattern. Most perform well on robustness and stability, maintaining confidence across harmless variations. However, they frequently fail the sensitivity test: they cannot reliably distinguish between answers that mean different things. Researchers suspect this gap stems from methods not fully leveraging information from the generation process itself.

Implications for deployment#

The framework exposes overlooked weaknesses in current evaluation practices and offers practical guidance for choosing confidence estimators suited to real-world use cases—where consistency across variations and meaningful sensitivity both matter.