AI
Study reveals how LLMs secretly compute confidence before users ask
New research shows language models pre-compute uncertainty measures during answer generation rather than calculating them on demand, suggesting sophisticated built-in self-evaluation.
When researchers ask large language models to express confidence in their answers as a number or rating, the models appear to have a ready-made response. A new study reveals that this "verbal confidence" isn't being constructed in the moment — instead, it's being retrieved from internal computations that happened automatically during answer generation.
How confidence gets stored#
Computer scientists studying Gemma 3, Qwen 2.5, and other models used a series of experiments to trace where confidence information lives inside the network. Using techniques like activation steering and attention blocking, they found that confidence representations emerge at token positions immediately following the answer itself, before being cached at the first position after the answer ends. When the model is later asked to state its confidence verbally, it simply retrieves this pre-computed value rather than generating a new estimate.
Beyond simple probability estimates#
The research suggests that cached confidence goes deeper than just measuring how likely particular tokens are — a metric easily derived from standard language model outputs. Through linear probing analysis and variance partitioning, the team demonstrated that these internal confidence signals carry additional information that explains substantial variance beyond what token-level probabilities alone would provide. This indicates that models perform a richer, more nuanced evaluation of answer quality automatically as they generate responses.
Implications for AI reliability#
Understanding how confidence actually works inside language models matters for building safer, more trustworthy systems. If models are already conducting sophisticated self-evaluation during generation, this metacognitive capability could potentially be enhanced to improve calibration — ensuring that when a model claims high confidence, its answers are genuinely more likely to be correct. The findings suggest that verbal confidence reflects genuine internal reasoning about answer quality rather than post-hoc reconstruction, opening new avenues for improving how well AI systems understand their own limitations.
