AI
Larger language models produce more truthful explanations of their decisions, study finds
Research analyzing 75 AI models reveals that bigger language models offer more reliable self-explanations, though a tension persists between detail and brevity.
When large language models are asked to justify their choices, they typically generate explanations that sound reasonable to human readers. A new study investigates whether those explanations actually reveal the real factors behind the decisions—a property known as "faithfulness."
Measuring truthfulness in AI explanations#
Researchers examined 75 models across 13 different model families, assessing how faithfully their explanations matched their actual decision-making processes. The work introduces a critical distinction: an explanation can sound plausible without being truthful. The team focused on counterfactual faithfulness, which measures whether explanations remain consistent when relevant inputs are altered.
The analysis exposes a key tension in AI explanation: models must balance between being concise and being comprehensive. Shorter explanations may be easier to understand but might omit crucial reasoning steps, while longer ones can capture more details but risk obscuring the core logic.
New methods for testing explanation quality#
The researchers proposed two novel metrics to better evaluate model explanations:
- phi-CCT: A streamlined version of the Correlational Counterfactual Test that removes the need for token probability calculations, yet preserves most of the original test's analytical power
- F-AUROC: A metric that neutralizes the effects of imbalanced data distributions and measures a model's ability to generate explanations at varying levels of detail
These tools also reveal how easily explanation metrics can be manipulated—an important finding for understanding the robustness of evaluation methods.
Scale drives faithfulness#
The most striking discovery emerging from the research is a consistent scaling pattern: more capable and larger models consistently produce more truthful explanations across all tested metrics. This trend suggests that as language models grow in size and capability, they develop a stronger ability to articulate the actual reasoning behind their outputs.
The research code is available for further study and validation by the broader AI research community.
