New Technique Maps How LLMs Blend Training Knowledge with Context to Generate Explanations

Large language models are often asked to explain their reasoning by generating natural language explanations that draw on two distinct sources: the knowledge embedded in their parameters during training, and the external context provided in their prompts. Yet how these two knowledge sources interact during multi-step generation remains poorly understood—a gap that matters for verifying whether explanations are grounded in facts or hallucinated.

A Richer Lens for Knowledge Dynamics#

Prior research has treated the interaction between parametric knowledge and context knowledge as a binary choice within a simple one-dimensional mathematical space. This approach misses important nuances: knowledge can be complementary, supportive, or conflicting. Researchers have now proposed a rank-2 subspace framework that operates in two dimensions instead of one, allowing for a more precise decomposition of how these knowledge sources contribute to each generated token.

Testing Across Multiple Models and Datasets#

The team evaluated their technique on four question-answering datasets using three open-weight language models. The experiments revealed a critical limitation of the rank-1 approach: single-dimensional models struggle to capture the diversity of knowledge interactions. By contrast, the rank-2 formulation successfully identifies distinct patterns—parametric knowledge alignment correlates with supportive interactions, while context knowledge alignment marks conflicting ones.

Insights Into Hallucination Patterns#

A key finding emerged from analyzing full generation sequences rather than isolated steps: when models produce hallucinated responses, their outputs show strong alignment toward the parametric knowledge direction. In contrast, when models generate contextually faithful explanations, the alignment remains more balanced between the two knowledge sources. This distinction could help identify unreliable generations before they reach users.