How LLMs Geometrically Reshape Truth Representations When Given Context

Researchers have uncovered how large language models internally reshape their representations of truth when given contextual information. Rather than treating truth as a fixed concept, these models appear to dynamically adjust their truth vectors—the mathematical encodings that represent whether a statement is true—based on the surrounding context.

The Geometric Picture of Truth in Neural Networks#

The study tracked how truth vectors (vectors in a model's residual stream activations) behave across different layers of the network. Across four different LLMs and four datasets, the researchers measured both the directional shift between truth vectors with and without context, and how the magnitude of these vectors changed. The results revealed a consistent pattern: truth vectors begin nearly orthogonal in early layers, gradually converge in middle layers, and then either stabilize or continue strengthening in later layers.

Context Amplifies the Signal#

Adding context generally amplifies the separation between true and false representations in the activation space. This means that when models receive contextual information, they increase the "strength" of their truth signals, making it easier to distinguish between correct and incorrect statements at the representational level.

Size Matters: Larger Models Handle Context Differently#

An intriguing finding emerged when comparing models of different scales. Larger models rely primarily on directional changes—shifting where their truth vectors point—to distinguish relevant from irrelevant context. Smaller models, by contrast, accomplish this distinction through magnitude variations, changing how strong their truth signals are rather than their direction.

When Context Conflicts with Internal Knowledge#

The research also showed that contradictory context produces more dramatic geometric transformations than context aligned with a model's learned parameters. This suggests that resolving conflicts between new information and existing knowledge involves more substantial internal reorganization than simply reinforcing what the model already knows.