AI
How Linear Relations Drive Language Model Hallucinations
Researchers find that the way AI models represent relational knowledge makes them prone to inventing facts about unfamiliar subjects, especially when relationships follow linear patterns.
Language models frequently fabricate plausible-sounding answers to questions about entities they don't actually know. A new study from arXiv suggests that the mathematical structure underlying how these models learn relationships may be the culprit.
The Core Problem#
When asked about an unfamiliar person or thing—say, "Which instrument did Glenn Gould play?" but about a made-up musician—large language models like Gemma-7B-IT often confidently invent an answer rather than admitting ignorance. This happens despite the false premise in the question. The researchers behind this work hypothesize that the problem stems from how neural language models encode relationships internally.
Linear vs. Nonlinear Relations#
The key insight centers on relational linearity. The researchers propose that language models rely on abstract schemes to represent relationships between entities—particularly linear ones, where one object maps predictably from another (e.g., "musician → instrument"). When a model encounters an unknown subject within a linear relationship, its internal mechanisms can easily generate a plausible object as if the relationship were real.
For nonlinear relations, however, this automatic generation mechanism doesn't work as smoothly, making false assertions harder to produce. The model is more likely to recognize it lacks the necessary knowledge.
Testing the Theory#
To investigate, the team created SynthHal, a synthetic benchmark containing 15 different relations, each populated with unknown entities. They tested four instruction-tuned models against this benchmark and found strong correlations between relational linearity and hallucination tendency—ranging from 0.58 to 0.84 across models.
What This Means#
While these correlations don't prove the proposed mechanism is the direct cause, they point researchers toward a promising direction for understanding why language models hallucinate. Recognizing that linear relations make hallucinations more likely could eventually help developers design better safeguards against confident false answers.
