Researchers Propose DFT-Based Method to Tame LLM Hallucinations Through Semantic Hierarchy Structures

Researchers have developed a novel perspective on why large language models hallucinate, framing the problem through measure theory and frequency analysis. They argue that hallucination stems from a fundamental structural mismatch: the embedding space in transformers cannot faithfully represent the true semantic meaning needed for reliable reasoning.

The Core Problem#

The authors reformulate how transformer attention mechanisms function by treating the embedding space as a conditional expectation over a mathematical structure (σ-algebra). The key insight is that this space fails to be isomorphic to actual semantic truth—in other words, the internal representations simply cannot capture complete logical consistency, making hallucination theoretically inevitable under the current architecture.

WavePhaseNet: A Frequency-Based Solution#

To address this limitation, the team proposes WavePhaseNet, which explicitly constructs what they call a Semantic Conceptual Hierarchy Structure (SCHS) using Discrete Fourier Transform (DFT) analysis. The method works by decomposing semantic information along the sequence dimension into distinct frequency bands:

• Low-frequency components capture global meaning and intent
• High-frequency components represent local syntax and linguistic expression

This staged separation permits precise manipulation of semantic information in transformed space, allowing the model to distinguish between core meaning and surface-level expression details.

Dimensionality Reduction and Efficiency#

The researchers analyzed GPT-4's embedding space—which contains 24,576 dimensions—and found it exhibits a 1/f power-law spectral structure consistent with natural language self-similarity and Zipf's law. Through cumulative energy analysis, they determined that approximately 3,000 dimensions represent a lower bound for preserving complete semantic representation. Reducing the space from 24,576 to 3,000 dimensions can maintain meaning and intent while enabling stricter reasoning and reducing hallucination tendencies.

Consistency Control Through Topology#

The reduced embedding space uses cohomological regularization—a mathematical technique from algebraic topology—to construct graph structures and define consistency metrics. Inconsistencies among local inferences are quantified as losses based on coboundary operators. Applying harmonic projection and Hodge theory enables the framework to use cohomology as a practical regularization principle, extracting representations that maximize global logical consistency.