AI Safety
Researchers map hidden geometry in AI prompts to predict unsafe outputs
A new technique measures how densely packed information flows through language models, revealing that malicious prompts have a distinct geometric signature detectable before generation.
Researchers studying how large language models process information internally have discovered that prompts leave a measurable geometric fingerprint—and that fingerprint can flag harmful instructions with 90-95% accuracy before the model ever generates a response.
Mapping the invisible landscape#
The work treats transformer neural networks as physical systems where tokens (text fragments) exist in high-dimensional space. By measuring the intrinsic dimension—essentially how much of that space is actually occupied by meaningful information—the researchers found a clear pattern: information density peaks in early-to-middle layers, then drops as the model prepares its output.
This dimension also shifts predictably when text is disrupted. Shuffling token order sharply increases the dimension, suggesting the model struggles to find compact representations of malformed input.
From theory to safety#
As a practical application, the team trained a simple linear detector on the layer-by-layer dimension profiles of prompts. This probe learned to distinguish benign instructions from adversarial or malicious requests with accuracy between 90-95%, outperforming existing safety tools like Llama Guard and Shield Gemma—commercial guardrails deployed in production systems.
The geometry-based approach also proved complementary to other techniques. When compared against probes built from layer-wise entropy (derived via the Tuned Lens), the intrinsic dimension signal remained competitive and offered a more interpretable, compact representation of risk signals spread across the network.
Why it matters#
The findings bridge two worlds: mechanistic interpretability (understanding how models work internally) and practical safety (stopping harmful outputs). Rather than treating prompt safety as a black-box classification problem, this method reveals where and how a model's internal geometry responds to suspicious input—offering a window into LLM behavior that's both actionable and interpretable.
