New control-theory method makes language models steer away from harmful outputs without slowing them down

A team of researchers has introduced a new approach to steering language model behavior that operates efficiently and without sacrificing generation speed. The method, called Linear Semantic Control (LiSeCo), uses control-theory principles to guide the internal activations of large language models (LLMs) away from regions that produce undesirable outputs.

How it works#

The core insight is that natural language generation can be modeled as movement through a semantic space defined by a language model's hidden layers. Rather than trying to steer an LLM toward desired meanings, LiSeCo takes a different approach: it uses classical control-theory techniques to keep activations confined to a safe region of embedding space that corresponds to acceptable semantics.

The intervention happens online, adjusting the activations of each token as it's being generated. Crucially, the control signal is computed in closed form—meaning it's calculated exactly rather than through iterative optimization—so it adds minimal computational overhead.

Key advantages#

• Gradient-free: No need to backpropagate through the model, making it lightweight and fast
• Performance guarantees: Uses control theory to ensure activations stay within defined semantic boundaries
• Context-aware: Adjusts steering dynamically based on the current generation state, rather than applying a one-size-fits-all constraint
• Minimal latency impact: The closed-form solution avoids expensive iterative procedures

What it controls#

The researchers tested LiSeCo on three distinct tasks: reducing toxic language, steering sentiment in generated text, and controlling which language an output is produced in (English vs. Spanish). In all cases, the method maintained text quality while successfully suppressing unwanted attributes.

This work addresses a growing need in the field: as LLMs become more common in high-stakes applications, the ability to reliably restrict certain outputs—without degrading model performance or adding computational burden—becomes increasingly important. Read the full paper on arXiv.