New method improves AI steering by better identifying and calibrating latent feature controls

A research team has introduced an improved approach to steering language model behavior at inference time by more reliably identifying and adjusting the internal features that control model outputs. The work, published on arXiv, tackles two fundamental problems that prevent existing steering techniques from working reliably.

The Core Problem#

Sparse autoencoders (SAEs) can theoretically allow us to steer LLM outputs by manipulating specific internal feature activations—essentially turning certain topics or behaviors up or down. However, current methods frequently fail in practice: they either miss the features needed for steering, or they adjust those features by the wrong amount, leading to off-target or incoherent responses.

How ContrastiveSteer Works#

The proposed ContrastiveSteer method addresses both failure modes through two complementary mechanisms:

• Feature identification: Features are ranked by measuring how much more strongly they activate when processing text from a target domain compared to general text. This contrast-based scoring makes it easier to spot features that genuinely matter for the desired behavior.
• Calibration: Rather than guessing steering strength, the method uses a model-specific calibration procedure to set how much to push each feature—recovering at least 93% of optimal performance across different model families.

The researchers also introduce a diagnostic called contamination, which detects when post-steering activations emphasize weakly aligned features. This metric helps distinguish successful steering from under-steering and complete collapse.

Results and Impact#

Testing across multiple LLM families and SAE configurations, ContrastiveSteer achieved:

• Up to 3.9× improvement in topic alignment over unsteered baselines and competing steering methods
• Target-domain classification accuracy reaching 93% in some cases
• Consistent performance across different model architectures

The activation-scale normalization approach proved robust, maintaining at least 93% of optimal domain-score gains regardless of which model family was being tested.