Researchers Use Sparse Autoencoders to Steer Large Language Models' Reasoning Strategies

Large reasoning models naturally develop human-like cognitive strategies—such as backtracking to reconsider earlier steps or cross-checking their logic—to tackle difficult problems. Yet these models currently make their own choices about which strategies to use, often leading to inefficient or mistaken reasoning paths. A new research effort addresses this limitation by providing a way to deliberately control how models reason.

The Challenge of Hidden Reasoning#

The difficulty lies in the hidden layers of these language models, where reasoning strategies become tangled together in ways that are hard to disentangle. Existing techniques struggle to isolate and manage individual reasoning approaches because the model's internal representations mix multiple concepts at once.

SAE-Steering: A Two-Stage Solution#

Researchers have developed SAE-Steering, a method that leverages sparse autoencoders (SAEs) to break apart these entangled hidden states into cleaner, more manageable features. The approach works in two phases:

• Feature filtering: The system scans for features that amplify keyword signals linked to specific reasoning strategies, discarding over 99% of unrelated features in this initial sweep.
• Effectiveness ranking: The remaining candidate features are ranked based on how well they actually control the model's behavior.

Stronger Control and Better Accuracy#

Tests show that SAE-Steering exceeds existing methods by more than 15% in control effectiveness. When researchers deliberately steered models away from erroneous reasoning paths toward correct ones, the systems achieved a 7% absolute improvement in accuracy on complex tasks. This demonstrates that guided reasoning can prevent models from getting stuck in logical dead ends.