New Training Method Boosts Sparse Autoencoders for Instruction-Tuned Language Models

Sparse Autoencoders (SAEs) have emerged as a valuable tool for understanding how large language models work internally. However, training these interpretability tools on instruction-tuned models has proven challenging, particularly when using conventional block-based training methods adapted from general language model pre-training.

The Problem with Block Training#

The standard block training paradigm, borrowed from LLM pre-training workflows, introduces a specific problem when applied to instruction-tuned models. When training data is processed in isolated blocks, attention mechanisms can leak information from unrelated contexts, creating a form of destructive noise that degrades the quality of the trained SAEs. This interference undermines both the model's ability to reconstruct activations and the interpretability of learned features.

Introducing FAST#

To address this limitation, researchers have developed Finetuning-aligned Sequential Training (FAST), a training methodology tailored specifically for instruction-tuned models. Rather than breaking data into disconnected blocks, FAST maintains the continuity of sequences during training, allowing SAEs to learn patterns that align with how instruction models actually process information.

Performance Improvements#

The approach delivers substantial gains across multiple metrics:

• Reconstruction quality: FAST achieves a log-scaled mean squared error of 0.6468, dramatically outperforming the baseline's 5.1985
• Feature quality: On Llama-3.2-3B-it, FAST generates 21.1% high-quality interpretable features, compared to 7.0%–10.2% for existing methods
• Stability: The method maintains nearly zero performance loss (-0.51% to 0.37%), indicating training robustness

New Capabilities#

Beyond improved reconstruction, the work demonstrates that intervening on special token activations through trained SAEs can enhance generation quality. This finding opens pathways for more granular control over model behavior, potentially enabling practitioners to steer instruction-following models at the level of individual learned features.