Selective Gradients Outperform Full Projections in Explainable AI for Large Language Models

Training data influence methods help researchers understand which examples shaped a language model's behavior, but applying them at scale is computationally prohibitive due to the staggering size of model gradients. A new study addresses this bottleneck by comparing two fundamentally different strategies for making influence estimation tractable.

The Dimensionality Problem#

Gradient-based explanation techniques for large language models face a practical barrier: the sheer number of parameters. Rather than work with full gradients, researchers typically restrict their analysis to some subset of parameters, though these choices have historically been made on intuition rather than rigorous justification. This paper seeks to establish a principled framework for deciding which parameters matter most.

Selection vs. Projection#

The researchers compared two approaches. The first uses greedy selection to identify a small but architecturally meaningful subset of model components. The second employs random projection, a common technique for compressing high-dimensional data into lower-dimensional spaces. Using a custom benchmark designed to test influence estimation, the team evaluated how well each method preserved the signal needed to trace training data effects.

Key Finding#

The greedy selection of specific components proved superior. Not only did targeted component selection capture influence information more effectively than either full gradients or randomized projections, it also required less computation than the projection-based approach. This demonstrates that intelligent subset selection—informed by model architecture rather than arbitrary—offers a practical way to make instance-based explanations computationally feasible without sacrificing performance.

The work suggests that explainability methods for large models need not sacrifice accuracy for efficiency when parameter choices are made strategically.