Overlooked baselines undermine model interpretation methods, researchers show

Researchers have identified a critical oversight in how scientists currently explain and interpret machine learning models: most existing techniques ignore the importance of establishing a proper baseline for comparison.

In a paper submitted to arXiv, the team demonstrates that this neglect of baselines can produce imprecise or entirely incorrect interpretations. They argue that the field needs to rethink how explanation methods are formulated and evaluated.

Unifying fragmented approaches#

The study brings together three previously separate families of interpretation techniques—gradient-based methods, Integrated Gradients (IG), and Taylor expansion—and shows how they relate to one another. For the first time, the researchers explicitly identify which baseline applies to each approach. This unified framework reveals fundamental relationships that have remained obscured and suggests pathways for improving performance across these methods.

Identifying flaws in existing techniques#

The authors scrutinize popular interpretation methods including Integrated Gradients, LayerCAM, ODAM, and Difference Map, uncovering specific errors and limitations in how each handles baseline assumptions. They argue that current evaluation practices are themselves flawed—relying on marginal-effect measures or assuming perfect model performance rather than directly measuring attribution accuracy.

A corrected approach#

The researchers propose a revised interpretation method with a transparent, theoretically justified baseline. Their approach supports feature-based interpretation at any layer of a neural network. Notably, they argue that interpretations derived from different layers are all valid; variations between them reflect differences in how much feature extraction has occurred at each stage of the network.

The work suggests that precision in model interpretation—measured as the gap between what the method claims and what it should claim—should become the standard for evaluation rather than proxy measures currently in use.