How AI Explanations Reveal What Language Models Share with the Human Brain

Large language models process text in ways that map onto human brain activity during language understanding, but the reasons behind this alignment have remained opaque. A new study uses explainability techniques to decode exactly what drives this brain-AI connection.

Understanding the Alignment#

Researchers applied attribution methods—tools that measure how much each input word contributes to an LLM's next prediction—to examine fMRI scans from people listening to spoken narratives. The approach reveals that gradient-based attribution explanations robustly predict patterns in neural activity, adding explanatory power beyond simpler confounds like acoustic features or word timing.

When tested against standard internal model representations, these attribution-based explanations outperformed them in capturing early auditory brain regions, suggesting that how models explain their decisions tracks human processing more closely than raw activations alone.

Layer-by-Layer Brain Alignment#

By extending their analysis to individual layers using conductance—a technique for attributing importance across the network's depth—the team found striking patterns: early layers in the model show heightened sensitivity to word identity and align with sensory auditory areas, whereas deeper, final layers are dominated by positional information and exhibit broader cortical alignment across the brain.

This mapping suggests that different computational stages in language models may recapitulate the hierarchical organization observed in human language processing, moving from low-level acoustic and lexical features toward higher-level syntactic and structural relationships.

Why It Matters#

These findings demonstrate that explainable AI methods serve dual purposes: they illuminate what language models are doing internally while simultaneously revealing the neural mechanisms humans employ for language comprehension. By linking attribution explanations to brain data, the work bridges neuroscience and machine learning, offering a window into shared principles of language representation across biological and artificial systems.

The research opens pathways for using AI interpretability tools not just to debug models, but to understand cognition itself.