New Framework Reveals How RAG Systems Fail to Align Retriever and Generator Components

Retrieval-Augmented Generation (RAG) systems promise to make large language models more reliable by grounding their responses in external documents. Yet the critical handoff between the retriever component—which finds relevant documents—and the generator component—which produces the final answer—remains largely invisible. A new study from arXiv reveals that this opacity masks a significant problem: generators frequently ignore top-ranked documents and instead rely on materials the retriever ranked as less relevant.

The Misalignment Problem#

The interaction between a RAG system's two main pieces has been hard to measure and audit. Researchers at the arXiv paper present RAG-E, an end-to-end explainability framework designed to quantify this gap. Their experiments on PopQA, QAMPARI, and TREC CAST datasets show substantial misalignment across different models and settings, suggesting that the generator's behavior and the retriever's ranking often work at cross-purposes.

How the Framework Works#

RAG-E combines three technical innovations to shed light on this hidden mismatch:

• Adapted Integrated Gradients for analyzing how the retriever influences document ranking
• Monte Carlo-stabilized Shapley Value approximation for attributing the generator's outputs to specific documents
• Weighted Attribution-Relevance Gap (WARG) metric, which directly measures how closely the generator's actual document usage matches the retriever's confidence rankings

The WARG metric outperforms simpler correlation measures like Pearson and Spearman correlations at capturing true retriever-generator alignment and can predict overall RAG system performance.

Implications for Deployment#

These findings have practical consequences for deploying RAG systems in high-stakes settings such as healthcare, law, or finance—domains where misaligned components could lead to incorrect or misleading outputs. By making the interaction transparent and measurable, RAG-E enables practitioners to audit their systems and identify when components are working at odds with each other. The framework offers a path toward building more reliable and trustworthy retrieval-augmented systems.