New RL-Based Memory Agent Pushes LLMs to Handle 3.5M Tokens With Minimal Performance Drop

A team of researchers has unveiled MemAgent, an agent-based approach designed to dramatically extend the long-context capabilities of large language models beyond their training scope. The work addresses a fundamental challenge in handling document processing: scaling to extremely long texts without the performance degradation typically seen when models exceed their training context windows.

The Challenge#

Despite recent advances in length extrapolation, efficient attention mechanisms, and specialized memory modules, language models still struggle when asked to process documents far longer than they were trained to handle. The core problem remains difficult: how to maintain accuracy when working with contexts that are orders of magnitude longer than the model's original training window.

MemAgent's Approach#

Rather than trying to patch existing model architecture, the researchers took a different route—they reformulated the problem through an agent workflow that fundamentally changes how the model processes long documents. The system reads text in manageable segments and maintains a dynamic memory that gets updated using an overwrite strategy. This segmented reading pattern, combined with carefully managed state, allows the model to handle vastly extended contexts.

To make training feasible, the team extended the DAPO algorithm to support independent-context multi-conversation generation, enabling efficient reinforcement learning optimization across long sequences without prohibitive computational costs.

Impressive Results#

The performance metrics are striking. When trained on just 32K tokens of context, MemAgent can extrapolate to handle a 3.5 million token question-answering task with less than 5% performance loss. In standardized benchmarks like the 512K RULER test, the system achieves over 95% accuracy.

Why It Matters#

These results suggest a path forward for practical long-document processing. Instead of building larger models with bigger context windows—which brings exponential computational costs—this approach enables existing models to operate effectively on texts orders of magnitude longer than their training data, opening possibilities for handling entire books, codebases, or lengthy research corpora in single context windows.