Agentic AI
Memory architecture helps AI code agents retain only essential context across sessions
A new selective memory system lets AI agents preserve reusable task and data information while discarding session-specific reasoning, improving performance at lower token costs.
Code-generating AI agents face a persistent efficiency problem: each conversation session wipes the slate clean, losing valuable domain knowledge and configuration details that made prior interactions successful. Researchers have developed a selective memory architecture that preserves the right categories of context while pruning session noise.
The core challenge#
When agentic language models operate across multiple turns—calling tools, generating code, and iterating—they typically start each new session without institutional knowledge. Task specifications, data schemas, tool configurations, and output formatting rules must be reestablished from scratch, consuming tokens and reducing consistency. The alternative—keeping full conversation history—rapidly balloons context windows and costs without proportional gains.
Selective retention approach#
The proposed system distinguishes between context worth keeping and reasoning traces worth discarding. It retains four categories across sessions: task specifications, data schemas, tool configurations, and output constraints. These are packaged into reusable workspaces that can transfer between users under role-based access controls. Session-specific reasoning and intermediate steps are discarded.
Testing results#
On a controlled experiment using four public datasets, the performance gap was striking:
- No memory: 0 of 12 trials completed at 3.8K tokens
- Selective memory: 12 of 12 trials at 3.9K tokens
- Full history: 8 of 12 trials at 7.7K tokens
The selective approach nearly matched no-memory cost while dramatically improving outcomes—demonstrating that what gets retained matters far more than how much. Both improvements held up to statistical scrutiny (Bonferroni-corrected McNemar tests, p = 0.0005 and p = 0.008).
Real-world implementation#
The team implemented the system in a deployed platform where agents generate git-versioned code artifacts from CSV, SQL, REST, and MCP data sources. A complementary data refresh mechanism decouples generated programs from runtime data without requiring model calls, succeeding in all 12 test cases at a median 0.08 seconds per refresh. Using summaries instead of raw data injection reduced token costs by 97–431×.
