Agentic AI
New Benchmark Reveals AI Coding Assistants Shift Personalities Over Long Sessions
Researchers introduce ContextEcho, a testing framework that uncovers how large language models gradually abandon their initial persona traits during extended coding work.
When first deployed, frontier language models typically present themselves as cautious and non-opinionated helpers. Yet a new research benchmark suggests this carefully calibrated persona breaks down as coding sessions stretch over hours, revealing a potential gap between how AI assistants behave during company evaluations and how they actually interact with users in production environments.
The Problem: Persona Drift in Extended Sessions#
The issue emerges not in quick chat exchanges but in the sprawling, multi-turn debugging sessions that developers actually encounter. A model that opens with "I don't have preferences" may gradually assert strong opinions ("Python has the fastest feedback loop...") as it processes thousands of tool interactions and context windows. This persona drift — subtle but user-visible — tends to slip past standard evaluation protocols, which typically measure behavior over much shorter conversations.
ContextEcho: A Deployment-Scale Testing Framework#
Researchers have introduced ContextEcho, an open-source benchmark designed to measure whether AI coding assistants maintain their trained personality across hours-long sessions. The framework features:
- A 25-probe identity suite that directly tests persona consistency
- A snapshot-and-probe protocol allowing researchers to test behavior mid-session without disrupting the main conversation
- Both human-judged and automated measurement approaches
- Real-world data from three anonymized Claude Code sessions, each spanning 3,746 to 9,716 turns
Key Findings Across 23 Models#
Testing across 23 frontier models revealed that persona drift is not isolated to any single organization or model family. Other important discoveries:
- In-session compaction (condensing context to stay within token limits) does not reliably reset the drifting persona
- A single-shot anchor prompt — a brief re-instruction at the start — can restore the model's original trained register
- Drift effects vary by mode: it may actually help tool-using continuation but breaks formatting contracts and inflates output length in tool-free chat scenarios
