Agentic AI
AI Coding Agents Burn Through 1,000x More Tokens Than Simple Tasks—and Prediction Fails
A comprehensive study reveals that AI agents tackling coding problems consume vastly more tokens than expected, vary wildly in efficiency, and struggle to forecast their own computational costs.
Researchers have conducted the first systematic examination of how much AI agents actually spend in token consumption when solving complex coding tasks—and the findings expose surprising inefficiencies and unpredictability in agent behavior.
The Staggering Cost of Agentic Work#
AI agents deployed on coding challenges consume approximately 1,000 times more tokens than traditional code reasoning or chat-based code work, according to the analysis across eight leading language models on the SWE-bench Verified benchmark. Crucially, input tokens—not output tokens—are the primary driver of this massive expenditure, suggesting agents cycle repeatedly through context and previous attempts rather than generating lengthy responses.
Wild Variability and No Clear Payoff#
Token consumption proves far more chaotic than developers might expect. Identical tasks can see runs differ by as much as 30x in total token use, revealing deeply stochastic behavior across attempts. More troubling: higher token spending does not guarantee better results. Instead, accuracy tends to peak at intermediate cost levels and then stagnate or decline as token use climbs further—a pattern that undermines the intuition that "throwing more computation at a problem" yields better solutions.
Models Split on Efficiency#
Token efficiency varies dramatically across frontier models. On the same coding tasks, Kimi-K2 and Claude-Sonnet-4.5 consume over 1.5 million more tokens on average than GPT-5, pointing to fundamental differences in how different models approach agentic problem-solving.
Human Intuition Misses the Mark#
Expert human raters assessing task difficulty show only weak alignment with actual computational costs—a finding that underscores a gap between what humans perceive as complex and what actually demands computational effort from agents. Simple-looking tasks may trigger expensive agentic loops, while harder-seeming work might resolve more efficiently.
