Agentic AI
Do AI agents actually follow their instructions? New study reveals widespread plan abandonment
Researchers find that large language model agents frequently ignore the task plans they're given, instead reverting to learned patterns that may harm their performance.
A new study from arXiv challenges a fundamental assumption about how AI agents work: that they reliably execute the plans they're given. Researchers analyzing over 21,000 programming agent trajectories found that when left without explicit guidance, agents fall back on ingrained behavioral patterns from training that are often incomplete or poorly generalized.
The problem with invisible workflows#
When agents operate without a formal plan, they default to internal problem-solving strategies learned during training. These implicit workflows frequently fail—they may overfit to specific benchmark scenarios, show inconsistent application across contexts, or simply lack the flexibility needed for novel tasks. This raises a critical question for AI safety and capability assessment: how can researchers distinguish between an agent that genuinely solved a problem through sound reasoning versus one that merely got lucky through data contamination or benchmark-specific pattern matching?
How plans actually help#
The team tested SWE-agent across four different large language models on two software engineering benchmarks, evaluating performance under eight different plan configurations. Their key findings suggest that providing an explicit plan meaningfully improves issue resolution rates. Periodic reminders to follow the plan during execution further mitigated violations and lifted overall success rates. However, the relationship between plans and performance proved more nuanced than expected: a poorly designed plan actually harmed results more than having no plan at all.
The surprising cost of adding steps#
Counterintuitively, inserting additional task-relevant phases early in a plan sometimes degraded performance—particularly when those new steps conflicted with the model's internal problem-solving tendencies. This suggests that agents don't flexibly integrate new instructions; instead, they struggle when external guidance clashes with learned behaviors.
Implications for AI training#
The authors argue that current approaches embed task-specific workflows directly into model weights during training. They advocate for a shift toward training paradigms that teach agents to genuinely follow instructed plans, enabling them to reason and adapt based on new guidance rather than rigidly replaying memorized routines. Such a change could make AI systems more robust, transparent, and trustworthy in real-world deployment.
