Agentic AI
How AI Coding Agents' Tools Are Designed Shapes What They Can Achieve
Research shows that the way tools are organized matters as much as what they can do for language model agents tackling code fixes.
Researchers testing large language model agents on software tasks have uncovered a critical insight: the way tools are structured and presented to an agent can be just as influential as the underlying capabilities those tools provide. Even when agents have access to the same information and actions, reorganizing how those features are exposed dramatically changes agent behavior.
The Study's Scope#
A team ran controlled experiments using multiple coding agents to fix repository-level issues. They tested six different tool architectures—holding capabilities constant while varying organization and presentation—across three different models and nearly 12,000 execution trajectories. This methodical approach isolated the effect of tool design from other variables.
What Tool Architecture Actually Changes#
The findings reveal that tool design shapes agent performance in measurable ways:
- Structured interfaces outpace basics: Compared to a simple setup where agents only had access to bash commands, more organized lower-level interfaces boosted consistency across repeated attempts by as much as 4.7 times.
- Natural-language search expands reach: Adding natural-language-powered search capabilities let agents explore repositories more thoroughly and locate relevant files more often—improving file discovery by over 11%.
- Python-style execution is efficient: Adopting Python CodeAct-style interfaces delivered comparable results while cutting the number of steps agents needed by 41.6% and reducing token consumption by 56.3%.
What Doesn't Move the Needle#
Lighter tools designed to help agents "think through" problems—such as features that record intermediate reasoning or provide cognitive scaffolding—showed surprisingly little impact on how agents actually behaved or performed.
Why This Matters#
The research underscores that building better agentic systems isn't only about adding new capabilities; it's also about thoughtfully architecting how those capabilities are accessed. Tool design is emerging as a critical but often-overlooked lever for improving agent effectiveness and efficiency at scale.
