Shell beats specialized tools for AI enterprise agents, study finds

A new empirical study compares how different interfaces allow AI agents to tackle enterprise work, and the findings challenge conventional wisdom about specialized tool design.

The experiment#

Researchers tested five distinct interface approaches for AI agents running realistic enterprise tasks: traditional typed tools, typed tools supplemented with bash access, bash alone, bash combined with agent-created custom tools, and programmatic tool calling—a restricted model that limits programs to execute only actions within a predefined typed catalog.

The experiments ran on two enterprise simulation benchmarks (TheAgentCompany and APEX-Agents) and deployed advanced language models including Opus-4.8 and GPT-5.5.

Bash pulls ahead#

Plain bash access came out the clear winner. It achieved improvements of 21.8–24.5 percentage points over typed tools on TheAgentCompany and 4.8–7.4 percentage points on APEX-Agents, all while consuming 19–72% fewer tokens overall.

Interestingly, adding typed tools or agent-synthesized custom tools on top of bash delivered no measurable performance boost. The researchers detected no statistically significant pooled score gains from these combinations.

When specialized tools make sense#

Programmatic tool calling consumed fewer tokens than direct typed function calls and delivered comparable performance on many tasks. However, it generally trailed bash in both quality and token efficiency.

The practical implication is clear: for environments where unrestricted execution can be safely isolated, bash alone is the optimal choice. For settings where security or compliance policies mandate a fixed, approved tool catalog, programmatic tool calling offers a workable middle ground.

The study is available at arXiv (cs.SE).