Agentic AI
LLM Agents That Build Their Own Tools Cut Production Latency in Half
Researchers show that when AI agents compile their repeated procedural steps into reusable tools before deployment, they can dramatically speed up real-world systems while improving reliability.
In production environments, large language model agents typically regenerate code for the same procedural tasks on every request, creating unnecessary latency and error variation. A team working on fulfillment center alarm triage has proposed a different approach: let agents build and maintain their own tool libraries before deployment.
The Tool-Making Pipeline#
Rather than continuously regenerating code at inference time, the researchers developed an agentic tool-making system that runs before production deployment. The pipeline collects execution traces from the live environment, observes backend schemas and actual values, generates candidate tools, and validates them against labeled test cases. This upfront work transforms repeated standard operating procedure (SOP) steps into versioned, validated tools that agents can call directly during operation.
Real-World Results#
The team deployed the approach in a fulfillment center alarm-triage system where an agent had to diagnose problems against a 44-node SOP while querying data from heterogeneous metric backends. The results were substantial:
- Latency improvements: Tool calls reduced median latency by 42%, with an additional 62% latency reduction when the system switched to a simpler direct-call architecture that leverages the tools' structured output format.
- Error reduction: On 1,500 historical alarms, using tools cut end-to-end error rates by up to 53%, primarily by eliminating run-to-run inconsistencies in repeated procedural steps.
- Operational benefits: Versioned tools made systems more auditable and surfaced specification gaps and data drift in upstream systems.
Why This Matters#
The approach represents a shift toward self-evolving agents—systems that don't just perform tasks but actively improve their own capabilities by building and maintaining tool libraries. By replacing inference-time code generation with pre-compiled tools, agents can focus on actual decision-making rather than constantly reinventing solutions. The structured outputs from tools also simplify the overall system architecture, making it faster and easier to operate at scale.
