AI
TomasuLLM speeds up AI coding agents by running tool commands out of order
A new runtime system lets LLM agents predict and run future tasks early while waiting for slow tools, delivering 27–35% speedups without sacrificing correctness.
Long-running tools such as compilers, test suites, and repository queries create bottlenecks for AI coding agents—they can take seconds or minutes to complete, leaving the agent idle in the meantime. Researchers have now borrowed a strategy from chip design to address this problem.
The Problem: Idle Time During Tool Execution#
Coding agents often spend much of their runtime waiting for external tools to finish. A compiler might take 30 seconds, a test suite might run for several minutes, and the agent cannot move forward until these operations complete. This sequential constraint mirrors a challenge that computer processors faced decades ago: the tension between a simple, ordered interface and the potential for hidden parallelizable work.
Out-of-Order Execution for Agent Actions#
A new system called TomasuLLM applies out-of-order execution—a processor optimization technique—to LLM agents. Rather than waiting for each tool call to finish before predicting the next step, TomasuLLM drafts future actions speculatively. It runs these predicted tasks in isolated, copy-on-write sandboxes, tracks their dependencies and side effects, and only commits results once all earlier steps have been validated.
This approach maintains task correctness by ensuring that results become visible only after every preceding action in the logical sequence has been confirmed against the agent's actual state.
Measured Performance Gains#
Testing across three distinct benchmarks showed consistent improvements:
- SWE-bench Verified: 1.31× speedup across 100 tasks
- Terminal-Bench 2.0: 1.35× speedup across 28 tasks
- SWE-Marathon: 1.27× matched progress across 18 sessions
Gains scale with tool latency—the longer the tool calls, the more speculative work can run in parallel. Across over 4,000 audited validation records, the system produced zero false positives, confirming that correctness is preserved.
Why It Matters#
This research bridges two fields: processor design and AI agent optimization. By eliminating idle time during slow tool execution, TomasuLLM makes coding agents more practical for real-world software engineering tasks where compilation, testing, and querying remain unavoidable bottlenecks.
