AI
New Framework Helps AI Agents Stay on Track While Fixing Code
LivePlan monitors and redirects programming agents in real time, improving their ability to resolve GitHub issues without costly LLM interventions.
Automated agents designed to fix software bugs often stumble when tasks require multiple code changes or incomplete problem descriptions. They can wander from their initial strategy, retry failed approaches, or give up before reaching a solution. Researchers have developed LivePlan, a monitoring system that tracks these agents and steers them back on course without breaking the bank.
The Challenge#
Resolving GitHub issues at scale presents a particularly tough test for autonomous programming agents. When fixes span multiple locations in a codebase or when issue descriptions lack critical context, agents can follow inefficient paths. The result: long sequences of actions riddled with wasted effort, redundant attempts, and premature exits.
How LivePlan Works#
Rather than constantly consulting large language models to rethink a strategy, LivePlan takes a two-part approach. A rule-based monitor watches for warning signs in the agent's trajectory—things like repetition, drift from the goal, or stalled progress—without invoking expensive LLM calls. Only when such a problem appears does the system ask an advisor LLM for a corrective nudge. This design avoids the pitfall of premature re-planning that can send agents further astray.
Results and Impact#
Built atop SWE-agent, LivePlan was tested across five different language models (three executing the fix attempts, two serving as advisors) using two large benchmarks of real GitHub issues. Compared to the baseline system, LivePlan pushed the success rate up by as much as 15.2%, with an average improvement of 9.9%. The cost premium: roughly $0.08 per problem solved. Notably, the gains concentrated on harder instances—problems that the original system either failed outright or solved inefficiently.
The framework demonstrated strong performance across the board: it consistently outperformed alternatives in resolution rates while rarely backtracking on cases the agent could already handle, and it cracked several problems that no competing approach solved.
