New AI Agent Doubles Code Optimization Success by Using Profiler Feedback

Large language models have become adept at fixing bugs and adding features to production codebases, but optimizing code for speed has remained a stubborn challenge. A new system called PerfAgent tackles this gap by pairing an LLM agent with profiler data and automated verification, achieving roughly double the success rate of existing approaches.

The Problem with Code Optimization#

Traditional LLM agents excel when the goal is correctness—they can read test suites and iterate until code passes. Optimization is different. A patch must pass all existing tests, actually speed up the program, and match the improvements a human expert would make. Current systems often stumble because they miss performance bottlenecks hidden behind abstractions or native code, settle for modest gains after the first quick win, or rely solely on wall-clock timing without deeper insight into where the slowness actually lives.

How PerfAgent Works#

PerfAgent wraps an off-the-shelf coding agent in a feedback loop powered by profiler output. Rather than asking the model to guess which code is slow, the system runs a profiler, surfaces the actual hotspots, and feeds that evidence back into the agent's reasoning loop. This cycle repeats: optimize, test, profile, refine. A verifier ensures patches don't break edge cases or change program behavior.

Benchmark Results#

On two challenging optimization datasets—GSO and SWE-fficiency-Lite—PerfAgent more than doubled the number of patches that matched expert-level speedups. For GSO, the success rate jumped from 19.6% to 39.2%. On SWE-fficiency-Lite, it soared from 26% to 74%. Importantly, the gains came from better feedback rather than brute-force sampling; PerfAgent surpassed an oracle baseline that simply tested multiple candidate patches five times over, and did so at substantially lower computational cost.

Why It Matters#

Code performance optimization at scale is a real industry need, but it's been one of the least automated aspects of software engineering. This work suggests that coupling LLM agents with traditional developer tools—profilers and test suites—unlocks a new frontier. The profiler-guided approach sidesteps the agent's tendency to hallucinate which code needs fixing and instead grounds reasoning in empirical data, much like how a human engineer would work.