EffiPair: Using Performance Comparison to Guide LLM Code Optimization

Researchers have unveiled a novel approach to making code generated by large language models faster and more efficient. Rather than examining each piece of code in isolation, the new method compares structurally similar implementations that differ in performance to guide refinements—a strategy that yielded top results across multiple benchmark sets.

The Optimization Challenge#

When large language models produce working code, the implementations often vary dramatically in how fast they execute. Current optimization strategies typically rely on single-point feedback—measuring how slow an implementation runs or pinpointing where the bottleneck occurs. However, this kind of information tells engineers what is costly but offers little direction on how to restructure the solution.

How Differential Execution Feedback Works#

The researchers developed EffiPair, a training-free framework that operates at test time by identifying pairs of programs that occupy similar regions of structural program space yet exhibit measurably different efficiency profiles. By analyzing both the execution traces and the implementation differences between these paired solutions, the system distills key optimization insights into compact feedback signals.

The method then uses these relational clues to iteratively refine a pool of candidate implementations, steering them toward better performance without requiring additional model training.

Results Across Benchmarks#

Testing across three established evaluation suites—EvalPerf, Mercury, and ENAMEL—with multiple models including GPT-4o mini, DeepSeek-V4.1 Flash, and GPT-5 mini, EffiPair achieved the best efficiency results in all nine model-benchmark combinations under equivalent evaluation conditions. Two rounds of contrastive refinement consistently improved efficiency relative to the initial selected draft while maintaining or enhancing the Pass@1 correctness metric across every setting.

Why It Matters#

The results underscore that comparing execution behavior between related candidate solutions can serve as an effective, lightweight signal for optimization without the overhead of training new models. This lightweight approach to test-time refinement demonstrates a practical pathway for making AI-generated code both functionally correct and computationally efficient.