LLMs Beat Compiler Optimizers at Assembly Code Superoptimization

Large language models have proven capable of optimizing assembly code to outperform traditionally optimized programs, marking the first systematic investigation of LLMs applied to program superoptimization at scale.

Superoptimization—the process of transforming code into its fastest possible version while maintaining identical behavior—has long been dominated by compiler heuristics. A new study from arXiv explores whether LLMs can surpass these industry-standard baselines on real-world assembly programs.

Building a Benchmark for Assembly Optimization#

Prior research on LLM-based code optimization relied on tiny datasets, typically containing only 2–15 simple, loop-free programs. The researchers constructed SuperCoder's benchmark with 8,072 assembly programs, each averaging 130 lines of code—a scale orders of magnitude larger than previous efforts. This size enables meaningful evaluation of how LLMs handle genuine optimization challenges.

Testing 23 LLM Models#

When evaluated on this benchmark, Claude-opus-4 emerged as the strongest baseline among 23 models tested, achieving a 51.5% test-passing rate and delivering 1.43x average speedup over code optimized by gcc -O3, the standard aggressive optimization level.

Fine-Tuning with Reinforcement Learning#

To push performance further, the team fine-tuned models using reinforcement learning, with a reward function that balances both correctness and runtime speedup. Starting from Qwen2.5-Coder-7B-Instruct—which initially achieved 61.4% correctness and 1.10x speedup—they developed SuperCoder, which reached:

• 95.0% correctness rate
• 1.46x average speedup over gcc -O3
• Additional gains through Best-of-N sampling and iterative refinement strategies

What This Means for Code Optimization#

These results establish that LLMs can compete with and exceed traditional compiler optimizers on assembly programs, suggesting a new avenue for performance optimization beyond the heuristic-based approaches that have dominated the field. The work provides both a large-scale benchmark and proof-of-concept for future research in neural program optimization.