AI
LLM Learns to Self-Correct Binary Code Through Iterative Refinement
A new reinforcement-learning approach trains AI to reverse-engineer binary code by iteratively improving results based on compilation and execution feedback.
Converting compiled binary code back to readable source—a process called decompilation—remains a crucial step in security research, malware analysis, and understanding legacy software. While recent language models have begun tackling this task, they typically generate output once and stop, leaving reconstructed code that may look correct but fail to replicate the original program's actual behavior.
The Refinement Loop#
Researchers have introduced AutoDecompiler, a specialized LLM trained via reinforcement learning to treat decompilation as an iterative process rather than a single-shot problem. The system takes a different approach: after generating initial code from assembly input, it reviews compiler error messages, runs the reconstructed program against test cases, and compares its output behavior to the original binary. Using this feedback, the model revises its code across multiple turns, gradually closing the gap between what it produced and what the original program actually does.
Training for Correctness#
The model is trained with custom reward signals that weigh four key objectives: whether the code compiles without errors, whether it produces identical output to the original binary when executed, and whether it maintains semantic fidelity to the source program. The team also designed feedback mechanisms that point the model toward specific failure modes—compiler errors, runtime crashes, or failed test cases—while penalizing revisions that backslide into previously solved problems.
Results Across Scales#
Testing AutoDecompiler across different model sizes and benchmark datasets shows consistent gains over single-turn alternatives. The multi-turn version achieves notably better behavioral re-executability, meaning the recovered code behaves like the original more often. This suggests that teaching models to learn from program feedback through reinforcement learning is a promising direction for improving the functional accuracy of AI-assisted reverse engineering.
