Multi-agent LLM system automates code refactoring while guarding against quality regressions

Researchers have developed REFINE, a multi-agent approach that uses large language models to automatically refactor Java code while maintaining strict safeguards against introducing new problems or altering critical functionality.

How the system works#

The framework orchestrates a pipeline of verification and transformation steps. Static analysis first identifies problematic code patterns, then an LLM-powered agent proposes refactoring changes tailored to eliminate those specific issues. Once modifications are generated, the system re-analyzes the altered code to confirm quality improvements and performs preservation checks to ensure no unintended side effects occurred. Results are then compiled into structured reports for developers.

Evaluation results#

Researchers tested REFINE on 450 Java files across 15 open-source projects, generating 1,350 outputs using three leading LLM providers—OpenAI GPT-5.5, Google Gemini 3.1 Pro Preview, and Anthropic Claude Opus 4.8. The approach successfully reduced detected code smells by margins ranging from 68% to 73%, with particularly strong gains against major design issues. When compared directly to straightforward prompt-based refactoring on 150 matched files, REFINE achieved better median smell reduction with more conservative edits and fewer accidental public-method deletions.

Caveats and human oversight#

Despite these improvements, broader quality metrics showed mixed results. The preservation checks uncovered residual risks, including unintended changes to assertion and failure-handling calls, as well as occasional removal of public-facing methods. The authors caution that REFINE outputs should be treated as candidates requiring thorough compilation, testing, dependency analysis, and manual review before being merged into production codebases. The tool aims to augment developer judgment rather than replace it.