AI
LLM-based refactoring makes automatically generated unit tests readable and maintainable
Researchers combine search-based test generation with language models to produce human-friendly code while preserving test validity and coverage.
Automated test generation tools like EvoSuite can create large numbers of valid, high-coverage unit tests quickly—but the resulting code is often difficult for developers to understand and maintain. A new research approach addresses this gap by using large language models as a refinement layer rather than standalone generators.
The Problem with Fully Automated Testing#
Search-based software testing (SBST) tools excel at generating compilable tests at scale, yet their output tends toward cryptic variable names, convoluted structure, and low readability. Attempts to have LLMs generate tests directly encounter brittleness, with compilation success rates between just 51–78%, making them unreliable for production use.
A Hybrid Solution: TestHumanizer#
Researchers introduced TestHumanizer, a system that pairs search-based test generation with controlled LLM refactoring. Rather than asking LLMs to write tests from scratch, the approach uses them as validation-gated editors: they take compilable SBST suites and rewrite them for clarity, naming, and structure while preserving correctness and coverage.
The study evaluated this method on 350 classes from two test datasets (Defects4J and SF110). EvoSuite generated 15 test suites per class, each then refactored under three different prompt configurations using OpenAI's GPT-4o and Mistral's large-2407 model, yielding 31,500 refactored suites.
Results and Trade-offs#
The hybrid approach achieved compilation rates of 88–98%, dramatically outperforming direct LLM generation while staying close to EvoSuite's perfect baseline. Coverage remained largely intact, typically dropping only 1–2 percentage points. Between 86–95% of refactorings met a "faithful-refactoring" standard, meaning they preserved behavior while improving readability.
Beyond metrics, refactored tests showed measurable improvements in code quality:
- Reduced control-flow and cognitive complexity
- Fewer structural "code smells"
- Improved predicted readability scores
The prompt strategy mattered: summary-based approaches offered the best stability, while longer, code-focused prompts proved more prone to hallucination-induced errors.
