AI
New Method Improves AI-Generated Bug Tests by Smarter Code Retrieval
A correlation-aware retrieval system helps language models generate more effective bug reproduction tests by fetching the right code context and learning from past attempts.
Researchers have developed iCoRe, a new approach to help large language models automatically generate tests that reproduce software bugs—a tedious but essential maintenance task. The key innovation is a smarter way of finding and feeding the relevant code to the AI, rather than relying on generic document search.
The Retrieval Problem#
Most current LLM-based systems generate bug reproduction tests by asking the model to search a codebase or pick from a list of provided files. However, these retrieval strategies have significant blind spots. They treat all code the same way, apply only semantic matching without understanding how functions call each other, and don't learn from whether the generated tests actually work.
As a result, the models often get stuck with irrelevant code context, making it harder to produce accurate tests.
How iCoRe Works#
The new system addresses three critical gaps:
- Differentiated retrieval: Source code and test cases have different characteristics and need separate retrieval strategies.
- Structural awareness: The system considers not just what code means but how functions call each other—improving relevance.
- Feedback loops: When a generated test fails or passes, the system uses that signal to refine and re-retrieve better context for the next attempt.
This iterative, feedback-driven approach lets the retriever learn and improve as it goes.
Results#
When integrated with an LLM-based test generator and evaluated on two benchmarks—SWT-bench Lite and TDD-bench Verified—iCoRe achieved a Fail-to-Pass rate of 42.0% and 52.8% respectively. Those numbers represent improvements of 19.7% to 31.7% over existing retrieval methods.
The work appears on arXiv (cs.SE, submitted April 2026, revised June 2026).
