CodeEvo: AI agents collaborate to generate high-quality code training data

A new approach to synthesizing training data for code-generating AI models uses a pair of collaborating agents to overcome limitations in existing automated methods. Researchers behind CodeEvo argue that current data generation techniques rely on inflexible rules and often produce instruction-code pairs that lack grounding or meaningful complexity.

The two-agent architecture#

CodeEvo operates through a Coder agent that iteratively develops solutions and a Reviewer agent that steers the generation process. Rather than following rigid heuristics, the Reviewer creates a Schema—a structured framework for designing logic and complexity—that guides the interleaved creation of instructions and code. This design allows the system to move beyond simple template-based synthesis and produce more nuanced training examples.

Hybrid verification and quality control#

The framework reinforces data quality through a dual verification approach. Deterministic compiler feedback checks whether generated code runs without errors, while semantic evaluation assesses whether the code actually solves the intended problem. This combination of syntactic and semantic validation helps filter out examples that appear correct on the surface but fail under scrutiny.

Scale and performance gains#

The team constructed CodeEvo-100K, a dataset of 100,000 instruction-code pairs organized by difficulty levels. Models fine-tuned on this data demonstrated consistent improvements over established baselines in code generation benchmarks, suggesting that the data synthesis strategy produces material with genuine pedagogical value.

Key takeaway#

By automating data creation through agent interaction rather than static rules, CodeEvo addresses a bottleneck in training large language models for code. The work points toward a path for scaling high-quality code training data without relying entirely on expensive manual annotation. Code and data are available at this URL.