Self-Evolving Code Detectors Learn to Spot Flaws Without Hand-Written Rules

Automated metrics are crucial for training AI agents on complex tasks—particularly those where ground truth is elusive, like report generation—but designing reliable scorers remains one of AI research's stubborn challenges. A new approach sidesteps manual metric engineering by having the evaluator discover its own shortcomings.

The Core Insight#

Rather than asking a language model to generate evaluation rules directly, researchers at arXiv propose treating metric design as a refinement problem borrowed from formal program verification. The system maintains a pool of small Python operators, each flagged to detect a specific flaw or abstain from judgment. When these operators fail to distinguish between a correct answer and an incorrect one scored identically, that collision becomes the signal to create a new operator—not a vague prompt, but a concrete counterexample pair.

How EvalCEGAR Works#

The approach, called EvalCEGAR (Counterexample-Guided Abstraction Refinement), cycles through three phases:

• Search for collisions: Scan the current operator pool for cases where correct and incorrect answers receive identical scores
• Author from examples: Use the collision pair to request a new operator, grounded in real failure modes
• Expand the search space: If no suitable operator emerges, broaden what operators are allowed to read from the candidates

This data-driven loop avoids the combinatorial explosion of naive generation: direct prompting of an LLM produced 183 operators with only 96 distinct behaviors, clustered in a narrow region of possibility space.

Results on Code Benchmarks#

Testing on MBPP+ and HumanEval+—sandboxed benchmarks with hidden unit tests providing exact ground truth—the system evolved a 55-line operator that recovered 15.4% of the gap between flagging nothing and a perfect filter across 428 unseen tasks. This single learned operator matched the effect of a hand-written baseline on roughly one-third of its detections, while reducing false positives by a quarter.

In six of eight independent runs, the evolved operator generalized to held-out data. By contrast, combining all eight hand-written reference operators as a single filter actually degraded accuracy. An LLM judge using the same information tied the improvement but required one model call per candidate—a computational cost the learned operator avoids entirely.