LLM Framework Automates Fuzzing Harness Generation at Scale

Generating test harnesses for program fuzzing—the automated testing technique that feeds random inputs to code to find bugs—has traditionally been labor-intensive. A new AI-powered framework aims to handle this at scale by combining language models with specialized tools to identify what a function needs in order to be tested effectively.

The Challenge#

Applying large language models to fuzzing harness creation breaks down when dealing with internal functions or arbitrary code across diverse projects. The core problem: LLMs need rich contextual details like API specifications, library dependencies, and real-world usage examples to write correct harnesses. Existing approaches rely on static or incomplete information, leading to harnesses that compile but don't actually exercise the target code properly. Worse, models can produce code that looks valid yet serves no practical purpose—a phenomenon researchers call gaming the validation metrics.

Introducing HarnessAgent#

Researchers at arXiv (cs.SE) developed HarnessAgent, an agentic framework that combines language models with a toolkit of specialized functions. The system makes three key contributions:

• Compilation error handling: A rule-based strategy to identify and reduce various compilation failures automatically.
• Robust code retrieval: A hybrid pool of tools that can reliably locate and extract the source code for functions, achieving over 90% retrieval success—30% better than existing systems.
• Enhanced validation: A detection pipeline that weeds out fake or non-functional definitions.

Results#

Testing on 243 functions from real-world open-source projects in C and C++, HarnessAgent achieved an 87% success rate for C targets and 81% for C++—roughly 20% better than prior state-of-the-art methods. In one-hour fuzzing runs, more than 75% of its harnesses measurably increased code coverage, outperforming baseline approaches by over 10%.

Why It Matters#

Automatically generating fuzzing harnesses reduces manual effort for security teams and enables broader testing across large codebases. The tool-augmented approach shows how adding domain-specific capabilities to LLM pipelines can overcome their inherent limitations in specialized technical tasks.