SpecMine: New Dataset Maps How AI Agents Turn Specifications into Code

A team of researchers has released SpecMine, a comprehensive dataset documenting spec-driven development (SDD)—an emerging workflow in which developers write structured specifications that AI coding agents then implement. The corpus provides the first large-scale empirical window into this practice as it unfolds across open-source repositories.

What Is Spec-Driven Development?#

In spec-driven development, a developer writes (or an AI tool drafts and a developer refines) a natural-language specification that serves as the blueprint for implementation. Rather than manually coding the solution, an AI agent reads the spec and generates the code. Since 2025, a wave of commercial and open tools—including GitHub Spec Kit, OpenSpec, and AWS Kiro—have emerged to automate and manage this workflow, but their artifacts have never been systematically studied at scale.

The SpecMine Dataset#

SpecMine captures SDD activity across public GitHub repositories through two separate censuses:

• Broad census: 470,795 spec files from 73,030 repositories, covering 17 named tools
• Kiro census: 98,574 files with Kiro's distinctive requirements/design/tasks structure, spanning 12,910 repositories

Each specification is enriched with full repository metadata, complete commit history, and parsed document structure.

Tracking Spec-to-Code Translation#

To understand how specifications transform into working code, the researchers examined pull requests that touched specs in repositories with at least ten stars across 11 different tools. They captured 5,992 such PRs from 581 repositories, including complete changesets. This approach made the most direct workflow—spec and code changing together in a single PR—directly observable.

Beyond PRs, the team indexed 2.4 million typed references linking specs to their corresponding code, including over 1.2 million direct links to code files, 863,000 to sibling documents, and connections to pull requests, references, branches, and issues.

Why It Matters#

SpecMine enables the research community to investigate, for the first time, empirical patterns in how software is being specified and developed in an era of AI-assisted coding. The dataset's rich annotation and traceability should illuminate questions about specification quality, how well AI agents follow specs, and what makes the practice effective.