Large Language Models Fall Short as Autonomous Requirements Checkers, Study Finds

A new empirical study challenges the assumption that large language models are ready to autonomously assess software requirements quality, finding that even top-performing LLMs miss the majority of defects that human experts catch.

The Study's Scope#

Researchers conducted the first systematic evaluation of off-the-shelf LLMs for requirements quality assessment, testing ten models from OpenAI and Anthropic across multiple generations. The evaluation involved one hundred independent runs, two separate requirement sets, and varied sampling temperatures—a parameter that influences model output diversity.

Key Findings#

The results paint a cautionary picture. The best-performing Anthropic model detected only 47% of issues identified by experts, while simultaneously producing false alarms on 11% of valid requirements. The researchers measured performance against ground truth standards derived from INCOSE quality criteria, which are widely used in systems engineering.

More troubling, performance deteriorated sharply when requirements demanded deeper software engineering judgment. Necessity and correctness issues—matters requiring nuanced technical reasoning—were almost never identified correctly. Additionally, newer generation models did not consistently outperform older versions, suggesting that raw model advancement does not automatically translate to improved capability in this specialized domain.

Why This Matters#

Requirements engineering sits at the foundation of system design; flawed specifications propagate downstream into costly redesigns, schedule delays, and budget overruns. The expectation that generative AI could automate what is typically a slow, expertise-intensive human task has been high. Yet this benchmark reveals that LLMs are not yet sufficiently reliable for unsupervised deployment in this role.

The findings carry particular weight for teams building agentic AI systems—autonomous agents that orchestrate multiple LLM calls. Combining unreliable modules into larger architectures risks amplifying their deficiencies rather than canceling them out.

The Defensible Path Forward#

The researchers recommend a human-in-the-loop approach as the near-term responsible role for off-the-shelf LLMs in requirements assessment: using them as assistants to surface candidate issues for expert review, rather than as final decision-makers.