Large-Scale Study Reveals Security Gaps in Agent Context Files for AI Coding Tools

A new empirical study of agent context files—the instruction documents that guide AI coding tools—reveals a significant mismatch between what developers document and what's necessary for safe, performant code generation.

What the Study Found#

Researchers examined 2,303 context files from 1,925 repositories to map how developers instruct agentic coding systems. The analysis uncovered a stark imbalance in what gets documented: while developers consistently specify functional requirements like test procedures (75.9%), implementation details (70.8%), and system architecture (68.1%), they rarely address non-functional concerns that matter for production systems.

The gap is particularly pronounced for security (14.8%) and performance (14.5%), suggesting developers treat context files primarily as guides for what code should do rather than how it should behave under real-world constraints.

How Context Files Evolve#

Rather than static documentation, these files function more like configuration code. Researchers found they undergo frequent, incremental updates—accumulating instructions over time in ways that make them complex and difficult to parse. This evolution mirrors how developers manage configuration systems, indicating that context files have become a distinct category of artifact requiring specialized tooling and practices.

The Guardrails Problem#

The study highlights a critical gap in agent-assisted development: while context files successfully make agents functional, they provide few safeguards to ensure the code agents write is secure or resilient. With millions of developers now using agentic coding tools, the absence of standardized practices for documenting security requirements and performance expectations represents an emerging risk.

The findings suggest the field needs better tools and conventions for writing context files that account for non-functional requirements alongside functional goals.