Agentic AI
Study reveals widespread unmanaged configuration in LLM coding agents, proposes deterministic control framework
A research analysis of 10,000+ GitHub repositories finds that LLM agent configurations are treated as undeclared shared components with minimal oversight, prompting a call for structured governance.
Researchers examining LLM-powered coding agents have identified critical gaps in how their configurations are managed across the industry. A study published on arXiv surveyed over 10,000 public GitHub repositories, analyzing 6,145 agent configuration files to understand current practices.
Key Findings#
The analysis uncovered several concerning patterns:
-
Configuration reuse without tracking: Nearly 10% of tracked configuration file paths are exact SHA-256 duplicates across independent repositories. When adjusted for forks and without arbitrary thresholds, 75.5% of these clones cross organizational boundaries—suggesting configurations propagate as unmanaged shared components.
-
Infrequent updates: Agent configurations are rarely revised—58% exist in repositories with only a single commit, and are updated at just 0.4-0.6 commits per month when normalized for age against CI/CD workflows.
-
Minimal permission controls: Less than 1% of agent configurations declare permission boundaries, compared to 33% of GitHub Actions workflows.
Proposed Solution: Rel(AI)Build#
To address these gaps, the researchers propose a deterministic control plane that treats agent definitions as a managed supply chain. The framework includes:
- Content addressing and audit trails: SHA-256 content addressing with HMAC-stamped lockfiles and hash-chained audit logs
- Tiered permissions: Enforced permission boundaries and attack-derived blocklists before LLM invocation
- Structured workflows: Phase state machines with requirement-to-file-to-test traceability
- Multi-target compilation: Single canonical definitions compile to seven IDE targets
- Drift detection: Prompt changes identified via Jaccard similarity analysis
The researchers emphasize that governance of this layer must remain deterministic and tool-agnostic rather than delegated to further LLM orchestration. Conformance tests validate each mechanism, though developer outcome evaluations remain future work.
