AI
AI in Systems Engineering Needs Epistemic Guardrails, Not Just Model Access
Researchers argue that giving AI tools read-write access to structured engineering models risks hallucination unless we build verification layers into the data architecture itself.
Machine-readable models like SysML v2 are now within reach of AI systems—but raw access alone is not enough to ensure trustworthy machine participation in systems engineering. According to a new paper submitted to arXiv, the real challenge lies beneath the surface, in the data architecture that surrounds the model itself.
The Problem: Filling Gaps with Training Data#
When an AI system queries a seemingly complete engineering model—searching for derivation chains, status information, or evidence—it frequently encounters voids. Missing epistemic labels, broken provenance trails, and gaps that the model cannot resolve do not cause the system to pause or decline. Instead, the AI fills those blanks from its training data, a source that is neither verifiable nor subject to governance. This gap-filling mechanism can inject unverified information into the record.
To ground the concern in real-world practice, the authors examined a public SysML v2 reconstruction of the Apollo 11 mission, a project that exemplifies current standards rather than representing obvious deficiency.
Read-Side Adequacy and Write-Side Admissibility#
The researchers introduce epistemic adequacy as a missing property in model architecture. The concept splits into two complementary halves:
- Read-side adequacy: When an AI queries the model for derivation, status, or provenance, the model answers directly from structured data rather than inviting the AI to guess or extrapolate.
- Write-side admissibility: Before an AI contribution is recorded, it passes through governance gates designed to verify its reliability.
These two principles break down into five operational criteria, four on the read side (supported by the Apollo case and convergent research) and one on the participation side (presented as a hypothesis awaiting empirical testing).
A Framework for Governed Participation#
Rather than redesigning models from scratch, the authors propose the Governed-Query Architecture Framework, which steers AI participation through conventions that systems engineers already use—specifically, the viewpoint mechanisms embedded in their standard practices. The architecture can range from lightweight inline metadata extensions to a full substrate-native multi-model store.
