AI-assisted coding in science needs governance standards, not bans

Scientists are already using large language models to generate code, design test cases, and write documentation in their research—yet these practices remain largely invisible in formal methods and published papers. Rather than preventing AI-assisted development, the scientific software community needs to establish clear governance frameworks that maintain quality assurance and accountability.

The hidden risk#

This informal, largely undocumented use of AI in scientific software poses a particular danger for systems subject to stringent oversight. Tools involved in safety-critical modeling and simulation—especially those governed by Nuclear Quality Assurance Level 1 (NQA-1) standards—demand rigorous traceability, independent verification, and thoroughly documented procedures. When AI enters this landscape without structured guidelines, it undermines the very controls designed to ensure reliability.

A framework for responsible adoption#

Rather than asking whether scientific software should embrace AI assistance, researchers should focus on how to govern it responsibly. A new study demonstrates this approach using TMAP8, an open-source fusion energy code for modeling tritium migration. The authors propose a structured framework specifically designed for AI-assisted verification and validation (V&V) case development within strict quality assurance constraints.

Validation cases offer an ideal testing ground for such governance: their solutions are known in advance, correctness can be measured objectively, errors are detectable by design, and all artifacts are fully auditable.

Key principles#

The proposed guidance operates within existing NQA-1 requirements while preserving human accountability and establishing mandatory disclosure and review standards. The approach demonstrates that responsible AI-assisted scientific software development is achievable when built on transparency, traceability, and clear governance protocols.