Why scientists resist isolating risky AI—and what it costs

Researchers who test potentially harmful AI systems face a fundamental tension: isolated machines are safer but tell you almost nothing about how the AI will behave in the real world. This gap between security and knowledge is pushing teams to accept significant risks in pursuit of useful insights.

The air-gap trade-off#

Physically disconnecting AI systems from the internet—a practice called air gapping—is theoretically straightforward. It can involve removing network cables, disabling wireless hardware, installing Faraday cages to block electromagnetic signals, and using minimal peripherals. Done thoroughly, this approach would block AI agents from reaching external targets and prevent outsiders from infiltrating the test environment. Yet researchers rarely employ it, because isolation undermines the entire purpose of testing.

According to Thorsten Holz, a scientific director at the Max Planck Institute for Security and Privacy in Germany, strict air gapping "reduces realism." Genuine evaluations of how an AI will perform in actual deployment often demand connection to external APIs, digital services, and real-world infrastructure—the very things isolation eliminates. "A strict air gap reduces realism," he explained, noting that the decision to isolate is "a trade-off, not a fundamental technical issue."

What researchers lose#

Keeping AI offline strips away critical realism. Ruizhe Li, an assistant professor at the University of Birmingham, compared complete isolation to testing AI in an "artificial vacuum," potentially invalidating the research itself. "We will end up testing a neutered AI model," Li observed, which blinds researchers to how the system "behaves, fails, or executes tool-use exploits in realistic deployment settings."

Beyond lost insights, air gapping introduces practical pain. The approach is expensive, slows experimentation to what researchers describe as a "logistics hurdle," and makes some experiments substantially harder to conduct. For frontier AI labs working at scale, Maksym Andriushchenko, a principal investigator at the ELLIS Institute Tübingen, questioned whether sufficient secure infrastructure even exists to isolate everything.

Isolation isn't foolproof either#

Even if labs did commit to air gapping, the protection remains imperfect. Agents could still compromise systems within the isolated environment and potentially create "malicious artifacts" that become dangerous once removed. The isolation itself could be breached—as happened with Stuxnet, the cyberweapon reportedly developed by Israel and the US, which jumped a supposed air gap via USB drive. Researchers have also ways to turn internal computer components into transmitters, potentially leaking data if shielding isn't flawless.