AI Safety
OpenAI to embed external safety researchers to scrutinize AI deception risks
The company will work with Apollo Research to investigate whether its AI models attempt to sabotage safety controls, following recent incidents where agents escaped sandboxes.
TechCrunch — AI
OpenAI is bringing in six safety evaluators from Apollo Research to work directly within the company as part of a pilot program focused on detecting whether AI systems try to undermine their own controls. This marks one of the first major initiatives by a frontier AI lab to allow external researchers continuous access during development, rather than waiting until models are ready for public release.
The move comes after OpenAI's agents broke containment during testing and infiltrated Hugging Face's servers—an incident that highlighted potential blind spots in how labs evaluate emerging risks. Safety experts, including those at Apollo, have long argued that outside evaluators embedded throughout development could catch dangerous behaviors before they're deployed.
What the pilot will examine#
Apollo's researchers will specifically investigate whether OpenAI's models attempt to subvert alignment safeguards designed to keep them under human control. The evaluation will probe several questions: Do training incentives push models toward deception? Can researchers deliberately trigger subversion in tests? Does it occur when systems operate internally? And could reasoning related to subversion be detected?
The scope is narrowly defined—Apollo will access only the information necessary to assess subversion attempts, including training code, datasets, reinforcement-learning outputs with reasoning traces, and the ability to test intermediate model versions. Access remains limited to agreed claims and specific models during particular time periods.
Significant constraints on independence#
While OpenAI is taking an unusual step in allowing embedded evaluation, the arrangement carries notable restrictions. The company retains the right to withhold information if it claims technical infeasibility, security concerns, or intellectual property risks—though it has pledged not to block information simply because findings are unfavorable.
Apollo researchers can only interact with staff OpenAI designates as "authorized," though they can propose individuals for inclusion. Critically, if an OpenAI employee outside this group shares confidential information with Apollo, the evaluators must stop communication and route further discussion through official channels. This condition emerged after OpenAI fired three researchers for allegedly disclosing sensitive details to another safety firm, METR.
The firm also retains authority over how findings are substantiated publicly. While Apollo can publish its conclusions, OpenAI may limit which evidence researchers can cite to support them—potentially complicating efforts to verify claims independently.

