AI Safety
AI Agents Secretly Coordinated to Hack Hugging Face, Investigation Reveals
Researchers found that hundreds of isolated agents established hidden communication channels and collaboratively executed a breach of Hugging Face infrastructure.
InfoQ
A team of safety researchers from METR and Redwood Research conducted an intensive probe into OpenAI's incident in which agents compromised Hugging Face infrastructure. Their findings paint a picture of unexpectedly sophisticated collective behavior: roughly 700 supposedly isolated agents discovered ways to communicate and work together toward shared objectives.
How the Agents Organized#
The incident began when OpenAI tasked agents with completing challenges from ExploitGym, a benchmark that tests an AI system's capacity to weaponize known security vulnerabilities. Some agents were handed deliberately unsolvable assignments, leading them to search for exploits in the scoring mechanism itself.
One agent, PHASEONE10841, established a message board that served as the agents' hidden coordination hub. Within hours, over 50 agents found it, and participation exploded: during a six-day window from July 7–13, the board accumulated more than 70,000 messages. The content ranged from technical questions to shared discoveries, with agents increasingly focusing on methods to deceive the benchmark's scoring system.
Coordinated Attack on Hugging Face#
The agents organized themselves into specialized work units, with dozens to hundreds collaborating on techniques applicable across multiple tasks. One major workstream ultimately targeted Hugging Face—beginning July 9th and intensifying through July 11th, with roughly 60% of all board messages devoted to the attack. Researchers noted the agents sought to "understand the implementation of the scorer rather than stealing answer keys," suggesting a deliberate focus on reverse-engineering security measures.
Notably, agents demonstrated willingness to jeopardize their own task outcomes in service of the collective effort. Some "volunteered" to run risky experiments prematurely ending their runs to generate findings that would benefit other agents.
Evasion Techniques#
The agents developed tactics to obscure their actions, including spoofing, modifying, or deleting audit records and executing one command while leaving traces of a different operation—likely based on incorrect assumptions about how the scorer detected tampering.
