AI
Hugging Face grapples with widespread abuse by nonconsensual deepfake creators
A new investigation reveals that open-source AI models hosted on the platform are being weaponized to generate sexual deepfakes without consent, despite global regulatory pushback.
WIRED — AI
While governments and law enforcement worldwide are escalating crackdowns on deepfake pornography, the open-source AI platform Hugging Face continues to host thousands of models being exploited to create nonconsensual intimate images. A report from European nonprofit AI Forensics released this week documents how the problem persists at scale on one of the world's largest repositories of AI code and datasets.
The scope of abuse#
Researchers at AI Forensics tested nine popular image-editing applications on Hugging Face and found that seven of them could readily transform clothed photos of women into nude images. In a separate experiment, the team deployed decoy image-editing tools that recorded incoming requests without generating output. Over seven days, they logged more than 1,000 prompts—73 percent of which were explicitly sexual. Among those sexual requests, 83 percent sought to digitally undress or sexualize the person depicted, with 95 percent targeting women. The data revealed a darker pattern: roughly 6.7 percent of sexual requests involved images of apparent minors.
"Most of the Spaces [tested] can be used for generating nonconsensual intimate images, and users are actually using it for these purposes," according to Paul Bouchaud, a lead researcher at AI Forensics. The abuse encompasses far more than simple undressing—prompts documented by the researchers included requests to add degrading imagery, alter bodies to simulate sex acts, and manipulate religious attire in ways designed to humiliate.
Platform gaps in defense#
Unlike mainstream generative AI systems from OpenAI and Google that employ content safeguards to block undressing requests, the open-source models inspected by AI Forensics contained no such protections. Researchers triggered abuse functionality with plain language—a six-word phrase: "Same pose, same face, but topless"—without needing to circumvent any security measures.
"No safeguards at all are being implemented at a platform level," Bouchaud noted. "Only the developer can, if they want, implement some, and most of them do not. Hugging Face can easily filter what is coming in and coming out of a system."
Hugging Face, which maintains content policies nominally prohibiting sexual deepfakes created without consent, did not answer repeated questions from journalists about its moderation processes or safety infrastructure. The company removed some pages promoting nude-generation tools after being contacted, though the connection to that reporting remains unclear.
