Technology
San Francisco Demands Meta Stop Distributing Ads With AI-Generated Child Abuse Content
The city attorney issued a cease-and-desist order after more than 350 ads featuring AI-manipulated images of real children were found running repeatedly across Meta's platforms.
WIRED — AI
San Francisco's city attorney has formally ordered Meta to halt paid advertisements containing AI-generated child sexual abuse content and provide a full accounting of how such material repeatedly bypassed the company's safety systems.
The action follows WIRED's reporting on findings by the Tech Transparency Project, which identified over 350 ads that used artificial intelligence to transform photographs of minors—including real individuals and members of a European royal family—into video clips depicting sexual acts. These ads were designed to drive traffic to apps that generate nude imagery and other nonconsensual intimate content.
The Scale of the Problem#
The problematic advertisements reached approximately 29,000 accounts across the European Union, with additional exposure in the US, Australia, and India. They appeared on Facebook, Instagram, and Threads. Notably, more than 250 of the ads continued running after the initial public exposure in August, signaling Meta's failure to address the issue comprehensively. According to Meta, the total ad spend across all 300-plus identified ads was under $5,000.
San Francisco's Response#
City Attorney David Chiu characterized the situation as evidence of systemic failure, stating that "Meta has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them." The four-page cease-and-desist letter demands Meta explain how its moderation systems permitted the content, detail its reporting procedures to the National Center for Missing and Exploited Children, and describe how it handles repeat offenders. Meta has 28 days to respond.
Chiu emphasized that the persistence and volume of violations contradict Meta's stated zero-tolerance policy for child exploitation. "No company—particularly one that claims all advertisements are reviewed and approved prior to distribution—should permit its advertising systems to be used in this manner," he wrote.
Meta's Defense and Ongoing Issues#
Meta argues it lacks jurisdiction over the San Francisco complaint, claiming there is "no indication" the ads displayed within city limits. The company has maintained that the vast majority of identified ads had already been removed before public reporting and that most generated fewer than 200 impressions. However, researchers continue documenting thousands of ads promoting "nudify" apps and similar harmful tools across Meta platforms, many from easily identifiable repeat advertisers.
