Meta slow to stop ads promoting AI tools for generating child abuse imagery

Meta left hundreds of ads promoting AI-powered apps designed to create sexually abusive imagery of minors on its platforms for days before taking action. The material—including deepfakes created from photographs of real children—violated both platform policies and federal law.

Scope of the Problem#

The Tech Transparency Project released an investigation revealing that Meta failed to catch at least 332 ads containing AI-generated child sexual abuse material (CSAM) circulating across Facebook and Instagram this year. The bulk of these promotional materials originated from Chinese developers, many hawking "nudify" applications that leverage artificial intelligence to digitally alter images of children.

Researchers were able to trace numerous CSAM ads directly back to genuine photographs of minors. Victims included a young member of a European royal family, whose publicly available image was converted into video depicting a sexual act, and a preteen Instagram user whose sports uniform photo was transformed into abusive content. In another case, stock photography showing a child in athletic wear was manipulated to create a video simulating abuse.

Legal and Platform Violations#

The presence of these ads raises serious legal concerns. The U.S. Department of Justice has determined that AI-synthesized child abuse material carries the same legal weight as traditional CSAM, making the ads' continued presence a potential violation of federal pornography statutes.

How the Ads Targeted Users#

Advertisements used predatory messaging to lure clicks, with captions such as "our AI transforms your hidden desires into hypnotic videos" and claims that apps had "no restrictions." One particularly explicit ad promised potential users access to a tool where "all the characters are real people," attempting to normalize the abuse.

The preponderance of victims depicted in these ads were preteen girls, according to the investigation.