AI Safety
OpenAI launches AI-powered classifier and guardrails to curb child exploitation material
The company released a new image detection tool, ChatGPT safety warnings, and developer guidelines as part of a broader push to combat child sexual abuse material.
TechCrunch — AI
OpenAI has rolled out a suite of protective measures targeting child sexual abuse material (CSAM) amid mounting scrutiny over potential harms to minors. The initiative spans detection technology, user deterrents, developer frameworks, and law enforcement support.
New image detection technology#
The centerpiece is a fresh image classifier built in partnership with the National Center for Missing and Exploited Children (NCMEC). The tool identifies both naturally occurring and AI-generated CSAM imagery by spotting visual indicators of abuse and assigning confidence scores to guide reviewer workflows. Early testing reveals a substantial efficiency gain: the classifier reduces false positives by 48% while maintaining detection accuracy, and images it flags are twice as likely to genuinely depict abuse.
OpenAI is offering free API access to select organizations through collaborative networks including the Tech Coalition and ROOST.
In-app warnings and account safeguards#
ChatGPT now displays warning screens when users attempt prompts seeking child exploitation material. The system alerts users to the harm, directs them toward specialized support resources, and temporarily suspends accounts for 24 hours. Persistent violations trigger permanent removal. These deterrent messages were developed with input from organizations like Protect Children, the Lucy Faithfull Foundation, and researchers who independently tested similar interventions.
Developer tools and law enforcement support#
OpenAI is publishing safety guidelines for developers building on its platform, with early results showing increased adoption of child safety mechanisms and NCMEC reporting registration. For law enforcement, the company is providing dedicated plugins, investigative tools, advanced analytics capabilities, and training materials—all gated behind administrator controls.

