OpenAI has announced new tools and features aimed at combating child sexual abuse material (CSAM) amidst growing safety concerns surrounding AI. These include a new image classifier, warnings and temporary account blocks within ChatGPT, updated guidelines for developers, and tools for law enforcement.
The new image classifier can detect CSAM images, including those generated by AI. OpenAI is offering select companies free access to this tool via its API. Early testing with partners showed the classifier flagged 48% fewer images while identifying the same number of abusive images, reducing reviewer workload, and images flagged by the classifier were twice as likely to be abusive.
ChatGPT will now feature warning screens for users attempting to access content that sexualises children. Such attempts will trigger an alert, direct users to specialised support, and result in a 24-hour account suspension. Repeated offences could lead to permanent account removal.
OpenAI is also publishing guidelines for developers using its tools to help prevent, detect, and report CSAM. Additionally, it is equipping law enforcement with AI-powered features, including dedicated plugins, investigative tools, and complex data analysis, along with training and a field guide.