LLMs Outperform Current Tools at Detecting Harmful Social Media Content

Harmful content on social media platforms creates pressing challenges for user safety and society at large. Traditional moderation methods—which typically depend on human reviewers, rule-based filters, and systems trained on large labeled datasets—struggle to keep pace with the volume of posts, adapt to emerging threats, and handle the inherently subjective nature of what constitutes harm.

A New Approach Using LLMs#

Researchers have developed a content moderation system powered by large language models that leverages few-shot learning through in-context examples. Rather than requiring massive amounts of pre-labeled training data, the approach teaches models to identify harmful material by providing just a handful of examples within a prompt. This method allows systems to adapt dynamically as new forms of harmful content emerge.

Outperforming Existing Solutions#

In testing across multiple LLMs, the team found that their few-shot approaches exceeded the performance of widely used proprietary tools, including Perspective and OpenAI's Moderation API, as well as previous state-of-the-art few-shot methods for the same task. The research underscores LLMs' capacity to recognize nuanced indicators of harm—including violent content and dangerous trend challenges—without extensive retraining.

Adding Visual Context#

The researchers also explored whether incorporating visual information could enhance results. They integrated video thumbnails into their moderation pipeline and tested various multimodal techniques to determine whether combining text and image analysis improved detection rates. The findings suggest that LLM-based moderation, especially when paired with visual signals, offers a scalable path forward for platforms managing content at scale.