New framework reveals how large language models diverge from human judgment in content moderation

A withdrawn research paper from arXiv explores a critical gap in how we evaluate language models: most studies focus on raw performance metrics, while relatively little attention goes to understanding when and why these systems diverge from human judgment—especially as LLMs increasingly generate training data themselves.

The Ghost Annotator Framework#

The researchers propose combining conformal prediction—a statistical technique for quantifying uncertainty—with a representation system inspired by collaborative filtering to model how LLMs behave relative to human annotators. Rather than treating disagreement as error, they use Non-Conformity Scores to identify and analyze patterns of agreement and misalignment.

The core innovation is the Ghost Prediction metric: a way to flag instances where model outputs diverge from all available human annotations simultaneously. This allows researchers to isolate cases where an LLM stands entirely alone, rather than representing a minority position within a distribution of human labels.

Key Findings Across Models and Datasets#

The team evaluated four LLMs of varying sizes and architectures on four content moderation datasets. Their results reveal a striking pattern:

• All models grow more uncertain when human annotators themselves disagree—a sensible behavior.
• Larger models, paradoxically, become more confident when making predictions that no human annotator supports, suggesting they may be overfitting to training signal absent in real-world human consensus.
• A consistent demographic bias emerges across sociodemographic dimensions, implying the models carry structural misalignment rooted in their pretraining data rather than downstream fine-tuning.

Implications for Content Moderation#

The work highlights a fundamental challenge: as organizations rely on LLMs to bootstrap annotated datasets at scale, these systems may be quietly encoding their own biases into the very training data that will evaluate future versions. The framework offers a diagnostic tool to detect such drift before it compounds.