AI
Smaller models can evaluate better than they generate, new research finds
A new study challenges the convention of using large language models as judges, showing that smaller models can assess quality more efficiently by analyzing their internal representations rather than generating text.
Researchers investigating how language models evaluate text have discovered an unexpected asymmetry: smaller models often contain strong signals for judging quality even when their text generation is weak. This finding suggests evaluation can be handled more efficiently without relying on massive, costly models.
The problem with LLM-as-a-Judge#
The current standard for evaluating language model outputs treats these models as judges—researchers prompt large language models to rate the quality of text, explanations, or answers. While this approach works, it comes with notable drawbacks: the method is expensive, the evaluation process remains largely opaque, and results depend heavily on how the prompts are worded.
A new approach: Representation-as-a-Judge#
Rather than asking smaller models to generate evaluation scores through prompting, researchers propose extracting evaluative signals directly from the hidden computational layers within these models. The team discovered that even small language models encode rich information about quality in their intermediate representations, despite producing weaker outputs when asked to generate text.
This led to the Semantic Capacity Asymmetry Hypothesis: evaluation requires substantially less computational "capacity" than text generation and can operate on the latent features within a model, rather than its surface-level output. In practical terms, this means evaluation is not fundamentally dependent on model scale.
INSPECTOR: A new evaluation framework#
The researchers developed INSPECTOR, a probing-based system that predicts evaluation scores directly from the internal representations of smaller models. Instead of prompting for output, it analyzes what the model "knows" internally and converts that into aspect-level quality assessments.
Testing on reasoning benchmarks—including GSM8K, MATH, and GPQA—showed that INSPECTOR substantially outperformed smaller models using traditional prompting and performed comparably to full-scale LLM judges. The benefits include lower computational cost, greater reliability, and improved interpretability since the system is analyzing explicit internal patterns rather than opaque text generation.
