AI
Stacking Reliability Checks on LLMs Doesn't Improve Accuracy—It Just Sorts Answers
A rigorous study of six verification layers wrapped around language models reveals they withhold uncertain results rather than correct errors, exposing the hard ceiling of extraction-based approaches.
InfoQ
Researchers testing multiple verification layers designed to boost large language model reliability found that the layers sort answers by trustworthiness but rarely fix mistakes. The finding challenges the assumption that stacking checks—each catching errors the previous layer missed—leads to cumulative accuracy gains.
The team evaluated their approach on CADEC, an annotated corpus of 1,250 forum posts discussing adverse drug reactions, where every reaction is tagged with both text spans and standardized medical codes. Starting with a base extractor that correctly identified only about one answer in three, they added six layers of checks: vocabulary validation, model self-correction, majority voting, a second-model judge, and human refusal (withholding uncertain results for manual review).
How the Layers Performed#
The most telling finding: none of the paid verification layers actually corrected an answer. The free vocabulary check reliably sorted results—marking answers as accept, reject, or uncertain—without changing a single output. Self-correction barely triggered. Voting changed roughly equal numbers of right answers to wrong ones, producing near-zero net improvement. Even the second-model judge worked well when shown the menu of candidates but its verdicts went unread by downstream layers.
The vocabulary check did its job precisely: it flagged non-existent medical codes (a model invented 13–18% of codes when asked to recall them) and detected when quoted text didn't appear in the original post. Yet "working" meant withholding, not fixing. On ACCEPT verdicts, 21–23% of records passed with 74–82% accuracy—but that gain came entirely from discarding uncertain results, not improving them.
Recognition Beats Generation#
A crucial design choice emerged: asking the model to pick from a menu vastly outperformed asking it to generate codes from memory. When forced to recall SNOMED codes directly, performance was eight to twelve times worse, with the model inventing nonexistent codes roughly one time in six. Switching to a retrieval-based approach—embedding each quoted reaction and offering the model the 20 nearest medical concepts to choose from—improved F1 scores tenfold.
This framing has implications beyond this single task: the shape of the problem constrains what verification can fix. Once a mention escapes the extraction step entirely, no layer above it can recover it. On the held-out test set, the ceiling was just over half the annotated mentions detected at all.

