New sampling method beats greedy decoding for extracting information from documents

A team of researchers has demonstrated that generating multiple candidate outputs and choosing among them outperforms the standard practice of using greedy decoding for document-level information extraction (DocIE) tasks. Their approach challenges the conventional wisdom that variability in model outputs should be minimized.

The Problem with Greedy Decoding#

Document-level information extraction aims to automatically identify and structure entities, relationships, and events mentioned in text. The standard industry approach uses greedy decoding — selecting the single highest-probability token at each step — to ensure consistent, reproducible outputs. However, this strategy may forfeit opportunities to find better overall solutions.

ThinkTwice: A Sampling and Selection Framework#

The researchers introduce ThinkTwice, a framework that reverses this logic. Rather than accepting the first high-confidence answer, the system:

• Generates multiple candidate templates for the same document
• Applies a selection module to identify the most suitable candidate

The team proposes two selection strategies:

1. Unsupervised selection: Exploits agreement patterns across the generated candidates, assuming widespread consensus points to correctness
2. Supervised selection: Uses reward models trained on labeled DocIE data to learn which templates are most likely correct

Addressing Data Scarcity#

A key challenge in training supervised selection models is the scarcity of annotated reasoning traces paired with output templates. To overcome this, the researchers developed a rejection-sampling-based method that generates synthetic training data, pairing output templates with reasoning explanations. This silver-standard data enables effective supervised model training without requiring extensive manual annotation.

Performance Results#

Empirical testing shows that both unsupervised and supervised versions of ThinkTwice consistently outperform greedy baselines and existing state-of-the-art supervised approaches, particularly when paired with reasoning models. The findings suggest that treating output variability as a feature rather than a bug opens new pathways for improving language model performance on structured extraction tasks.