New Method Helps Large Language Models Better Identify Specific Text Spans

Large language models are now commonly applied to span labeling tasks—identifying and categorizing specific phrases or sections of text—but their generative architecture presents a fundamental challenge: unlike specialized encoder models, they lack a built-in way to precisely reference portions of their input.

The Problem with Current Approaches#

Researchers at arXiv have documented how this limitation has spawned a range of makeshift prompting strategies, each with varying degrees of reliability. These workarounds fall into three broad families: directly annotating regions of the input text itself, using numerical indices to mark span boundaries, and attempting to reconstruct the exact text content that needs labeling. Content-matching approaches, in particular, struggle with consistency—models can misidentify or distort the spans they're supposed to extract.

Introducing LogitMatch#

To overcome these constraints, the research team developed LogitMatch, a constrained decoding method that enforces alignment between the model's output and valid spans actually present in the source text. Rather than letting the model generate arbitrary text and hope it matches, LogitMatch constrains the decoding process so the output must reference spans that genuinely exist in the input.

Performance Across Tasks#

The team tested all three strategy families—tagging, indexing, and matching—alongside LogitMatch on four diverse span labeling benchmarks. Their findings suggest that while simple text-tagging approaches remain dependable baselines, LogitMatch delivers meaningful improvements by solving the core misalignment problem in content-matching methods and outperforming alternatives in several scenarios.

Why It Matters#

Accurate span labeling is foundational for real-world NLP applications, from extracting named entities to detecting grammatical errors. By removing a key source of failure in generative models, LogitMatch helps bring them closer to the precision that specialized models can achieve—while retaining the flexibility and broad capability that large language models provide.