New Training Method Cuts LLM Hallucinations Below 5% by Teaching Models to Admit Uncertainty

Hallucination remains a critical problem for large language models—they confidently produce false information at rates between 20–40% on factuality benchmarks. A new approach tackles this head-on by flipping the training paradigm: instead of pushing models to always generate answers, researchers teach them to recognize the limits of their knowledge.

Training Models to Say "I Don't Know"#

The foundation of the new method, called ConfQA, is elegantly straightforward. During fine-tuning, when a model generates a correct answer, training reinforces that response. When it would otherwise hallucinate, the model learns to output "I am unsure" instead. This simple reformulation, applied across multiple benchmarks, slashes hallucination rates below 5%.

Two design elements make this strategy work:

• A dampening prompt ("answer only if you are confident") that explicitly discourages overconfident predictions
• Training data composed of atomic factual statements (such as knowledge graph attributes), which helps calibrate the model's confidence judgment and improves generalization across different domains and question types

Triggering Retrieval Only When Needed#

Building on ConfQA, the researchers developed ConfRAG, a retrieval-triggering framework that activates external knowledge retrieval only when the model expresses uncertainty. Rather than always augmenting generation with retrieved documents—a costly operation—ConfRAG uses the model's confidence signal as a gate.

This hybrid approach achieves two wins:

• Accuracy above 95% in ideal conditions
• Over 30% reduction in unnecessary external retrievals, lowering computational overhead

The framework effectively addresses longstanding tensions in retrieval-augmented generation: when to retrieve, whether to trust the model's internal knowledge, and how to balance accuracy against computational cost.

Why This Matters#

Factual reliability is central to deploying LLMs in high-stakes domains—medicine, law, journalism, customer support. A model that confidently lies is worse than one that admits uncertainty and triggers a lookup. ConfRAG demonstrates that teaching confidence calibration and selective retrieval can substantially improve both trustworthiness and efficiency.