Large language models show mixed ability to inject factual commonsense into logical reasoning

Determining whether one statement logically follows from another—a task known as natural language inference—often hinges on real-world understanding that humans take for granted. A new study explores how well large language models can generate and apply this kind of commonsense knowledge to improve their reasoning performance.

The challenge of inference with commonsense#

Natural language inference systems must evaluate whether a premise entails, contradicts, or bears no relation to a hypothesis. Unlike formal logic problems, these determinations frequently require background knowledge about how the world works. The research team investigated whether LLMs could reliably produce factual commonsense statements to feed into inference tasks, and whether such knowledge actually improves results.

Evaluating model reliability#

Testing two major models—Llama-3.1-70B and gpt-oss-120b—the researchers found a stark divide in performance. One model generated axioms that were predominantly accurate, while the other produced more incorrect statements than correct ones. Because commonsense facts lack explicit textual grounding, the team developed a novel reference-free evaluation method using an LLM-as-Judge framework to assess factuality without relying on existing sources.

Three approaches to inference#

The research compared three strategies across the SNLI and ANLI benchmarks:

• Direct inference: Models answering without additional context
• Augmented inference: Models drawing on generated commonsense axioms
• Hybrid approach: Selectively using only those axioms the model itself judged to be factual

The hybrid method delivered the strongest gains, improving accuracy by 3.87% to 8.5% depending on configuration. Beyond raw accuracy, targeted commonsense knowledge helped models overcome a systematic bias toward classifying examples as neutral, by grounding responses in concrete real-world details.