Language models show inconsistent reasoning across deduction, induction, and abduction

Modern large language models can acquire new abilities through multiple pathways—but these capabilities do not align in the systematic way we might expect. Researchers studying how LMs learn found a stark disconnect between different reasoning modes that reveals gaps in how these systems process information.

Three Paths to Learning#

Language models can absorb tasks in three distinct ways. Instruction following occurs when users directly describe a task in plain language, invoking deductive reasoning. Few-shot prompting works by providing a handful of examples without explanation, relying on inductive reasoning to infer the pattern. Instruction inference presents examples and asks the model to first articulate the task rule in natural language before answering, engaging abductive reasoning.

The Disconnect#

A team analyzing four models from the GPT and Llama families across two problem types—arithmetic functions and machine translation—uncovered a surprising fragmentation. Across these tests, the researchers identified a fundamental dissociation between reasoning types: models sometimes mastered tasks through few-shot examples despite being unable to articulate why they worked, while in other cases they generated useful task descriptions yet completely failed when given human-written versions of those same rules. This lack of coherence challenges the notion that modern LMs reason in an integrated, principled way.

What It Means#

The findings underscore that—even at the frontier of LM capability—the underlying learning mechanisms are far less unified than the similarity of these prompting methods might suggest. Rather than reasoning systematically across different modalities, large language models appear to engage separate pathways that can succeed or fail independently, pointing to fundamental limits in their reasoning architecture that merit further investigation.