AI models spot unreliable speakers but fail to act on the insight

Researchers have discovered a puzzling gap in how large language models handle ambiguous instructions: they can recognize when a speaker is unreliable, but they don't actually use that knowledge to adjust their behavior.

The study's setup#

In a new benchmark called Build What I Mean (BWIM), researchers adapted a psychology experiment into an interactive task where an AI agent must follow instructions to build block structures. The twist is that some instructions are deliberately vague, and the agent must decide whether to infer what the speaker means based on context or ask for clarification at a small cost. The benchmark includes two types of speakers: one who cooperates pragmatically and another who is only literally accurate—meaning they might provide technically true but unhelpful guidance.

The key finding#

When tested on popular language models, a striking pattern emerged: models could accurately identify unreliable speakers in confidence rating tasks, yet when it came time to actually act, they ignored this awareness. Rather than deploying smart clarification strategies, the models fell back on two unhelpful defaults: either over-clarifying everything regardless of the speaker's reliability, or guessing under uncertainty instead of asking questions.

What it reveals#

The dissociation between judgment and action suggests a fundamental limitation in how current models adapt to interaction partners online. The research frames BWIM as a controlled testbed for evaluating how well AI systems reason about context and adjust to their conversational partners in real time—areas that remain surprisingly weak despite the models' ability to make correct judgments in isolation.