New benchmark tests AI agents' ability to learn hidden user preferences from examples

Embodied AI agents face a fundamental gap between what they're told to do and what they actually need to know. When asked to "prepare an apple," a robot must make dozens of unstated choices: wash or peel it? Where should it go? What's the right sequence of steps? These decisions hinge on personal preferences that users demonstrate through their behavior but rarely spell out.

A new study from arXiv investigates whether AI agents can fill this gap by learning user preferences from a handful of behavioral demonstrations—and whether they can then apply those learned preferences to entirely new tasks.

The Preference-based Planning Benchmark#

The researchers introduce Preference-based Planning (PbP), an evaluation framework designed to systematically measure this capability. The benchmark contains 5,000 test groups organized around 290 distinct preferences across three categories:

• Atomic action parameters: details like where to place an object or how to manipulate it
• Strategic interaction and placement policies: higher-level choices about how to arrange items or interact with the environment
• Temporal ordering constraints: the sequence in which actions should unfold

InTU: A Two-Stage Learning Framework#

To tackle this challenge, the team proposes Inferring the Unspoken (InTU), a two-phase approach. First, the system watches video demonstrations and attempts to articulate—in natural language—what preference is being expressed. Then, it uses that explicit verbal description to generate action plans for new situations.

The key insight is that language serves as an interpretable bridge between observed behavior and future planning, making preferences portable across different visual contexts.

The Preference-Acquisition Gap#

Experiments reveal a striking limitation in current video-language and language models. When given explicit, ground-truth descriptions of what a user wants, models plan actions quite effectively. But performance drops sharply when the same preference must be inferred from watching behavior unfold.

Notably, verbalization—the step of explicitly stating the inferred preference—consistently outperforms direct end-to-end planning, especially for stronger multimodal models. This suggests that forcing the system to articulate what it has learned before planning makes it more robust and transferable, particularly when the visual environment changes between training and test scenarios.