Agentic AI
New Framework Learns Hidden User Preferences to Build Smarter Mobile Agents
Researchers have developed a system that captures both explicit task steps and implicit personal habits from human behavior to create mobile agents that better match individual user preferences.
Multimodal AI models can now automate tasks on smartphones by watching and mimicking how humans interact with apps and interfaces. Yet existing mobile agents struggle with personalization because they only learn the obvious steps users take—not the hidden preferences and habits that shape how individuals actually work.
The Personalization Gap#
Mobile-use agents trained on human demonstrations typically focus on replicating explicit action sequences: tap here, type there, swipe that way. But this shallow approach misses the deeper layer—the implicit intention flows that reflect how specific users prefer to accomplish goals. One person might always check notifications first; another skips straight to the main task. Current systems treat all users the same, building generic agents rather than personalized ones.
A New Dataset and Framework#
Researchers from arXiv's Computation and Language community created MobileIAR, a benchmark dataset of real human-phone interactions from 40 English and Chinese-speaking participants. The dataset captures both what users intended to do and what actions they actually performed, allowing researchers to measure how well agents understand human intent.
Building on this foundation, the team proposed IFRAgent, a framework that disentangles two types of intention flows from demonstrations:
- Explicit flows: step-by-step task procedures that become standardized operating procedures (SOPs)
- Implicit flows: personal habits and preferences that get stored in a user-specific habit repository
How It Works#
IFRAgent combines several techniques to generate more personalized responses. An SOP extractor identifies standard procedures from demonstrations. A retrieval system with augmented generation finds relevant past experiences. A query rewriter—trained through knowledge distillation—refines vague user requests into clearer, personalized instructions that account for individual habits.
Results#
When tested on benchmark models, IFRAgent showed substantial gains: an average improvement of nearly 10 percentage points (53% relative improvement) in intention alignment with humans, and an 8-point gain (32% relative) in task completion rate. These results suggest that recognizing and capturing personal preferences alongside procedural steps is key to building mobile agents that feel less generic and more genuinely personalized.
