New Study Maps the Design Space for AI Agent User Experiences

Computer use agents—AI systems that perform tasks within software interfaces based on user instructions—are rapidly advancing in capability, but their user experience remains largely unexplored. A new study from arXiv addresses this gap by mapping out the design landscape for agent UX and identifying the tools developers need to build and test these systems.

The Design Challenge for Agents#

While much research focuses on improving the underlying AI models that power these agents, little attention has been paid to how people actually interact with them. The researchers tackled this by constructing a comprehensive taxonomy spanning 21 subcategories of UX considerations—essentially a guidebook for thinking about agent design. This taxonomy reveals the range of factors that shape whether users find agents helpful or frustrating.

Building Better Prototyping Tools#

To understand what developers need, the team conducted a requirements elicitation study with 12 participants, six of whom were agents experts. This revealed five key activities and six desired capabilities that prototyping tools must support. Armed with these insights, the researchers built AgentUXlab, a design probe that lets developers experiment with different UX approaches on websites and test them directly in a browser environment.

Evaluating the Prototype Platform#

The team then studied 14 participants using AgentUXlab, which yielded practical tooling insights and design implications for future agent UX platforms. The work illustrates how intentional interface design choices can meaningfully influence user satisfaction and effectiveness when working with AI agents.

This research suggests that as agents become more prevalent in everyday software, deliberate attention to their user experience design will be as important as the algorithms that power them.