Human-AI Collaboration
How Users Signal When They Want to Switch Roles with AI Partners
A study reveals that eye-movement patterns and other behavioral cues can predict when people want to change how they divide tasks with AI during collaborative work.
Effective human-AI teamwork often depends on both parties knowing which responsibilities belong to whom. But as tasks evolve, people may want to reassign who does what—and new research shows that AI systems could recognize these shifts through subtle behavioral signals.
The Chess Study#
Researchers at an HCI lab conducted an experiment using a creative framework: hand-and-brain chess. On each turn, 21 chess players made a binary choice about role allocation. They could either select which chess piece to move (the "brain" role) while their AI partner chose the specific move (the "hand" role), or reverse that arrangement. Across 1,100+ individual decisions, this yielded a rich dataset of moments when players stuck with their current role versus when they switched.
What the Data Revealed#
The results paint a nuanced picture of collaboration dynamics:
- Most of the time, players stuck with their assigned role rather than frequently switching between them
- When switching occurred, it coincided with more exploratory eye movements—players gazed around the board more tentatively before deciding to change roles
- Subsequent moves were lower quality after a role switch, suggesting the transition itself carried a cost
Using these signals plus task-specific features, the team trained a machine-learning classifier to predict switch-versus-stay decisions. It achieved a PR-AUC of 0.56, outperforming a random baseline of 0.39. Eye-tracking data and task context proved to be the strongest predictors.
Beyond Simple Control Dynamics#
Interviews with participants revealed that role switching was not simply about delegating versus controlling. Both roles required players to make one decision and trust their partner with another. However, many participants felt that choosing the piece type gave them greater control over the game's direction, even though they still depended on the AI for the move itself.
What drove these preferences? Participants cited three main factors: how capable they perceived the AI to be, which subtask they found harder, and how much influence they wanted. These subjective judgments shaped when they chose to switch.
