New Benchmark Tests How Well AI Agents Juggle Farm Work and Social Life in Virtual Worlds

Researchers have introduced StarDojo, a benchmark designed to push multimodal large language model (MLLM) agents to navigate complex real-world scenarios that blend practical work with social engagement. Using the farming simulation Stardew Valley as its testing ground, the framework assesses whether advanced AI systems can manage both livelihood activities and community relationships simultaneously.

A Dual-Skill Evaluation#

Traditional benchmarks tend to isolate narrow skill domains, but StarDojo demands agents balance multiple responsibility types at once. The benchmark includes 1,000 carefully constructed tasks spanning farming, crafting, exploration, combat, and social interactions. For researchers focused on efficiency, a streamlined set of 100 representative tasks is also available. Rather than requiring agents to manipulate keyboards and mice, the interface offers a more direct command structure that works across major operating systems and allows parallel evaluation of multiple agents.

Performance Gaps Remain Substantial#

Testing leading MLLM-powered agents revealed significant shortcomings. The highest-performing model—GPT-4.1—managed only a 12.7% success rate, pointing to fundamental limitations in visual understanding, multimodal reasoning, and fine-grained control. These results suggest that even state-of-the-art systems struggle with the integrated decision-making that production-living simulations demand.

Why This Matters for AI Research#

Autonomous systems destined for real-world deployment must eventually grapple with messy, interconnected tasks rather than isolated problems. By combining economic production (farming and crafting) with interpersonal elements (building friendships, managing community reputation), StarDojo mirrors the hybrid cognitive demands of human society—and exposes where foundation agents currently fall short.