New multi-agent framework helps robots adapt on the fly during complex tasks

Researchers have developed a collaborative planning framework that allows multiple robots to tackle long-horizon manipulation tasks by continuously assessing and refining their strategies in response to real-world obstacles.

The system, called REMAC, leverages vision-language models to help robots understand their environment and break down complex jobs into manageable steps. Rather than relying on pre-loaded knowledge or hand-crafted instructions, it lets robots observe the scene directly and reason about what to do—then adjust course when conditions shift unexpectedly.

How it works#

The approach hinges on two core capabilities:

• Self-reflection: Robots continuously check whether task preconditions are met and evaluate whether completed actions achieved their intended outcomes. This lets them spot planning mistakes and course-correct.
• Self-evolution: When reflection reveals that the current plan won't work, robots dynamically generate alternative strategies tailored to what they observe in the scene.

Once a robot has made sufficient progress, it can signal a second robot to start working on a parallel subtask—boosting overall execution speed without requiring fixed multi-robot coordination from the start.

Validation and results#

The team built a benchmark environment with 27 distinct task styles across 4 categories, drawing on the RoboCasa simulation suite and populated with 50+ different objects. They tested REMAC against several state-of-the-art reasoning models, including DeepSeek-R1, o3-mini, QwQ, and Grok3.

Compared to single-robot baselines, the multi-agent approach achieved an average success rate improvement of 40% and execution efficiency gains of 52.7%.

Why it matters#

Traditional robot planners struggle when the environment deviates from their training assumptions—for example, discovering a microwave door is already closed when the task expects it to be open. By embedding reflection and adaptive reasoning into the planning loop, REMAC sidesteps the need for exhaustive task-specific prompt engineering and allows robots to operate more robustly in real-world variability.