New Data Augmentation Method Helps Robots Recover from Execution Failures

Vision-Language-Action (VLA) models have demonstrated remarkable ability to manipulate objects when trained on large-scale imitation learning datasets. Yet most of these datasets focus almost exclusively on successful execution sequences, leaving a critical gap: they provide little to no guidance for what robots should do when things go wrong.

When a robot is deployed in the physical world, small deviations from the original training conditions are inevitable. The system encounters situations—distributional shifts—that fall outside its training experience. Without explicit instruction on how to recover from these failures, the policy often has no way to correct course and falls back into error states.

The Core Problem#

Traditional imitation learning datasets overwhelmingly capture optimal trajectories where tasks succeed as planned. This creates a blind spot: when real-world physics or environmental variations cause a robot to deviate slightly from a demonstrated action sequence, the policy lacks any recovery data to learn from. The system essentially has no experience with failure modes, even plausible ones that naturally occur during physical operation.

RESample's Solution#

Researchers have developed RESample, a coverage-guided data augmentation framework that actively expands demonstration datasets with failure recovery trajectories. The method works by:

• Identifying realistic failures: A conservative coverage function scans the existing dataset to detect failure cases that likely occur in the real world but are absent from the successful demonstrations
• Generating corrective actions: The framework performs exploratory sampling to synthetically create scenarios where a robot experiences a deviation, then immediately learns a recovery action
• Extending training coverage: These augmented failure-recovery sequences are added back to the training dataset, expanding what the policy can learn from

Real-World Results#

Testing on the LIBERO benchmark and live manipulation tasks showed consistent improvements. Policies trained with RESample achieved absolute success rate gains of up to 12 percent while requiring no more than 20 percent additional training samples. The approach efficiently amplifies the robustness of learned policies without explosive growth in dataset size.