AI
New method trains AI agents to learn from both successes and failures without human-designed rewards
Researchers propose a self-supervised learning framework that leverages agents' own experiences and negative feedback to improve autonomous system training.
Training autonomous systems typically relies on two key methods: human-specified reward functions or learning from pre-recorded demonstrations. Both approaches demand extensive human input tailored to each specific behaviour, which becomes impractical for systems designed to adapt over long periods in unpredictable environments.
A self-supervised alternative#
Recently, Goal-Conditioned Supervised Learning (GCSL) through self-imitation has emerged as a way around this limitation. The approach allows agents to extract policy insights from their own actions by strategically reassigning goals after the fact—a form of learning that doesn't require external supervision. Yet this framework has two key shortcomings: it can amplify an agent's existing biases since it learns only from self-generated data, and it focuses learning exclusively on successful trajectories, ignoring the valuable lessons that come from failure.
Introducing negative feedback into the framework#
To overcome these constraints, researchers have developed GCSL-NF (GCSL with Negative Feedback), which evaluates each trajectory in two ways. The agent learns from positive signals when trajectories align with relabelled goals, and from corrective signals when the same trajectory fails to match the originally intended objective. Rather than requiring a pre-built reward function or distance metric, the similarity function for corrections is learned through contrastive methods based on the neighbourhood relationships the agent itself discovers.
Performance and practical impact#
Experimental results indicate that GCSL-NF reduces agents' reliance on their initial assumptions, increasingly harnesses the value of negative feedback as training advances, and delivers performance comparable to or better than established GCSL and Hindsight Experience Replay (HER) approaches. This method proves especially valuable for self-adaptive systems that face diverse, shifting, or difficult-to-engineer adaptation goals—precisely the environments where pre-engineered reward functions fall short.
