AI
New Self-Play Algorithm Lets Language Models Attack and Defend Themselves to Improve Safety
Researchers introduce Self-RedTeam, an AI training method where language models continuously role-play as both attackers and defenders to find and patch vulnerabilities in real time.
Current approaches to making language models safer follow a slow, reactive pattern: attackers find flaws, defenders patch them, then the cycle repeats. This lag means security threats are always a step ahead. A new research effort proposes a fundamentally different strategy: letting a single model simultaneously play both attacker and defender, continuously learning to exploit and patch vulnerabilities.
How Self-RedTeam Works#
The method, called Self-RedTeam, uses multi-agent reinforcement learning where one policy alternates between two roles. When acting as an attacker, the model generates adversarial prompts designed to break safety guidelines. When acting as a defender, it attempts to respond safely to those same prompts. A separate reward model judges the outcomes, incentivizing the attacker to find new vulnerabilities and the defender to handle them securely.
Both roles employ hidden chain-of-thought reasoning—internal strategic planning invisible to the other—mirroring how human adversaries and defenders might work. This back-and-forth unfolds continuously during training, enabling the model to co-evolve its attack and defense capabilities.
Theoretical Grounding and Results#
The approach draws on two-player zero-sum game theory. Researchers prove that if the self-play process converges to a Nash Equilibrium, the resulting defender will produce safe responses even against unseen adversarial inputs. Testing across five models from the Llama and Qwen families, Self-RedTeam uncovered 17.80% more diverse attacks (measured by SBERT similarity) compared to baseline methods and improved safety scores by up to 95% across 14 safety benchmarks.
Why It Matters#
The shift from reactive patching to proactive co-evolution could transform how language models achieve robustness. Rather than waiting for new attack vectors to emerge in the wild, models can discover and defend against threats during development. This approach enables what researchers term "self-improvement" in safety—the model itself becomes an engine for generating new security challenges and solutions in parallel.
