AI Security
Researchers Demonstrate Attacks on Multi-Agent AI Systems Using Only Partial Information
A new study shows that attackers can generate adversarial strategies to sabotage multi-agent reinforcement learning systems even without full visibility into the target's state.
Security vulnerabilities in multi-agent reinforcement learning (MARL) systems pose an emerging risk as these technologies expand into real-world applications like drone swarms and robotic coordination. Researchers have now identified a critical gap in how we think about protecting these systems: adversaries can successfully craft sabotaging strategies even when they lack complete information about what their targets are observing.
The Attack Landscape#
Multi-agent reinforcement learning has unlocked exciting possibilities for coordinated automation—from synchronized drone operations to collaborative robotic manipulation. Yet recent work has revealed that well-trained MARL agents can be rapidly exploited through adversarial attacks. Prior demonstrations have shown attackers reducing the win rate of superhuman game-playing AI to around 20%. However, most prior research assumed attackers had perfect knowledge of the entire game state, a unrealistic constraint that limited understanding of real-world threats.
A New Attack Method for Partial Observability#
The new work introduces SUB-PLAY, a black-box attack strategy designed specifically for scenarios where attackers face partial observability—meaning they can only see a limited portion of the environment. The approach innovates by constructing multiple subgames that help overcome the limitations imposed by incomplete information. Additionally, it shares experience across different attack policies to strengthen their ability to find and exploit weaknesses.
Testing across three different types of partial observability constraints, the researchers demonstrated that SUB-PLAY effectively degrades victim agent performance. Detailed analysis of the targeted agents' internal network activations revealed that adversarial policies trigger substantially different patterns of neural activity than normal gameplay.
Defense Considerations#
The paper also evaluates three potential defensive measures to help mitigate such attacks, offering practical guidance for organizations deploying MARL in competitive settings. This balanced assessment—both attack and defense—aims to inform more secure development practices as these systems move toward production use.
