AAAI Conference 2021 Conference Paper
Resilient Multi-Agent Reinforcement Learning with Adversarial Value Decomposition
- Thomy Phan
- Lenz Belzner
- Thomas Gabor
- Andreas Sedlmeier
- Fabian Ritz
- Claudia Linnhoff-Popien
We focus on resilience in cooperative multi-agent systems, where agents can change their behavior due to udpates or failures of hardware and software components. Current stateof-the-art approaches to cooperative multi-agent reinforcement learning (MARL) have either focused on idealized settings without any changes or on very specialized scenarios, where the number of changing agents is fixed, e. g. , in extreme cases with only one productive agent. Therefore, we propose Resilient Adversarial value Decomposition with Antagonist-Ratios (RADAR). RADAR offers a value decomposition scheme to train competing teams of varying size for improved resilience against arbitrary agent changes. We evaluate RADAR in two cooperative multi-agent domains and show that RADAR achieves better worst case performance w. r. t. arbitrary agent changes than state-of-the-art MARL.