Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization

Generalization in Multi-agent Reinforcement Learning (MARL) is challenging. Introducing a diverse set of co-play agents typically boosts the agent’s generalization to unseen co-players. However, the extent to which an agent is influenced by co-players varies across scenarios and environments; thus, the improvement in generalization introduced by diversifying co-players also varies. In this work, we introduce Level of Influence (LoI), a novel metric measuring the interaction intensity among agents within a given scenario and environment. We show that LoI can effectively predict the disparities in the benefits of diversifying co-player distribution across scenarios, offering insights into optimizing training cost for varied situations. The code is available at: https: //github. com/ ThomasChen98/Level-of-Influence.

Authors

• Yuxin Chen Nanjing University of Science and Technology
• Chen Tang University of California, Los Angeles
• Ran Tian
• Chenran Li
• Jinning Li
• Masayoshi Tomizuka
• Wei Zhan

Keywords

• Multi-agent reinforcement learning
• Learning agent-to-agent interactions
• Multi-agent systems