Learning Structured Communication for Multi-Agent Reinforcement Learning

This paper investigates multi-agent reinforcement learning (MARL) communication mechanisms in large-scale scenarios. We propose a novel framework, Learning Structured Communication (LSC), that leverages a flexible and efficient communication topology. LSC enables adaptive agent grouping to create diverse hierarchical formations over episodes generated through an auxiliary task and a hierarchical routing protocol. We learn a hierarchical graph neural network with the formed topology that facilitates effective message generation and propagation between inter- and intra-group communications. Unlike state-of-the-art communication mechanisms, LSC possesses a detailed and learnable design for hierarchical communication. Numerical experiments on challenging tasks demonstrate that the proposed LSC exhibits high communication efficiency and global cooperation capability.

Authors

• Junjie Sheng
• Xiangfeng Wang
• Bo Jin The Third Research Institute of Ministry of Public Security
• Wenhao Li
• Jun Wang
• Junchi Yan School of Artificial Intelligence & Department of CSE & MoE Lab of AI, Shanghai Jiao Tong University
• Tsung-Hui Chang School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China Shenzhen Research Institute of Big Data, Shenzhen, China
• Hongyuan Zha Shenzhen Institute of Artificial Intelligence and Robotics for Society The Chinese University of Hong Kong, Shenzhen

Keywords

• Learning to Communicate
• Multi-Agent Reinforcement Learning
• Hierarchical Structure
• Graph Neural Networks