Consensus Learning for Cooperative Multi-Agent Reinforcement Learning

Almost all multi-agent reinforcement learning algorithms without communication follow the principle of centralized training with decentralized execution. During the centralized training, agents can be guided by the same signals, such as the global state. However, agents lack the shared signal and choose actions given local observations during execution. Inspired by viewpoint invariance and contrastive learning, we propose consensus learning for cooperative multi-agent reinforcement learning in this study. Although based on local observations, different agents can infer the same consensus in discrete spaces without communication. We feed the inferred one-hot consensus to the network of agents as an explicit input in a decentralized way, thereby fostering their cooperative spirit. With minor model modifications, our suggested framework can be extended to a variety of multi-agent reinforcement learning algorithms. Moreover, we carry out these variants on some fully cooperative tasks and get convincing results.

Authors

• Zhiwei Xu Shandong University
• Bin Zhang Institute of Automation, CASIA & School of Artificial Intelligence, UCAS
• Dapeng Li The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences School of Artificial Intelligence, University of Chinese Academy of Sciences
• Zeren Zhang The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences School of Artificial Intelligence, University of Chinese Academy of Sciences
• Guangchong Zhou The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences School of Artificial Intelligence, University of Chinese Academy of Sciences
• Hao Chen The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, CASIA & School of Artificial Intelligence, UCAS
• Guoliang Fan Institute of Automation, CASIA & School of Artificial Intelligence, UCAS

Keywords

• MAS: Multiagent Learning