AAMAS Conference 2026 Conference Paper
Deep Meta Coordination Graphs for Multi-Agent Reinforcement Learning
- Nikunj Gupta
- James Zachary Hare
- Jesse Milzman
- Rajgopal Kannan
- Viktor Prasanna
This paper presents deep meta coordination graphs (DMCG) for learning cooperative policies in multi-agent reinforcement learning (MARL). Coordinationgraphformulationsencodelocalinteractions and accordingly factorize the joint value function of all agents to improve efficiency in MARL. DMCG learns a more expressive representation of agent interactions and use them to integrate agent information through graph convolutional networks. The goal is to enable an evolving coordination graph to guide effective coordination in cooperative MARL tasks. The graphs are jointly optimized withagents’valuefunctionstolearntoimplicitlyreasonaboutjoint actions, facilitating the end-to-end learning of interaction representations and coordinated policies. We demonstrate that DMCG consistently achieves state-of-the-art coordination performance and sample efficiency on challenging cooperative tasks, outperforming several prior graph-based and non-graph-based MARL baselines. Through several ablations, we also isolate the impact of individual components in DMCG, showing that the observed improvements are due to the meaningful design choices in this approach. All codes can be found here: https: //github. com/Nikunj-Gupta/dmcg-marl.