Learning Individual Difference Rewards in Multi-Agent Reinforcement Learning

We investigate explicit solutions to multi-agent credit assignment problem. Specifically, we assign each agent individual difference rewards in addition to the team reward as to distinguish the contribution of different agents to the team. We present a novel reward decomposition network to estimate the influence of each agent’s action on the team reward, and distribute difference rewards accordingly. Furthermore, we combine difference rewards with actor-critic framework and propose a new approach called learning individual difference rewards (LIDR). We evaluate LIDR on a set of StarCraft II micromanagement problems. Results show that LIDR significantly outperforms previous state-of-the-art methods.

Authors

• Chen Yang Shanghai Jiao Tong University
• Guangkai Yang University of Chinese Academy of Sciences & Institute of Automation, Chinese Academy of Sciences
• Junge Zhang Institute of Automation, Chinese academy of science School of Artificial Intelligence, University of Chinese Academy of Sciences

Keywords

• Multi-Agent Systems
• Credit Assignment
• Reward Shaping