A Formal Model for Multiagent Q-Learning Dynamics on Regular Graphs

Modeling the dynamics of multi-agent learning has long been an important research topic. The focus of previous research has been either on 2-agent settings or well-mixed infinitely large agent populations. In this paper, we consider the scenario where n Q-learning agents locate on regular graphs, such that agents can only interact with their neighbors. We examine the local interactions between individuals and their neighbors, and derive a formal model to capture the Q-value dynamics of the entire population. Through comparisons with agent-based simulations on different types of regular graphs, we show that our model describes the agent learning dynamics in an exact manner.

Authors

• Chen Chu Yunnan University of Finance and Economics
• Yong Li
• Jinzhuo Liu Yunnan University Northwestern Polytechnical University
• Shuyue Hu Shanghai Artificial Intelligence Laboratory
• Xuelong Li Institute of Artificial Intelligence (TeleAI), China Telecom
• Zhen Wang Northwestern Polytechnical University

Keywords

• Agent-based and Multi-agent Systems: Agent Societies
• Agent-based and Multi-agent Systems: Agent-Based Simulation and Emergence
• Agent-based and Multi-agent Systems: Multi-agent Learning