Author name cluster

Dapeng Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

AAAI Conference 2025 Conference Paper

Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation

Dapeng Li
Na Lou
Zhiwei Xu
Bin Zhang
Guoliang Fan

A key challenge in multi-agent collaborative tasks is reducing uncertainty about teammates to enhance cooperative performance. Explicit communication methods can reduce uncertainty about teammates, but the associated high communication costs limit their practicality. Alternatively, implicit consensus learning can promote cooperation without incurring communication costs. However, its performance declines significantly when local observations are severely limited. This paper introduces a novel multi-agent learning framework that combines the strengths of these methods. In our framework, agents generate a consensus about the group based on their local observations and then use both the consensus and local observations to produce messages. Since the consensus provides a certain level of global guidance, communication can be disabled when not essential, thereby reducing overhead. Meanwhile, communication can provide supplementary information to the consensus when necessary. Experimental results demonstrate that our algorithm significantly reduces inter-agent communication overhead while ensuring efficient collaboration.

PDF Details DOI

AAMAS Conference 2025 Conference Paper

Unveiling Decision Intention for Cooperative Multi-Agent Reinforcement Learning

Zeren Zhang
Zhiwei Xu
Guangchong Zhou
Dapeng Li
Bin Zhang
Guoliang Fan

For cooperative multi-agent reinforcement learning, various methods have been proposed to enhance the collaborative strategy capabilities of agents. However, when agents make decisions, humans have no knowledge of their subsequent decision-making intentions or sub-goals. This lack of understanding hinders human comprehension of agent strategies and further research on agents. Currently, there are limited relevant studies. To address this problem, we propose a novel framework which can generate the decision intention of agents. We first formalize this problem and use states crucial to the task to express the decision intentions of agents. Then, we introduce the polarization index to measure the importance of states and select them for training. Finally, we learn the decision intentions through a diffusion model with rapid generation capability and generate them during the decision-making process. This study sheds light on the problem of agent decision intention and enhances the transparency of agent strategies, facilitating deeper research on This work is licensed under a Creative Commons Attribution International 4. 0 License. Proc. of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025), Y. Vorobeychik, S. Das, A. Nowé (eds.), May 19 – 23, 2025, Detroit, Michigan, USA. © 2025 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org). agents. The experimental results demonstrate the effectiveness of our approach.

PDF

AAMAS Conference 2024 Conference Paper

From Explicit Communication to Tacit Cooperation: A Novel Paradigm for Cooperative MARL

Dapeng Li
Zhiwei Xu
Bin Zhang
Guangchong Zhou
Zeren Zhang
Guoliang Fan

Centralized training with decentralized execution (CTDE) is a widely used learning paradigm that has achieved significant success in complex tasks. Drawing inspiration from human team cooperative learning, we propose a novel paradigm that facilitates a gradual shift from explicit communication to tacit cooperation. In the initial training stage, we promote cooperation by sharing relevant information among agents and concurrently reconstructing this information using each agent’s local trajectory in a self-supervised way. We then combine the explicitly communicated information with the reconstructed information to obtain mixed information. Throughout the training process, we progressively decrease the proportion of explicitly communicated information, facilitating a seamless transition to fully decentralized execution without communication.

PDF

AAAI Conference 2023 Conference Paper

Consensus Learning for Cooperative Multi-Agent Reinforcement Learning

Zhiwei Xu
Bin Zhang
Dapeng Li
Zeren Zhang
Guangchong Zhou
Hao Chen
Guoliang Fan

Almost all multi-agent reinforcement learning algorithms without communication follow the principle of centralized training with decentralized execution. During the centralized training, agents can be guided by the same signals, such as the global state. However, agents lack the shared signal and choose actions given local observations during execution. Inspired by viewpoint invariance and contrastive learning, we propose consensus learning for cooperative multi-agent reinforcement learning in this study. Although based on local observations, different agents can infer the same consensus in discrete spaces without communication. We feed the inferred one-hot consensus to the network of agents as an explicit input in a decentralized way, thereby fostering their cooperative spirit. With minor model modifications, our suggested framework can be extended to a variety of multi-agent reinforcement learning algorithms. Moreover, we carry out these variants on some fully cooperative tasks and get convincing results.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative MARL

Zhiwei Xu
Bin Zhang
Dapeng Li
Guangchong Zhou
Zeren Zhang
Guoliang Fan

Value decomposition methods have gained popularity in the field of cooperative multi-agent reinforcement learning. However, almost all existing methods follow the principle of Individual Global Max (IGM) or its variants, which limits their problem-solving capabilities. To address this, we propose a dual self-awareness value decomposition framework, inspired by the notion of dual self-awareness in psychology, that entirely rejects the IGM premise. Each agent consists of an ego policy for action selection and an alter ego value function to solve the credit assignment problem. The value function factorization can ignore the IGM assumption by utilizing an explicit search procedure. On the basis of the above, we also suggest a novel anti-ego exploration mechanism to avoid the algorithm becoming stuck in a local optimum. As the first fully IGM-free value decomposition method, our proposed framework achieves desirable performance in various cooperative tasks.

PDF Details

AAAI Conference 2023 Conference Paper

HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

Zhiwei Xu
Yunpeng Bai
Bin Zhang
Dapeng Li
Guoliang Fan

Recently, some challenging tasks in multi-agent systems have been solved by some hierarchical reinforcement learning methods. Inspired by the intra-level and inter-level coordination in the human nervous system, we propose a novel value decomposition framework HAVEN based on hierarchical reinforcement learning for fully cooperative multi-agent problems. To address the instability arising from the concurrent optimization of policies between various levels and agents, we introduce the dual coordination mechanism of inter-level and inter-agent strategies by designing reward functions in a two-level hierarchy. HAVEN does not require domain knowledge and pre-training, and can be applied to any value decomposition variant. Our method achieves desirable results on different decentralized partially observable Markov decision process domains and outperforms other popular multi-agent hierarchical reinforcement learning algorithms.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

Inducing Stackelberg Equilibrium through Spatio-Temporal Sequential Decision-Making in Multi-Agent Reinforcement Learning

Bin Zhang
Lijuan Li
Zhiwei Xu
Dapeng Li
Guoliang Fan

In multi-agent reinforcement learning (MARL), self-interested agents attempt to establish equilibrium and achieve coordination depending on game structure. However, existing MARL approaches are mostly bound by the simultaneous actions of all agents in the Markov game (MG) framework, and few works consider the formation of equilibrium strategies via asynchronous action coordination. In view of the advantages of Stackelberg equilibrium (SE) over Nash equilibrium, we construct a spatio-temporal sequential decision-making structure derived from the MG and propose an N-level policy model based on a conditional hypernetwork shared by all agents. This approach allows for asymmetric training with symmetric execution, with each agent responding optimally conditioned on the decisions made by superior agents. Agents can learn heterogeneous SE policies while still maintaining parameter sharing, which leads to reduced cost for learning and storage and enhanced scalability as the number of agents increases. Experiments demonstrate that our method effectively converges to the SE policies in repeated matrix game scenarios, and performs admirably in immensely complex settings including cooperative tasks and mixed tasks.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learning

Zhiwei Xu
Dapeng Li
Bin Zhang
Yuan Zhan
Yunpeng Baiia
Guoliang Fan

Recently, model-based agents have achieved better performance than model-free ones using the same computational budget and training time in single-agent environments. However, due to the complexity of multi-agent systems, it is tough to learn the model of the environment. The significant compounding error may hinder the learning process when model-based methods are applied to multi-agent tasks. This paper proposes an implicit model-based multi-agent reinforcement learning method based on value decomposition methods. Under this method, agents can interact with the learned virtual environment and evaluate the current state value according to imagined future states in the latent space, making agents have the foresight. Our approach can be applied to any multi-agent value decomposition method. The experimental results show that our method improves the sample efficiency in different partially observable Markov decision process domains.

PDF Details

AAMAS Conference 2022 Conference Paper

SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning

Zhiwei Xu
Yunpeng Bai
Dapeng Li
Bin Zhang
Guoliang Fan

As one of the solutions to the decentralized partially observable Markov decision process (Dec-POMDP) problems, the value decomposition method has achieved significant results recently. However, most value decomposition methods require the fully observable state of the environment during training, but this is not feasible in some scenarios where only incomplete and noisy observations can be obtained. Therefore, we propose a novel value decomposition framework, named State Inference for value DEcomposition (SIDE), which eliminates the need to know the global state by simultaneously seeking solutions to the two problems of optimal control and state inference. SIDE can be extended to any value decomposition method to tackle partially observable problems. By comparing with the performance of different algorithms in Star- Craft II micromanagement tasks, we verified that though without accessible states, SIDE can infer the current state that contributes to the reinforcement learning process based on past local observations and even achieve superior results to many baselines in some complex scenarios.

PDF

JBHI Journal 2020 Journal Article

Detection and Monitoring of Thermal Lesions Induced by Microwave Ablation Using Ultrasound Imaging and Convolutional Neural Networks

Siyuan Zhang
Shan Wu
Shaoqiang Shang
Xuewei Qin
Xin Jia
Dapeng Li
Zhiwei Cui
Tianqi Xu

Microwave ablation (MWA) for cancer treatment is frequently monitored by ultrasound (US) B-mode imaging in the clinic, which often fails due to the low intrinsic contrast between the thermal lesion and normal tissue. Deep learning, especially convolutional neural network (CNN), has shown significant improvements in medical image analysis. Here, we propose and evaluate an US imaging based on a CNN architecture for the detection and monitoring of thermal lesions induced by MWA in porcine livers. Unlike dealing with images in many visual object recognition tasks, US radiofrequency (RF) data backscattered from the ablated region were utilized to capture features related to the thermal lesion. The dataset comprised of 1640 US RF envelope data matrices and their corresponding gross-pathology images, and were utilized for training and testing. After envelope detection, US B-mode, segmentation results based on CNN (SI CNN ), and modified CNN (SI m-CNN ) for US data were simultaneously reconstructed to reveal the suitability for monitoring of MWA. The SI CNN and SI m-CNN outperformed B-mode images for the detection and monitoring of MWA-induced thermal lesions. The values of the area under the receiver operating characteristic curve were 0. 8728 and 0. 8948 for the SI CNN and Si m-CNN, respectively, which were both higher than the value of 0. 6904 for B-mode images. Ablated regions that were assessed using SI m-CNN showed a good correlation (J 0. 8845, r 0. 8739, and E 0. 410) to gross-pathology images. This study was the first to illustrate that SI m-CNN has the potential to detect and monitor thermal lesions, and may be utilized as an alternative modality for image-guided MWA treatments.

Details DOI

NeurIPS Conference 2020 Conference Paper

f-Divergence Variational Inference

Neng Wan
Dapeng Li
Naira Hovakimyan

This paper introduces the f-divergence variational inference (f-VI) that generalizes variational inference to all f-divergences. Initiated from minimizing a crafty surrogate f-divergence that shares the statistical consistency with the f-divergence, the f-VI framework not only unifies a number of existing VI methods, e. g. Kullback–Leibler VI, Renyi's alpha-VI, and chi-VI, but offers a standardized toolkit for VI subject to arbitrary divergences from f-divergence family. A general f-variational bound is derived and provides a sandwich estimate of marginal likelihood (or evidence). The development of the f-VI unfolds with a stochastic optimization scheme that utilizes the reparameterization trick, importance weighting and Monte Carlo approximation; a mean-field approximation scheme that generalizes the well-known coordinate ascent variational inference (CAVI) is also proposed for f-VI. Empirical examples, including variational autoencoders and Bayesian neural networks, are provided to demonstrate the effectiveness and the wide applicability of f-VI.

PDF Details

SAT Conference 2004 Conference Paper

A SAT Based Scheduler for Tournament Schedules

Hantao Zhang 0001
Dapeng Li
Haiou Shen

We present a scheduler with a web interface for generating fair game schedules of a tournament. The tournament can be either single or double round-robin or something in between. The search engine inside the scheduler is a SAT solver which can handle a mix of ordinary and TL (True-Literal) clauses. The latter are the formulas using the function TL which counts the number of true literals in a clause. By using TL clauses, we could solve a typical scheduling problem in a few of seconds. If we convert them into ordinary clauses, the state-of-the-art SAT solvers could not solve them in one week. We showed how to integrate TL clauses into a SAT solver and take advantages of the advanced SAT techniques. Our scheduler provides a free service to all the people who are interested in fair sports schedules.

Details