Chang Wang

EAAI Journal 2025 Journal Article

An invulnerable leader–follower collision-free unmanned aerial vehicle flocking system with attention-based Multi-Agent Reinforcement Learning

• Yunxiao Guo
• Dan Xu
• Chang Wang
• Jinxi Li
• Han Long

Deep reinforcement learning has been proved useful for the flocking control of Unmanned Aerial Vehicle (UAV) swarm with the leader–followers topology. However, it remains unclear how to fully utilize the spatial information among the followers to alleviate the problems of sparse reward and policy convergence. In this article, we propose a novel multi-agent reinforcement learning-based fixed-wing UAV flocking approach named Attention Based Cucker–Smale (ABCS) Flocking to learn collision-free leader–follower flocking by utilizing the information among followers with the attention mechanism. Specifically, we design an explainable Cucker–Smale criterion-based flocking reward named ABCS reward to associate the followers with high efficiency in flocking. Then, a leader-guide attention mechanism is proposed by transferring the difference between leader and follower as the weights to support the follower selectively utilizing the followers’ information. As a result, we prove that an optimal state can be achieved so that each follower can keep an optimal distance from the other followers when the ABCS reward is maximized. In addition, we prove the ABCS reward is bounded, which can be used to indicate learning convergence. To improve the invulnerability, we proposes a leader selection method based on ABCS flocking, which can effectively select a new leader when the old leader is destroyed. Finally, we demonstrate the effectiveness of ABCS Flocking over the Multi-Agent Deep Deterministic Policy Gradient approach using various reward functions with various numbers of followers in the obstacles and leader-destroyed scenarios. The code is published on github https: //github. com/YunxiaoGuo/ABCS-Flocking.

IROS Conference 2025 Conference Paper

Bridging the Reality Gap: Communication-Aware Task Allocation with Multi-Objective Asynchronous Policy Learning

• Zehao Xiong
• Yu Zhou
• Yexun Xi
• Yizhe Cao
• Chang Wang
• Jie Li

Distributed task allocation in the UAV swarm is sensitive to excessive communication overhead and frequent transmissions. Combining reinforcement learning and task allocation demonstrates great potential in enhancing algorithm performance and optimizing communication. However, existing studies rely on ideal communication assumptions and the nonphysical environment, making training and validation impractical in applying networked swarms. This paper proposes the Communication-Aware Task Allocation, which aims to train a gating mechanism policy to coordinate the transmission timing, improving robustness and timelessness of the task allocation. First, the policy learning problem is formalized as a POMDP, for which the channel access and other features are designed for observations, actions are inter-agent adaptive gating mechanisms, and the shared reward reflects global task conflicts. Second, to address the asynchronous learning under the CTDE, an asynchronous experience collection and splicing method is proposed to align trajectories. Then, the MOCPPO is proposed, which combines a primal-dual operator with proximal policy optimization, updating the optimal Lagrange multiplier and strategy parameters to simultaneously minimize task conflicts and communication overhead. Finally, sim-to-real experiments are conducted in the HIL environment, and results illustrate the best trade-off optimization of the proposed method over all state-of-the-art approaches.

EAAI Journal 2025 Journal Article

Driver's trust assessment based on situational awareness under human-machine collaboration driving

• Qinyu Sun
• Hang Zhou
• Rui Fu
• Yaning Xu
• Chang Wang
• Yingshi Guo

The human-machine co-driving can effectively take into account the superior characteristics of human driver and agent, and generate the ‘1 + 1>2’ collaboration mode. However, drivers with autonomous consciousness and different behaviors actively intervene in the control of vehicles, which will trigger the conflict between human and intelligent driving system. The human-machine mutual trust has gradually developed one of the key technologies to mitigate human-machine conflicts. In this study, from the perspective of machine trust towards human, a comprehensive trust evaluation model (CTEM) for intelligent systems towards driver intervention behavior was established. Based on the driver's situational awareness (SA) recovery process, the model focused on the hierarchical decision-making model from perception to cognition. A residual convolutional neural network based on attention mechanism was proposed to identify active intervention and accidental touches. Then the perceived trust assessment (PTA) model was established on the basis of visual geometry group16 (VGG16) network. For the cognitive trust assessment (CTA) model, the long short-term memory (LSTM) codec structure was employed to predict the vehicle trajectory, and the risk field was structured to quantify the risk value of the future trajectory. The CTEM model was constructed by integrating the PTA and CTA models, which serve as its foundational components. Finally, the driving simulator experiments was implement to verify the proposed model, and the results demonstrated that the CTEM could availably distinguish and assess the driver's SA ability. The construction of trust evaluation model will provide effective support for the improvement of human-machine co-driving safety.

IROS Conference 2025 Conference Paper

PI-WAN: A Physics-Informed Wind-Adaptive Network for Quadrotor Dynamics Prediction in Unknown Environments

• Mengyun Wang
• Bo Wang
• Yifeng Niu
• Chang Wang

Accurate dynamics modeling is essential for quadrotors to achieve precise trajectory tracking in various applications. Traditional physical knowledge-driven modeling methods face substantial limitations in unknown environments characterized by variable payloads, wind disturbances, and external perturbations. On the other hand, data-driven modeling methods suffer from poor generalization when handling outof-distribution (OoD) data, restricting their effectiveness in unknown scenarios. To address these challenges, we introduce the Physics-Informed Wind-Adaptive Network (PI-WAN), which combines knowledge-driven and data-driven modeling methods by embedding physical constraints directly into the training process for robust quadrotor dynamics learning. Specifically, PI-WAN employs a Temporal Convolutional Network (TCN) architecture that efficiently captures temporal dependencies from historical flight data, while a physics-informed loss function applies physical principles to improve model generalization and robustness across previously unseen conditions. By incorporating real-time prediction results into a model predictive control (MPC) framework, we achieve improvements in closed-loop tracking performance. Comprehensive simulations and real-world flight experiments demonstrate that our approach outperforms baseline methods in terms of prediction accuracy, tracking precision, and robustness to unknown environments.

IROS Conference 2025 Conference Paper

Reducing Scene Graph Generation Parameters Towards UAV Understanding of Structured Environments

• Xudong Li
• Chang Wang
• Yifeng Niu
• Man Yuan
• Lizhen Wu

Scene graph generation (SGG) is a structured approach to understanding real-world scenes with complex relations, which can enhance UAV autonomy in unfamiliar environments. However, SGG typically has numerous model parameters that require considerable computational resources. This paper proposes a refined SGG model and reduces the model parameters for its UAV applications. First, we use subject-object query pairs to predict triplets directly, eliminating the need for separate entity predictions. Additionally, the cross-attention mechanism enhances the model’s ability to query triplets. We use a single decoder to process subject and object entities simultaneously, enhancing computational speed and reducing the number of parameters. Then, we map the entities to the relational semantic space before performing relations classification, which improves the model performance by adding a small number of parameters. Finally, the set prediction loss function is designed for relation prediction to strengthen the role of relation prediction in triplets. Real-world UAV experiments show that our model can extract more triplets per second with fewer parameters than the benchmarks. Github: https://github.com/SupersPig/myLGTR.

IROS Conference 2024 Conference Paper

A Novel Variable Step-size Path Planning Framework with Step-Consistent Markov Decision Process For Large-Scale UAV Swarm

• Dan Xu
• Yunxiao Guo
• Han Long
• Chang Wang

In recent years, Deep Reinforcement Learning (DRL) has been a key approach to solving Unmanned Aerial Vehicle (UAV) swarm path planning problems. However, traditional DRL methods often face challenges in the initial learning stage and struggle to learn from variable step-size tasks. This paper introduces a novel training framework for large-scale UAV swarm variable step-size path planning: Rapidly-exploring Variable Step-size Deep Reinforcement Learning (RVSDRL). This framework involves common training on the ground local server and decentralized training on distributed UAVs. In the common training stage, we generate rapidly-exploring random graph samples to accelerate the common agent explore environment. In the decentralized training stages, we utilize the priority replay mechanism to improve efficiency. To enhance convergence stability, we restrict the returns of the equivalent paths and propose the Step-size Consistent Markov Decision Process (SCMDP) path planning model. Our method is compared with traditional methods, and the experiments demonstrate its superior performance in complex obstacle environments.

NeurIPS Conference 2024 Conference Paper

Contextual Decision-Making with Knapsacks Beyond the Worst Case

• Zhaohua Chen
• Rui Ai
• Mingwei Yang
• Yuqi Pan
• Chang Wang
• Xiaotie Deng

We study the framework of a dynamic decision-making scenario with resource constraints. In this framework, an agent, whose target is to maximize the total reward under the initial inventory, selects an action in each round upon observing a random request, leading to a reward and resource consumptions that are further associated with an unknown random external factor. While previous research has already established an $\widetilde{O}(\sqrt{T})$ worst-case regret for this problem, this work offers two results that go beyond the worst-case perspective: one for the worst-case gap between benchmarks and another for logarithmic regret rates. We first show that an $\Omega(\sqrt{T})$ distance between the commonly used fluid benchmark and the online optimum is unavoidable when the former has a degenerate optimal solution. On the algorithmic side, we merge the re-solving heuristic with distribution estimation skills and propose an algorithm that achieves an $\widetilde{O}(1)$ regret as long as the fluid LP has a unique and non-degenerate solution. Furthermore, we prove that our algorithm maintains a near-optimal $\widetilde{O}(\sqrt{T})$ regret even in the worst cases and extend these results to the setting where the request and external factor are continuous. Regarding information structure, our regret results are obtained under two feedback models, respectively, where the algorithm accesses the external factor at the end of each round and at the end of a round only when a non-null action is executed.

AAAI Conference 2024 Conference Paper

Dynamic Budget Throttling in Repeated Second-Price Auctions

• Zhaohua Chen
• Chang Wang
• Qian Wang
• Yuqi Pan
• Zhuming Shi
• Zheng Cai
• Yukun Ren
• Zhihua Zhu

In today's online advertising markets, a crucial requirement for an advertiser is to control her total expenditure within a time horizon under some budget. Among various budget control methods, throttling has emerged as a popular choice, managing an advertiser's total expenditure by selecting only a subset of auctions to participate in. This paper provides a theoretical panorama of a single advertiser's dynamic budget throttling process in repeated second-price auctions. We first establish a lower bound on the regret and an upper bound on the asymptotic competitive ratio for any throttling algorithm, respectively, when the advertiser's values are stochastic and adversarial. Regarding the algorithmic side, we propose the OGD-CB algorithm, which guarantees a near-optimal expected regret with stochastic values. On the other hand, when values are adversarial, we prove that this algorithm also reaches the upper bound on the asymptotic competitive ratio. We further compare throttling with pacing, another widely adopted budget control method, in repeated second-price auctions. In the stochastic case, we demonstrate that pacing is generally superior to throttling for the advertiser, supporting the well-known result that pacing is asymptotically optimal in this scenario. However, in the adversarial case, we give an exciting result indicating that throttling is also an asymptotically optimal dynamic bidding strategy. Our results bridge the gaps in theoretical research of throttling in repeated auctions and comprehensively reveal the ability of this popular budget-smoothing strategy.

YNIMG Journal 2021 Journal Article

Reliability of EEG microstate analysis at different electrode densities during propofol-induced transitions of brain states

• Kexu Zhang
• Wen Shi
• Chang Wang
• Yamin Li
• Zhian Liu
• Tun Liu
• Jing Li
• Xiangguo Yan

Electroencephalogram (EEG) microstate analysis is a promising and effective spatio-temporal method that can segment signals into several quasi-stable classes, providing a great opportunity to investigate short-range and long-range neural dynamics. However, there are still many controversies in terms of reproducibility and reliability when selecting different parameters or datatypes. In this study, five electrode configurations (91, 64, 32, 19, and 8 channels) were used to measure the reliability of microstate analysis at different electrode densities during propofol-induced sedation. First, the microstate topography and parameters at five different electrode densities were compared in the baseline (BS) condition and the moderate sedation (MD) condition, respectively. The intraclass correlation coefficient (ICC) and coefficient of variation (CV) were introduced to quantify the consistency of the microstate parameters. Second, statistical analysis and classification between BS and MD were performed to determine whether the microstate differences between different conditions remained stable at different electrode densities, and ICC was also calculated between the different conditions to measure the consistency of the results in a single condition. The results showed that in both the BS or MD condition, respectively, there were few significant differences in the microstate parameters among the 91-, 64-, and 32-channel configurations, with most of the differences observed between the 19- or 8-channel configurations and the other configurations. The ICC and CV data also showed that the consistency among the 91-, 64-, and 32-channel configurations was better than that among all five electrode configurations after including the 19- and 8-channel configurations. Furthermore, the significant differences between the conditions in the 91-channel configuration remained stable at the 64- and 32-channel resolutions, but disappeared at the 19- and 8-channel resolutions. In addition, the classification and ICC results showed that the microstate analysis became unreliable with fewer than 20 electrodes. The findings of this study support the hypothesis that microstate analysis of different brain states is more reliable with higher electrode densities; the use of a small number of channels is not recommended.

IJCAI Conference 2019 Conference Paper

Nostalgic Adam: Weighting More of the Past Gradients When Designing the Adaptive Learning Rate

• Haiwen Huang
• Chang Wang
• Bin Dong

First-order optimization algorithms have been proven prominent in deep learning. In particu- lar, algorithms such as RMSProp and Adam are extremely popular. However, recent works have pointed out the lack of “long-term memory” in Adam-like algorithms, which could hamper their performance and lead to divergence. In our study, we observe that there are benefits of weighting more of the past gradients when designing the adaptive learning rate. We therefore propose an algorithm called the Nostalgic Adam (NosAdam) with theoretically guaranteed convergence at the best known convergence rate. NosAdam can be regarded as a fix to the non-convergence issue of Adam in alternative to the recent work of [Reddi et al. , 2018]. Our preliminary numerical experiments show that NosAdam is a promising alternative al- gorithm to Adam. The proofs, code and other supplementary materials are already released.

IJCAI Conference 2016 Conference Paper

Building Joint Spaces for Relation Extraction

• Chang Wang
• LiangLiang Cao
• James Fan

In this paper, we present a novel approach for relation extraction using only term pairs as the input without textual features. We aim to build a single joint space for each relation which is then used to produce relation specific term embeddings. The proposed method fits particularly well for domains in which similar arguments are often associated with similar relations. It can also handle the situation when the labeled data is limited. The proposed method is evaluated both theoretically with a proof for the closed-form solution and experimentally with promising results on both DBpedia and medical relations.

IJCAI Conference 2015 Conference Paper

Medical Synonym Extraction with Concept Space Models

• Chang Wang
• LiangLiang Cao
• Bowen Zhou

In this paper, we present a novel approach for medical synonym extraction. We aim to integrate the term embedding with the medical domain knowledge for healthcare applications. One advantage of our method is that it is very scalable. Experiments on a dataset with more than 1M term pairs show that the proposed approach outperforms the baseline approaches by a large margin.

IJCAI Conference 2013 Conference Paper

Manifold Alignment Preserving Global Geometry

• Chang Wang
• Sridhar Mahadevan

This paper proposes a novel algorithm for manifold alignment preserving global geometry. This approach constructs mapping functions that project data instances from different input domains to a new lower-dimensional space, simultaneously matching the instances in correspondence and preserving global distances between instances within the original domains. In contrast to previous approaches, which are largely based on preserving local geometry, the proposed approach is suited to applications where the global manifold geometry needs to be respected. We evaluate the effectiveness of our algorithm for transfer learning in two real-world cross-lingual information retrieval tasks.

AAAI Conference 2013 Conference Paper

Multiscale Manifold Learning

• Chang Wang
• Sridhar Mahadevan

Many high-dimensional data sets that lie on a lowdimensional manifold exhibit nontrivial regularities at multiple scales. Most work in manifold learning ignores this multiscale structure. In this paper, we propose approaches to explore the deep structure of manifolds. The proposed approaches are based on the diffusion wavelets framework, data driven, and able to directly process directional neighborhood relationships without ad-hoc symmetrization. The proposed multiscale algorithms are evaluated using both synthetic and real-world data sets, and shown to outperform previous manifold learning methods.

IJCAI Conference 2011 Conference Paper

Heterogeneous Domain Adaptation Using Manifold Alignment

• Chang Wang
• Sridhar Mahadevan

We propose a manifold alignment based approach for heterogeneous domain adaptation. A key aspect of this approach is to construct mappings to link different feature spaces in order to transfer knowledge across domains. The new approach can reuse labeled data from multiple source domains in a target domain even in the case when the input domains do not share any common features or instances. As a pre-processing step, our approach can also be combined with existing domain adaptation approaches to learn a common feature space for all input domains. This paper extends existing manifold alignment approaches by making use of labels rather than correspondences to align the manifolds. This extension significantly broadens the application scope of manifold alignment, since the correspondence relationship required by existing alignment approaches is hard to obtain in many applications.

IJCAI Conference 2011 Conference Paper

Jointly Learning Data-Dependent Label and Locality-Preserving Projections

• Chang Wang
• Sridhar Mahadevan

This paper describes a novel framework to jointly learn data-dependent label and locality-preserving projections. Given a set of data instances from multiple classes, the proposed approach can automatically learn which classes are more similar to each other, and construct discriminative features using both labeled and unlabeled data to map similar classes to similar locations in a lower dimensional space. In contrast to linear discriminant analysis (LDA) and its variants, which can only return c-1 features for a problem with c classes, the proposed approach can generate d features, where d is bounded only by the number of the input features. We describe and evaluate the new approach both theoretically and experimentally, and compare its performance with other state of the art methods.

IJCAI Conference 2009 Conference Paper

• Chang Wang
• Sridhar Mahadevan

Manifold alignment has been found to be useful in many areas of machine learning and data mining. In this paper we introduce a novel manifold alignment approach, which differs from “semisupervised alignment” and “Procrustes alignment” in that it does not require predetermining correspondences. Our approach learns a projection that maps data instances (from two different spaces) to a lower dimensional space simultaneously matching the local geometry and preserving the neighborhood relationship within each set. This approach also builds connections between spaces de- fined by different features and makes direct knowledge transfer possible. The performance of our algorithm is demonstrated and validated in a series of carefully designed experiments in information retrieval and bioinformatics.

IJCAI Conference 2009 Conference Paper

• Chang Wang
• Sridhar Mahadevan

We introduce a nonparametric approach to multiscale analysis of document corpora using a hierarchical matrix analysis framework called diffusion wavelets. In contrast to eigenvector methods, diffusion wavelets construct multiscale basis functions. In this framework, a hierarchy is automatically constructed by an iterative series of dilation and orthogonalization steps beginning with an initial set of orthogonal basis functions, such as the unitvector bases. Each set of basis functions at a given level is constructed from the bases at the lower level by dilation using the dyadic powers of a diffusion operator. A novel aspect of our work is that the diffusion analysis is conducted on the space of variables (words), instead of instances (documents). This approach can automatically and efficiently determine the number of levels of the topical hierarchy, as well as the topics at each level. Multiscale analysis of document corpora is achieved by using the projections of the documents onto the spaces spanned by basis functions at different levels. Further, when the input term-term matrix is a “local” diffusion operator, the algorithm runs in time approximately linear in the number of non-zero elements of the matrix. The approach is illustrated on various data sets including NIPS conference papers, 20 Newsgroups and TDT2 data.