Arrow Research search

Author name cluster

Shuai Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

81 papers
2 author rows

Possible papers

81

EAAI Journal 2026 Journal Article

A hierarchical domain adaptive explainable vulnerability detection scheme for power systems based on dynamic slicing enhancement

  • Fangfang Dang
  • Jiyu Zhang
  • Kehe Wu
  • Shuai Li
  • Ying Zhu

Automated vulnerability detection using graph neural networks has demonstrated significant potential, yet models trained on general-source software often exhibit high false-positive rates when applied to specialized power system code due to severe domain shift. To bridge this gap, this paper proposes a novel hierarchical domain adaptive and explainable vulnerability detection framework (HDAvul), specifically designed for power systems, whose core innovation lies in three key components. First, a dynamic slicing mechanism enhanced with power-dedicated semantic edges is introduced to adaptively capture the complete functional context of vulnerabilities, thereby overcoming the semantic fragmentation inherent in traditional fixed-window methods. Second, a hierarchical domain adaptation module is developed, employing multi-granularity adversarial learning to align feature distributions simultaneously at the Node, Slice, and Graph levels to ensure the preservation of intricate structural logic during knowledge transfer. Third, a transparent explanation module is integrated to automatically generate structural vulnerability-triggering paths (VTPs), providing traceable evidence for expert validation in safety-critical environments. Experimental results on real-world power system projects demonstrate that HDAvul achieves a peak F1-Score of 85. 2%, outperforming state-of-the-art domain-free and single-level adaptation baselines, while achieving a superior explanation stability of 89. 0% that confirms its capacity to provide reliable, high-fidelity analysis for securing mission-critical power infrastructure.

Details DOI

TCS Journal 2026 Journal Article

Bandit learning in matching markets with relative feedback

  • Fang Kong
  • Xiaoxi Zhang
  • Xiao Huang
  • Lijun Zhang
  • Shuai Li

The two-sided matching market problem has been extensively studied in the literature. How to find a stable matching is a key focus in the field. A significant body of recent work considers scenarios where one side of the market (players) has uncertain preferences and learns them through the absolute rewards obtained during repeated interactions with the other side (arms). A common assumption in these works is that arms deterministically resolve conflicts when faced with multiple players. However, in practical applications, the arms’ selection process may also be stochastic due to fluctuations in players’ performances. Under such circumstances, it becomes challenging for players to observe absolute rewards that quantify the arms’ satisfaction. Instead, the relative feedback about which applicant wins in a competition is often more realistic. In this paper, we investigate the pure exploration problem for bandit learning in matching markets where players need to additionally learn the uncertain preferences of arms based on more practical relative feedback. We show that given confidence level δ ∈ (0, 1), the market can reach the player-optimal stable matching in at most O ( max { N, K } log ( 1 / δ ) / Δ 2 + N K log ( 1 / δ ) / ϵ 2 ) rounds with probability at least 1 − δ, where N, K correspond to market size, ϵ represents arms’ relative preference gap, and Δ corresponds to the players’ preference gap. We also conduct experiments to verify the performances of the proposed algorithms.

Details DOI

TCS Journal 2026 Journal Article

Contextual combinatorial conservative bandits

  • Xiaojin Zhang
  • Weiwen Liu
  • Shuai Li
  • Shengyu Zhang

The problem of multi-armed bandits (MAB) asks to make sequential decisions while balancing between exploitation and exploration, and have been successfully applied to a wide range of practical scenarios. Various algorithms have been designed to achieve a high reward in a long term. However, its short-term performance might be rather low, which is injurious in risk sensitive applications. Building on previous work of conservative bandits, we bring up a framework of contextual combinatorial conservative bandits. An algorithm is presented and a regret bound of $\tilde O(d^2+d\sqrt{T})$ is proven, where $d$ is the dimension of the feature vectors, and $T$ is the total number of time steps. We further provide an algorithm as well as regret analysis for the case when the conservative reward is unknown. Experiments are conducted, and the results validate the effectiveness of our algorithm.

Details DOI

AAMAS Conference 2026 Conference Paper

Decentralized Asynchronous Multi-player Bandits

  • Jingqi Fan
  • Canzhe Zhao
  • Shuai Li
  • Siwei Wang

In recent years, multi-player multi-armed bandits (MP-MAB) have been extensively studied due to their wide applications in cognitive radio networks and Internet of Things systems. Most existing works focus on synchronized settings, whereas real-world systems are often decentralized and asynchronous, with players entering and leaving arbitrarily and no shared global clock. This introduces two major challenges: avoiding collisions without time coordination, and estimating the number of active players in every step. In this paper, we propose an algorithm to address these challenges. During exploration, players uniformly explore the arms that are not currently exploited by others, which reduces the probability of collisions and solves the first challenge. Meanwhile, players occasionally pull the arms that are currently exploited by others, enabling them to detect other players’ departures and addressing the second challenge. We prove that our algorithm achieves a regret of O( √︁ 𝑇 log𝑇 + log𝑇/Δ2), where Δ is the minimum expected reward gap between any two arms. To the best of our knowledge, this isthefirstefficientalgorithmintheasynchronousanddecentralized environment.

PDF

AAAI Conference 2026 Conference Paper

DECON: Reconstruction of Clothed-Geometric Multiple Humans from a Single Image via Geometry-Guided Decoupling

  • Yiming Jiang
  • Wenfeng Song
  • Shuai Li
  • Aimin Hao

3D multi-human reconstruction from single images holds significant potential for advancing AR/VR applications. While remarkable progress has been made in single-human reconstruction, existing methods face challenges when reconstructing multiple humans. These challenges include: (1) severe inter-occlusion that disrupts individual body structures, and (2) the absence of physically plausible relative positioning among subjects. We present DECON, a novel DEcouple-and-reCONstruct framework that systematically addresses these limitations through two technical innovations: (1) a decouple-and-reconstruct framework with multi-view synthesis. It separates individuals and reconstructs detailed 3D bodies from a single image. (2) a Perspective-Aware Position Optimization (PAPO) approach. It ensures realistic positioning by fixing overlaps and gaps between subjects. Extensive experiments demonstrate our method's capability to reconstruct fully separated, anatomically complete 3D humans with clothed-geometric details and plausible interactions. Quantitative evaluations show a 54% reduction in Chamfer Distance and 35% in Point-to-Surface Distance compared to state-of-the-art methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

FT-NCFM: An Influence-Aware Data Distillation Framework for Efficient VLA Models

  • Kewei Chen
  • Yayu Long
  • Shuai Li
  • Mingsheng Shang

The powerful generalization of Vision-Language-Action (VLA) models is bottlenecked by their heavy reliance on massive, redundant, and unevenly valued datasets, hindering their widespread application. Existing model-centric optimization paths, such as model compression (which often leads to performance degradation) or policy distillation (whose products are model-dependent and lack generality), fail to fundamentally address this data-level challenge. To this end, this paper introduces FT-NCFM, a fundamentally different, data-centric generative data distillation framework. Our framework employs a self-contained Fact-Tracing (FT) engine that combines causal attribution with programmatic contrastive verification to assess the intrinsic value of samples. Guided by these assessments, an adversarial NCFM process synthesizes a model-agnostic, information-dense, and reusable data asset. Experimental results on several mainstream VLA benchmarks show that models trained on just 5\% of our distilled coreset achieve a success rate of 85-90\% compared with training on the full dataset, while reducing training time by over 80\%. Our work demonstrates that intelligent data distillation is a highly promising new path for building efficient, high-performance VLA models.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Hierarchical Frequency-Guided Alignment Transformer for Compressed Video Quality Enhancement

  • Liuhan Peng
  • Shuai Li
  • Yanbo Gao
  • Mao Ye
  • Chong Lv

During the video encoding process, the original spatial domain signal is first transformed into the frequency domain, followed by quantization and compression. As a result, the quality degradation in compressed videos primarily stems from distortions in the frequency domain information. However, existing video enhancement methods typically directly fuse information from adjacent frames in the spatial domain, making it difficult for models to effectively compensate for frequency domain distortions, which leads to suboptimal detail restoration. To address this issue, we propose a Hierarchical Frequency-Guided Alignment Transformer. Additionally, by analyzing the characteristics of the frequency domain, we find that different frequency bands exhibit both correlations and a certain degree of independence. Based on this, we introduce a Frequency-Aware Transformer module that employs a combination of independent and mixed processing to optimize information exchange across different frequency domains, effectively mitigating cross-interference from irrelevant information. Experimental results demonstrate that, compared to existing methods, our approach achieves state-of-the-art performance in objective metrics (PSNR/SSIM), perceptual quality (LPIPS), and subjective visual effects, while reducing model complexity.

PDF Details DOI

AAAI Conference 2026 Conference Paper

IntentMotion: Learning Intent-Aware Human Motion from Language in 3D Scenes

  • Wenfeng Song
  • Shi Zheng
  • Xinyu Zhang
  • Xingliang Jin
  • Aimin Hao
  • Fei Hou
  • Xia Hou
  • Shuai Li

Generating human motion in complex 3D scenes from text is a challenging task with broad applications. However, existing methods often overlook realistic physical contact, resulting in visually plausible but physically unrealistic motion, e.g., penetration. To alleviate this, we propose IntentMotion, a novel framework that generates human motion in 3D scenes from natural language instructions by explicitly modeling intent. We first introduce the Intention-Guided Contact Field (IGCF). This differentiable voxel-based contact region representation explicitly aligns parsed language roles with spatial contact regions through a hierarchical attention mechanism. IGCF is jointly trained with a diffusion-based motion generator, allowing contact predictions to adapt dynamically through gradient feedback. To improve the controllability and physics-aware motion, we further propose an Intention-Aware Diffusion Model (IADM), which decouples the high-level semantic planning from the low-level contact refinement in a coarse-to-fine process. The optimized contact cues are utilized to guide the synthesis of a coarse trajectory, followed by refining detailed pose sequences under IGCF supervision. Experiments on the HUMANISE and LINGO datasets demonstrate that our IntentMotion outperforms recent baselines in contact accuracy, semantic alignment, and generalization to unseen scenes.

PDF Details DOI

EAAI Journal 2026 Journal Article

Weight prediction of the oxidation film in aircraft aluminium alloy components with small samples using data augmentation and random forest

  • Shuai Li
  • Zhuo Yu
  • Yudong Chen
  • Jiaqi Mai
  • Xiaofeng Zhou
  • Weichen Yu
  • Yigeng Wang

Anodic oxidation stands as one of the pivotal processes in the surface modification of aircraft aluminum alloy components. The weight of the oxidation film typically exerts an influence on the comprehensive performance of the components, which also significantly impacts the service life of diverse aircrafts. Nevertheless, the intricate coupling characteristics stemming from multiple process parameters and small sample sizes present formidable challenges to the weight prediction of the oxidation film. In response to these issues, this study develops a weight prediction method of oxidation film using data augmentation and random forest (RF). Initially, given the scarcity of oxidation film weight data, this study designs a data augmentation method using quadratic B-spline interpolation and generative adversarial network (GAN) to augment the quantity of data and enhance representational capabilities. Subsequently, to assess the quality of the augmented data, a comprehensive evaluation index (CEI) using mean squared error (MSE) and Kullback-Leibler (KL) divergence is presented. Finally, considering complex coupling characteristics of process parameters, a weight prediction model using attention mechanism (AM) and RF is built to enhance the prediction performance. The results of data augmentation and oxidation film weight prediction in the actual anodic oxidation process of aircraft aluminum alloy component demonstate the feasibility and effectiveness.

Details DOI

IJCAI Conference 2025 Conference Paper

A Survey on Bandit Learning in Matching Markets

  • Shuai Li
  • Zilong Wang
  • Fang Kong

The two-sided matching market problem has attracted extensive research in both computer science and economics due to its wide-ranging applications in multiple fields. In various online matching platforms, market participants often have unclear preferences. As a result, a growing area of research focuses on the online scenario. Here, one-side participants (players) gradually figure out their unknown preferences through multiple rounds of interactions with the other-side participants (arms). This survey comprehensively reviews and systematically organizes the abundant literature on bandit learning in matching markets. It covers not only existing theoretical achievements but also various other related aspects. Based on the current research, several distinct directions for future study have emerged. We are convinced that delving deeper into these directions could potentially yield theoretical algorithms that are more suitable for real-world situations.

PDF Details DOI

AAMAS Conference 2025 Conference Paper

Adaptive Offline Data Replay in Offline-to-Online Reinforcement Learning

  • Xu Liu
  • Tong Yu
  • Shuai Li

Offline-to-online reinforcement learning combines the advantages of offline data utilization with online exploration to enhance sample efficiency and performance. A primary challenge lies in managing the distribution shift between offline and online data, which significantly impacts training effectiveness. Existing methods often employ fixed mixing ratios for data replay, but these require taskspecific tuning and may fail to generalize across different environments. To address this, we introduce a metric that evaluates policy quality relative to offline and online data, and propose a banditbased strategy to adjust the mixing ratio adaptively, optimizing policy quality during training. Experiments across diverse environments demonstrate that our approach outperforms static methods, offering robust adaptability and minimizing manual tuning.

PDF

AAAI Conference 2025 Conference Paper

Conditional Diffusion Models Based Conditional Independence Testing

  • Yanfeng Yang
  • Shuai Li
  • Yingjie Zhang
  • Zhuoran Sun
  • Hai Shu
  • Ziqi Chen
  • Renming Zhang

Conditional independence (CI) testing is a fundamental task in modern statistics and machine learning. The conditional randomization test (CRT) was recently introduced to test whether two random variables, X and Y, are conditionally independent given a potentially high-dimensional set of random variables, Z. The CRT operates exceptionally well under the assumption that the conditional distribution X|Z is known. However, since this distribution is typically unknown in practice, accurately approximating it becomes crucial. In this paper, we propose using conditional diffusion models (CDMs) to learn the distribution of X|Z. Theoretically and empirically, it is shown that CDMs closely approximate the true conditional distribution. Furthermore, CDMs offer a more accurate approximation of X|Z compared to GANs, potentially leading to a CRT that performs better than those based on GANs. To accommodate complex dependency structures, we utilize a computationally efficient classifier-based conditional mutual information (CMI) estimator as our test statistic. The proposed testing procedure performs effectively without requiring assumptions about specific distribution forms or feature dependencies, and is capable of handling mixed-type conditioning sets that include both continuous and discrete variables. Theoretical analysis shows that our proposed test achieves a valid control of the type I error. A series of experiments on synthetic data demonstrates that our new test effectively controls both type-I and type-II errors, even in high dimensional scenarios.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Convex Potential Mirror Langevin Algorithm for Efficient Sampling of Energy-Based Models

  • Zitao Yang
  • Amin Ullah
  • Shuai Li
  • Fuxin Li
  • Jun Li

This paper introduces the Convex Potential Mirror Langevin Algorithm (CPMLA), a novel method to improve sampling efficiency for Energy-Based Models (EBMs). CPMLA uses mirror Langevin dynamics with a convex potential flow as a dynamic mirror map for EBM sampling. This dynamic mirror map enables targeted geometric exploration on the data manifold, accelerating convergence to the target distribution. Theoretical analysis proves that CPMLA achieves exponential convergence with vanishing bias under relaxed log-concave conditions, supporting its efficiency in adapting to complex data distributions. Experiments on benchmarks like CIFAR-10, SVHN, and CelebA demonstrate CPMLA's improved sampling quality and inference efficiency over existing techniques.

PDF Details

AAAI Conference 2025 Conference Paper

CtrlAvatar: Controllable Avatars Generation via Disentangled Invertible Networks

  • Wenfeng Song
  • Yang Ding
  • Fei Hou
  • Shuai Li
  • Aimin Hao
  • Xia Hou

As virtual experiences grow in popularity, the demand for realistic, personalized, and animatable human avatars increases. Traditional methods, relying on fixed templates, often produce costly avatars that lack expressiveness and realism. To overcome these challenges, we introduce Controllable Avatars generation via disentangled invertible networks (CtrlAvatar), a real-time framework for generating lifelike and customizable avatars. CtrlAvatar uses disentangled invertible networks to separate the deformation process into implicit body geometry and explicit texture components. This approach eliminates the need for repeated occupancy reconstruction, enabling detailed and coherent animations. The body geometry component ensures anatomical accuracy, while the texture component allows for complex, artifact-free clothing customization. This architecture ensures smooth integration between body movements and surface details. By optimizing transformations with position-varying offsets from the avatar’s initial Linear Blend Skinning vertices, CtrlAvatar achieves flexible, natural deformations that adapt to various scenarios. Extensive experiments show that CtrlAvatar outperforms other methods in quality, diversity, controllability, and cost-efficiency, marking a significant advancement in avatar generation.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

DNAEdit: Direct Noise Alignment for Text-Guided Rectified Flow Editing

  • Chenxi Xie
  • Minghan Li
  • Shuai Li
  • Yuhui Wu
  • Qiaosi Yi
  • Lei Zhang

Leveraging the powerful generation capability of large-scale pretrained text-to-image models, training-free methods have demonstrated impressive image editing results. Conventional diffusion-based methods, as well as recent rectified flow (RF)-based methods, typically reverse synthesis trajectories by gradually adding noise to clean images, during which the noisy latent at the current timestep is used to approximate that at the next timesteps, introducing accumulated drift and degrading reconstruction accuracy. Considering the fact that in RF the noisy latent is estimated through direct interpolation between Gaussian noises and clean images at each timestep, we propose Direct Noise Alignment (DNA), which directly refines the desired Gaussian noise in the noise domain, significantly reducing the error accumulation in previous methods. Specifically, DNA estimates the velocity field of the interpolated noised latent at each timestep and adjusts the Gaussian noise by computing the difference between the predicted and expected velocity field. We validate the effectiveness of DNA and reveal its relationship with existing RF-based inversion methods. Additionally, we introduce a Mobile Velocity Guidance (MVG) to control the target prompt-guided generation process, balancing image background preservation and target object editability. DNA and MVG collectively constitute our proposed method, namely DNAEdit. Finally, we introduce DNA-Bench, a long-prompt benchmark, to evaluate the performance of advanced image editing models. Experimental results demonstrate that our DNAEdit achieves superior performance to state-of-the-art text-guided editing methods. Our code, model, and benchmark will be made publicly available.

PDF Details

NeurIPS Conference 2025 Conference Paper

DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution

  • Rongyuan Wu
  • Lingchen Sun
  • Zhengqiang Zhang
  • Shihao Wang
  • Tianhe Wu
  • Qiaosi Yi
  • Shuai Li
  • Lei Zhang

Benefiting from pre-trained text-to-image (T2I) diffusion models, real-world image super-resolution (Real-ISR) methods can synthesize rich and realistic details. However, due to the inherent stochasticity of T2I models, different noise inputs often lead to outputs with varying perceptual quality. Although this randomness is sometimes seen as a limitation, it also introduces a wider perceptual quality range, which can be exploited to improve Real-ISR performance. To this end, we introduce Direct Perceptual Preference Optimization for Real-ISR (DP²O-SR), a framework that aligns generative models with perceptual preferences without requiring costly human annotations. We construct a hybrid reward signal by combining full-reference and no-reference image quality assessment (IQA) models trained on large-scale human preference datasets. This reward encourages both structural fidelity and natural appearance. To better utilize perceptual diversity, we move beyond the standard best-vs-worst selection and construct multiple preference pairs from outputs of the same model. Our analysis reveals that the optimal selection ratio depends on model capacity: smaller models benefit from broader coverage, while larger models respond better to stronger contrast in supervision. Furthermore, we propose hierarchical preference optimization, which adaptively weights training pairs based on intra-group reward gaps and inter-group diversity, enabling more efficient and stable learning. Extensive experiments across both diffusion- and flow-based T2I backbones demonstrate that DP²O-SR significantly improves perceptual quality and generalizes well to real-world benchmarks.

PDF Details

IJCAI Conference 2025 Conference Paper

Fast Second-Order Online Kernel Learning Through Incremental Matrix Sketching and Decomposition

  • Dongxie Wen
  • Xiao Zhang
  • Zhewei Wei
  • Chenping Hou
  • Shuai Li
  • Weinan Zhang

Second-order Online Kernel Learning (OKL) has attracted considerable research interest due to its promising predictive performance in streaming environments. However, existing second-order OKL approaches suffer from at least quadratic time complexity with respect to the pre-set budget, rendering them unsuitable for large-scale datasets. Moreover, the singular value decomposition required to obtain explicit feature mapping is computationally expensive due to the complete decomposition process. To address these issues, we propose FORKS, a fast incremental matrix sketching and decomposition approach tailored for second-order OKL. FORKS constructs an incremental maintenance paradigm for second-order kernelized gradient descent, which includes incremental matrix sketching for kernel approximation and incremental matrix decomposition for explicit feature mapping construction. Theoretical analysis demonstrates that FORKS achieves a logarithmic regret guarantee on par with other second-order approaches while maintaining a linear time complexity w. r. t. the budget, significantly enhancing efficiency over existing methods. We validate the performance of our method through extensive experiments conducted on real-world datasets, demonstrating its superior scalability and robustness against adversarial attacks.

PDF Details DOI

IROS Conference 2025 Conference Paper

Heterogeneous Mixed Traffic Control and Coordination

  • Iftekharul Islam
  • Weizi Li
  • Xuan Wang
  • Shuai Li
  • Kevin Heaslip

Urban intersections with diverse vehicle types, from small cars to large semi-trailers, pose significant challenges for traffic control. This study explores how robot vehicles (RVs) can enhance heterogeneous traffic flow, particularly at unsignalized intersections where traditional methods fail during power outages. Using reinforcement learning (RL) and real-world data, we simulate mixed traffic at complex intersections with RV penetration rates ranging from 10% to 90%. Results show that average waiting times drop by up to 86% and 91% compared to signalized and unsignalized intersections, respectively. We observe a "rarity advantage, " where less frequent vehicles benefit the most (up to 87%). Although CO 2 emissions and fuel consumption increase with RV penetration, they remain well below those of traditional signalized traffic. Decreased space headways also indicate more efficient road usage. These findings highlight RVs’ potential to improve traffic efficiency and reduce environmental impact in complex, heterogeneous settings.

Details

NeurIPS Conference 2025 Conference Paper

Improving Evolutionary Multi-View Classification via Eliminating Individual Fitness Bias

  • Xinyan Liang
  • Shuai Li
  • Qian Guo
  • Yuhua Qian
  • Bingbing Jiang
  • Tingjin Luo
  • Liang Du

Evolutionary multi-view classification (EMVC) methods have gained wide recognition due to their adaptive mechanisms. Fitness evaluation (FE), which aims to calculate the classification performance of each individual in the population and provide reliable performance ranking for subsequent operations, is a core step in such methods. Its accuracy directly determines the correctness of the evolutionary direction. That is, when FE fails to correctly reflect the superiority-inferiority relationship among individuals, it will lead to confusion in individual performance ranking, which in turn misleads the evolutionary direction and results in trapping into local optima. This paper is the first to identify the aforementioned issue in the field of EMVC and call it as fitness evaluation bias (FEB). FEB may be caused by a variety of factors, and this paper approaches the issue from the perspective of view information content: existing methods generally adopt joint training strategies, which restrict the exploration of key information in views with low information content. This makes it difficult for multi-view model (MVM) to achieve optimal performance during convergence, which in turn leads to FE failing to accurately reflect individual performance rankings and ultimately triggering FEB. To address this issue, we propose an evolutionary multi-view classification via eliminating individual fitness bias (EFB-EMVC) method, which alleviates the FEB issue by introducing evolutionary navigators for each MVM, thereby providing more accurate individual ranking. Experimental results fully verify the effectiveness of the proposed method in alleviating the FEB problem, and the EMVC method equipped with this strategy exhibits more superior performance compared with the original EMVC method. (The code is available at https: //github. com/LiShuailzn/Neurips-2025-EFB-EMVC)

PDF Details

NeurIPS Conference 2025 Conference Paper

Learning Preferences without Interaction for Cooperative AI: A Hybrid Offline-Online Approach

  • Haitong Ma
  • Haoran Yu
  • Haobo Fu
  • Shuai Li

Reinforcement learning (RL) for collaborative agents capable of cooperating with humans to accomplish tasks has long been a central goal in the RL community. While prior approaches have made progress in adapting collaborative agents to diverse human partners, they often focus solely on optimizing task performance and overlook human preferences—despite the fact that such preferences often diverge from the reward-maximization objective of the environment. Addressing this discrepancy poses significant challenges: humans typically provide only a small amount of offline, preference-related feedback and are unable to engage in online interactions, resulting in a distributional mismatch between the agent’s online learning process and the offline human data. To tackle this, we formulate the problem as an online&offline reinforcement learning problem that jointly integrates online generalization and offline preference learning, entirely under an offline training regime. We propose a simple yet effective training framework built upon existing RL algorithms that alternates between offline preference learning and online generalization recovery, ensuring the stability and alignment of both learning objectives. We evaluate our approach on a benchmark built upon the Overcooked environment—a standard environment for human-agent collaboration—and demonstrate remarkable performance across diverse preference styles and cooperative scenarios.

PDF Details

AAAI Conference 2025 Conference Paper

Logarithmic Regret for Linear Markov Decision Processes with Adversarial Corruptions

  • Canzhe Zhao
  • Xiangcheng Zhang
  • Baoxiang Wang
  • Shuai Li

In this work, we study the logarithmic regret for reinforcement learning (RL) with linear function approximation and adversarial corruptions, in the formulation of linear Markov decision processes (MDPs). Specifically, we consider the case where there exist adversarial corruptions over the reward functions, and the total amount of the corruptions of each step h across all episodes K is bounded by a corruption level C ≥ 0. We propose an algorithm, double-weighted least-squares value iteration with UCB (DW-LSVI-UCB), which leverages weighted linear regressions to learn the (corrupted) unknown reward parameters and unknown transition parameters simultaneously. We prove that DW-LSVI-UCB attains an O( d2H4 log2(1+K/δ) gapmin + CdH2) regret (omitting the dependence on lower order terms), where d is the ambient dimension of the feature mapping, H is the horizon length, gapmin is the minimal sub-optimality gap, and K is the number of episodes. Additionally, when there are no adversarial corruptions over reward functions, the regret of our algorithm improves the previous best result by an O(dH/ log K) factor.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Multi-player Multi-armed Bandits with Delayed Feedback

  • Jingqi Fan
  • Zilong Wang
  • Shuai Li
  • Linghe Kong

Multi-player multi-armed bandits (MP-MAB) have been extensively studied due to their application in cognitive radio networks. In this setting, multiple players simultaneously select arms and instantly receive feedback. However, in realistic decentralized networks, feedback is often delayed due to sensing latency and signal processing. Without a central coordinator, explicit communication is impossible, and delayed feedback disrupts implicit coordination, since it depends on synchronous observations. As a result, collisions are frequent and system performance degrades significantly. In this paper, we propose an algorithm in MP-MAB with stochastic delay feedback. Each player in the algorithm independently maintains an estimate of the optimal arm set based on their own delayed rewards but only pulls arms from the set, which is, with high probability, identical to those of other players, thus avoiding collisions. The identical arm set also enables implicit communication, allowing players to utilize the exploration results of others. We establish a regret upper bound and derive a lower bound to prove the algorithm is near-optimal. Numerical experiments on both synthetic and real-world datasets validate the effectiveness of our algorithm.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Online-to-Offline RL for Agent Alignment

  • Xu Liu
  • Haobo Fu
  • Stefano V. Albrecht
  • Qiang Fu
  • Shuai Li

Reinforcement learning (RL) has shown remarkable success in training agents to achieve high-performing policies, particularly in domains like Game AI where simulation environments enable efficient interactions. However, despite their success in maximizing these returns, such online-trained policies often fail to align with human preferences concerning actions, styles, and values. The challenge lies in efficiently adapting these online-trained policies to align with human preferences, given the scarcity and high cost of collecting human behavior data. In this work, we formalize the problem as *online-to-offline* RL and propose ALIGNment of Game AI to Preferences (ALIGN-GAP), an innovative approach for the alignment of well-trained game agents to human preferences. Our method features a carefully designed reward model that encodes human preferences from limited offline data and incorporates curriculum-based preference learning to align RL agents with targeted human preferences. Experiments across diverse environments and preference types demonstrate the performance of ALIGN-GAP, achieving effective alignment with human preferences.

Details

EAAI Journal 2025 Journal Article

Prescribed-time convergence noise-tolerant zeroing neural network for multi-robot position management and coordination

  • Tinglei Wang
  • Yufei Wang
  • Cheng Hua
  • Xinwei Cao
  • Bolin Liao
  • Shuai Li

A zeroing neural network (ZNN) model with prescribed-time convergence and noise tolerance is constructed for the first time to address multi-robot position planning (MPP) tasks. Unlike traditional ZNN models, which converge within finite, fixed, or predefined times, this model ensures that system errors converge precisely within the predefined time T p. Furthermore, it significantly enhances noise tolerance while maintaining excellent convergence performance. Through rigorous mathematical analysis and detailed numerical simulations, the proposed model demonstrates exceptional convergence and robustness, even under noise disturbances. In a noise-free environment, the convergence time accuracy (CTA) of this model when solving the MPP task reach over 90%, which is approximately 30% higher than traditional predefined-time convergence neural networks.

Details DOI

EAAI Journal 2025 Journal Article

Reinforcement learning based multi-perspective motion planning of manned electric vertical take-off and landing vehicle in urban environment with wind fields

  • Songyang Liu
  • Weizi Li
  • Haochen Li
  • Shuai Li

Electric vertical-takeoff and landing (eVTOL) aircraft, known for their maneuverability and flexibility, offer a promising alternative to traditional transportation systems. However, these aircraft face significant challenges from various perspectives, including the need to increase energy efficiency, enhance passenger experience, and mitigate noise impact on urban environments. While mathematical modeling-based approaches have been employed for flight motion planning, they often struggle to adapt to dynamic and complex environments. In this work, we introduce a three-dimensional motion planning method based on deep reinforcement learning (DRL), tailored for manned eVTOL flights through urban wind fields. Our approach considers three crucial aspects: aircraft energy consumption, passenger experience, and noise impact on urban environment. We modify the Proximal Policy Optimization (PPO) algorithm and design comprehensive reward function that considers these objectives. By incorporating energy efficiency, passenger experience, and noise impact into our reward function, our method demonstrates improved policy learning compared to existing approaches. Comparative experiments conducted under various wind conditions show that our method outperforms commonly used techniques, effectively optimizing multiple objectives in challenging urban environments. Code of our work are available at https: //github. com/cgchrfchscyrh/eVTOL_RL/tree/main.

Details DOI

ICML Conference 2025 Conference Paper

Rethinking the Temperature for Federated Heterogeneous Distillation

  • Fan Qi
  • Daxu Shi
  • Chuokun Xu
  • Shuai Li
  • Changsheng Xu

Federated Distillation (FedKD) relies on lightweight knowledge carriers like logits for efficient client-server communication. Although logit-based methods have demonstrated promise in addressing statistical and architectural heterogeneity in federated learning (FL), current approaches remain constrained by suboptimal temperature calibration during knowledge fusion. To address these limitations, we propose ReT-FHD, a framework featuring: 1) Multi-level Elastic Temperature, which dynamically adjusts distillation intensities across model layers, achieving optimized knowledge transfer between heterogeneous local models; 2) Category-Aware Global Temperature Scaling that implements class-specific temperature calibration based on confidence distributions in global logits, enabling personalized distillation policies; 3) Z-Score Guard, a blockchain-verified validation mechanism mitigating 44% of label-flipping and model poisoning attacks. Evaluations across diverse benchmarks with varying model/data heterogeneity demonstrate that the ReT-FHD achieves significant accuracy improvements over baseline methods while substantially reducing communication costs compared to existing approaches. Our work establishes that properly calibrated logits can serve as self-sufficient carriers for building scalable and secure heterogeneous FL systems.

Details

ICML Conference 2025 Conference Paper

Scaling Laws for Floating-Point Quantization Training

  • Xingwu Sun
  • Shuaipeng Li
  • Ruobing Xie
  • Weidong Han 0006
  • Kan Wu
  • Zhen Yang
  • Yixing Li
  • An Wang

Low-precision training is considered an effective strategy for reducing both training and downstream inference costs. Previous scaling laws for precision mainly focus on integer quantization, which pay less attention to the constituents in floating-point (FP) quantization, and thus cannot well fit the LLM losses in this scenario. In contrast, while FP quantization training is more commonly implemented in production, it’s research has been relatively superficial. In this paper, we thoroughly explore the effects of FP quantization targets, exponent bits, mantissa bits, and the calculation granularity of the scaling factor in FP quantization training performance of LLM models. In addition to an accurate FP quantization unified scaling law, we also provide valuable suggestions for the community: (1) Exponent bits contribute slightly more to the model performance than mantissa bits. We provide the optimal exponent-mantissa bit ratio for different bit numbers, which is available for future reference by hardware manufacturers; (2) We discover the formation of the critical data size in low-precision LLM training. Too much training data exceeding the critical data size will inversely bring in degradation of LLM performance; (3) The optimal FP quantization precision is directly proportional to the computational power, but within a wide computational power range. We estimate that the best cost-performance precision should lie between 4-8 bits.

Details

JBHI Journal 2025 Journal Article

Video Object Segmentation with Optimal Frame Auto-selection Based on Prior Knowledge for Midbrain Assessment in Transcranial Ultrasound

  • Xinyi Wang
  • Sai Kit LAM
  • Hongyu KANG
  • Yu Sun
  • Chao HOU
  • Shuai Li
  • Xin Sun
  • Fangxian LI

Transcranial sonography (TCS) provides a non-invasive means of assessing movement disorders such as Parkinson's disease (PD). However, current TCS-based evaluations rely heavily on manual operation by experienced physicians, making the process time-consuming and physician-dependent. For the first time, we aimed to develop a hybrid pipeline for real-time video object segmentation (VOS) and automatic optimal frame selection. Eighty-three standardized TCS real-time data comprising 1, 992 midbrain frames from Beijing Tiantan Hospital were collected. We adopted three state-of-the-art VOS models (STCN, RDE-VOS, and XMEM) and incorporated anatomical priors to guide optimal frame selection. Specifically, we leveraged the anatomical trend of midbrain morphology to estimate the midbrain radius at the optimal frame and selected the frame where the VOS-segmented midbrain best matched this estimate. The XMEM-based pipeline achieved high segmentation performance (Jaccard: 0. 85, Boundary Accuracy: 0. 95, Dice: 0. 92) and optimal frame selection (Distance: 4. 87; Jaccard: 0. 92), with efficiency (51. 05 FPS, 0. 56 s/patient, 661. 55 MB). Subgroup analyses confirmed robustness across image quality and PD conditions. Assessment of a junior physician's selection suggests potential to reduce the expertise gap in optimal frame selection. The proposed hybrid pipeline offers an automated tool for midbrain assessment using TCS, which may help reduce physicians' workload and minimize subjectivity, particularly supporting junior physicians in mitigating the expertise-demanding nature of TCS. This approach may serve as a foundation for more promising TCS-based assessments in the future, contributing to broader adoption of non-invasive ultrasound techniques in PD evaluation.

Details DOI

NeurIPS Conference 2024 Conference Paper

Calibrating Reasoning in Language Models with Internal Consistency

  • Zhihui Xie
  • Jizhou Guo
  • Tong Yu
  • Shuai Li

Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks, aided by techniques like chain-of-thought prompting that elicits verbalized reasoning. However, LLMs often generate text with obvious mistakes and contradictions, raising doubts about their ability to robustly process and utilize generated rationales. In this work, we investigate reasoning in LLMs through the lens of internal representations, focusing on how these representations are influenced by generated rationales. Our preliminary analysis reveals that while generated rationales improve answer accuracy, inconsistencies emerge between the model’s internal representations in middle layers and those in final layers, potentially undermining the reliability of their reasoning processes. To address this, we propose internal consistency as a measure of the model’s confidence by examining the agreement of latent predictions decoded from intermediate layers. Extensive empirical studies across different models and datasets demonstrate that internal consistency effectively distinguishes between correct and incorrect reasoning paths. Motivated by this, we propose a new approach to calibrate reasoning by up-weighting reasoning paths with high internal consistency, resulting in a significant boost in reasoning performance. Further analysis uncovers distinct patterns in attention and feed-forward modules across layers, providing insights into the emergence of internal inconsistency. In summary, our results demonstrate the potential of using internal representations for self-evaluation of LLMs.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Decoupling Degradations with Recurrent Network for Video Restoration in Under-Display Camera

  • Chengxu Liu
  • Xuan Wang
  • Yuanting Fan
  • Shuai Li
  • Xueming Qian

Under-display camera (UDC) systems are the foundation of full-screen display devices in which the lens mounts under the display. The pixel array of light-emitting diodes used for display diffracts and attenuates incident light, causing various degradations as the light intensity changes. Unlike general video restoration which recovers video by treating different degradation factors equally, video restoration for UDC systems is more challenging that concerns removing diverse degradation over time while preserving temporal consistency. In this paper, we introduce a novel video restoration network, called D2RNet, specifically designed for UDC systems. It employs a set of Decoupling Attention Modules (DAM) that effectively separate the various video degradation factors. More specifically, a soft mask generation function is proposed to formulate each frame into flare and haze based on the diffraction arising from incident light of different intensities, followed by the proposed flare and haze removal components that leverage long- and short-term feature learning to handle the respective degradations. Such a design offers an targeted and effective solution to eliminating various types of degradation in UDC systems. We further extend our design into multi-scale to overcome the scale-changing of degradation that often occur in long-range videos. To demonstrate the superiority of D2RNet, we propose a large-scale UDC video benchmark by gathering HDR videos and generating realistically degraded videos using the point spread function measured by a commercial UDC system. Extensive quantitative and qualitative evaluations demonstrate the superiority of D2RNet compared to other state-of-the-art video restoration and UDC image restoration methods.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Few-Shot Diffusion Models Escape the Curse of Dimensionality

  • Ruofeng Yang
  • Bo Jiang
  • Cheng Chen
  • Ruinan Jin
  • Baoxiang Wang
  • Shuai Li

While diffusion models have demonstrated impressive performance, there is a growing need for generating samples tailored to specific user-defined concepts. The customized requirements promote the development of few-shot diffusion models, which use limited $n_{ta}$ target samples to fine-tune a pre-trained diffusion model trained on $n_s$ source samples. Despite the empirical success, no theoretical work specifically analyzes few-shot diffusion models. Moreover, the existing results for diffusion models without a fine-tuning phase can not explain why few-shot models generate great samples due to the curse of dimensionality. In this work, we analyze few-shot diffusion models under a linear structure distribution with a latent dimension $d$. From the approximation perspective, we prove that few-shot models have a $\widetilde{O}(n_s^{-2/d}+n_{ta}^{-1/2})$ bound to approximate the target score function, which is better than $n_{ta}^{-2/d}$ results. From the optimization perspective, we consider a latent Gaussian special case and prove that the optimization problem has a closed-form minimizer. This means few-shot models can directly obtain an approximated minimizer without a complex optimization process. Furthermore, we also provide the accuracy bound $\widetilde{O}(1/n_{ta}+1/\sqrt{n_s})$ for the empirical solution, which still has better dependence on $n_{ta}$ compared to $n_s$. The results of the real-world experiments also show that the models obtained by only fine-tuning the encoder and decoder specific to the target distribution can produce novel images with the target feature, which supports our theoretical results.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Improved Analysis for Bandit Learning in Matching Markets

  • Fang Kong
  • Zilong Wang
  • Shuai Li

A rich line of works study the bandit learning problem in two-sided matching markets, where one side of market participants (players) are uncertain about their preferences and hope to find a stable matching during iterative matchings with the other side (arms). The state-of-the-art analysis shows that the player-optimal stable regret is of order $O(K\log T/\Delta^2)$ where $K$ is the number of arms, $T$ is the horizon and $\Delta$ is the players' minimum preference gap. However, this result may be far from the lower bound $\Omega(\max\{N\log T/\Delta^2, K\log T/\Delta\})$ since the number $K$ of arms (workers, publisher slots) may be much larger than that $N$ of players (employers in labor markets, advertisers in online advertising, respectively). In this paper, we propose a new algorithm and show that the regret can be upper bounded by $O(N^2\log T/\Delta^2 + K \log T/\Delta)$. This result removes the dependence on $K$ in the main order term and improves the state-of-the-art guarantee in common cases where $N$ is much smaller than $K$. Such an advantage is also verified in experiments. In addition, we provide a refined analysis for the existing centralized UCB algorithm and show that, under $\alpha$-condition, it achieves an improved $O(N \log T/\Delta^2 + K \log T / \Delta)$ regret.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Improved Bandits in Many-to-One Matching Markets with Incentive Compatibility

  • Fang Kong
  • Shuai Li

Two-sided matching markets have been widely studied in the literature due to their rich applications. Since participants are usually uncertain about their preferences, online algorithms have recently been adopted to learn them through iterative interactions. An existing work initiates the study of this problem in a many-to-one setting with responsiveness. However, their results are far from optimal and lack guarantees of incentive compatibility. We first extend an existing algorithm for the one-to-one setting to this more general setting and show it achieves a near-optimal bound for player-optimal regret. Nevertheless, due to the substantial requirement for collaboration, a single player's deviation could lead to a huge increase in its own cumulative rewards and a linear regret for others. In this paper, we aim to enhance the regret bound in many-to-one markets while ensuring incentive compatibility. We first propose the adaptively explore-then-deferred-acceptance (AETDA) algorithm for responsiveness setting and derive an upper bound for player-optimal stable regret while demonstrating its guarantee of incentive compatibility. This result is a significant improvement over existing works. And to the best of our knowledge, it constitutes the first player-optimal guarantee in matching markets that offers such robust assurances. We also consider broader substitutable preferences, one of the most general conditions to ensure the existence of a stable matching and cover responsiveness. We devise an online DA (ODA) algorithm and establish an upper bound for the player-pessimal stable regret for this setting.

PDF Details DOI

TMLR Journal 2024 Journal Article

Improved Regret Bounds for Linear Adversarial MDPs via Linear Optimization

  • Fang Kong
  • Xiangcheng Zhang
  • Baoxiang Wang
  • Shuai Li

Learning Markov decision processes (MDP) in an adversarial environment has been a challenging problem. The problem becomes even more challenging with function approximation since the underlying structure of the loss function and transition kernel are especially hard to estimate in a varying environment. In fact, the state-of-the-art results for linear adversarial MDP achieve a regret of $\tilde{\mathcal{O}}({K^{6/7}})$ ($K$ denotes the number of episodes), which admits a large room for improvement. In this paper, we propose a novel explore-exploit algorithm framework and investigate the problem with a new view, which reduces linear MDP into linear optimization by subtly setting the feature maps of the bandit arms of linear optimization. This new technique, under an exploratory assumption, yields an improved bound of $\tilde{\mathcal{O}}({K^{4/5}})$ for linear adversarial MDP without access to a transition simulator. The new view could be of independent interest for solving other MDP problems that possess a linear structure.

PDF Details

NeurIPS Conference 2024 Conference Paper

Learning Versatile Skills with Curriculum Masking

  • Yao Tang
  • Zhihui Xie
  • Zichuan Lin
  • Deheng Ye
  • Shuai Li

Masked prediction has emerged as a promising pretraining paradigm in offline reinforcement learning (RL) due to its versatile masking schemes, enabling flexible inference across various downstream tasks with a unified model. Despite the versatility of masked prediction, it remains unclear how to balance the learning of skills at different levels of complexity. To address this, we propose CurrMask, a curriculum masking pretraining paradigm for sequential decision making. Motivated by how humans learn by organizing knowledge in a curriculum, CurrMask adjusts its masking scheme during pretraining for learning versatile skills. Through extensive experiments, we show that CurrMask exhibits superior zero-shot performance on skill prompting tasks, goal-conditioned planning tasks, and competitive finetuning performance on offline RL tasks. Additionally, our analysis of training dynamics reveals that CurrMask gradually acquires skills of varying complexity by dynamically adjusting its masking scheme.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Leveraging Drift to Improve Sample Complexity of Variance Exploding Diffusion Models

  • Ruofeng Yang
  • Zhijie Wang
  • Bo Jiang
  • Shuai Li

Variance exploding (VE) based diffusion models, an important class of diffusion models, have shown state-of-the-art (SOTA) performance. However, only a few theoretical works analyze VE-based models, and those works suffer from a worse forward convergence rate $1/\text{poly}(T)$ than the $\exp{(-T)}$ of variance preserving (VP) based models, where $T$ is the forward diffusion time and the rate measures the distance between forward marginal distribution $q_T$ and pure Gaussian noise. The slow rate is due to the Brownian Motion without a drift term. In this work, we design a new drifted VESDE forward process, which allows a faster $\exp{(-T)}$ forward convergence rate. With this process, we achieve the first efficient polynomial sample complexity for a series of VE-based models with reverse SDE under the manifold hypothesis. Furthermore, unlike previous works, we allow the diffusion coefficient to be unbounded instead of a constant, which is closer to the SOTA models. Besides the reverse SDE, the other common reverse process is the probability flow ODE (PFODE) process, which is deterministic and enjoys faster sample speed. To deepen the understanding of VE-based models, we consider a more general setting considering reverse SDE and PFODE simultaneously, propose a unified tangent-based analysis framework, and prove the first quantitative convergence guarantee for SOTA VE-based models with reverse PFODE. We also show that the drifted VESDE can balance different error terms and improve generated samples without training through synthetic and real-world experiments.

PDF Details DOI

TCS Journal 2024 Journal Article

Optimal analysis for bandit learning in matching markets with serial dictatorship

  • Zilong Wang
  • Shuai Li

The problem of two-sided matching markets is well-studied in computer science and economics, owing to its diverse applications across numerous domains. Since market participants are usually uncertain about their preferences in various online matching platforms, an emerging line of research is dedicated to the online setting where one-side participants (players) learn their unknown preferences through multiple rounds of interactions with the other side (arms). Sankararaman et al. [23] provide an Ω ( N log ⁡ ( T ) Δ 2 + K log ⁡ ( T ) Δ ) regret lower bound for this problem under serial dictatorship assumption, where N is the number of players, K ( ≥ N ) is the number of arms, Δ is the minimum reward gap across players and arms, and T is the time horizon. Serial dictatorship assumes arms have the same preferences, which is common in reality when one side participants have a unified evaluation standard. Recently, the work of Kong and Li [10] proposes the ET-GS algorithm and achieves an O ( K log ⁡ ( T ) Δ 2 ) regret upper bound, which is the best upper bound attained so far. Nonetheless, a gap between the lower and upper bounds, ranging from N to K, persists. It remains unclear whether the lower bound or the upper bound needs to be improved. In this paper, we propose a multi-level successive selection algorithm that obtains an O ( N log ⁡ ( T ) Δ 2 + K log ⁡ ( T ) Δ ) regret bound when the market satisfies serial dictatorship. To the best of our knowledge, we are the first to propose an algorithm that matches the lower bound in the problem of matching markets with bandits.

Details DOI

NeurIPS Conference 2024 Conference Paper

The Closeness of In-Context Learning and Weight Shifting for Softmax Regression

  • Shuai Li
  • Zhao Song
  • Yu Xia
  • Tong Yu
  • Tianyi Zhou

Large language models (LLMs) are known for their exceptional performance in natural language processing, making them highly effective in many human life-related tasks. The attention mechanism in the Transformer architecture is a critical component of LLMs, as it allows the model to selectively focus on specific input parts. The softmax unit, which is a key part of the attention mechanism, normalizes the attention scores. Hence, the performance of LLMs in various NLP tasks depends significantly on the crucial role played by the attention mechanism with the softmax unit. In-context learning is one of the celebrated abilities of recent LLMs. Without further parameter updates, Transformers can learn to predict based on few in-context examples. However, the reason why Transformers becomes in-context learners is not well understood. Recently, in-context learning has been studied from a mathematical perspective with simplified linear self-attention without softmax unit. Based on a linear regression formulation $\min_x\| Ax - b \|_2$, existing works show linear Transformers' capability of learning linear functions in context. The capability of Transformers with softmax unit approaching full Transformers, however, remains unexplored. In this work, we study the in-context learning based on a softmax regression formulation $\min_{x} \| \langle \exp(Ax), {\bf 1}_n \rangle^{-1} \exp(Ax) - b \|_2$. We show the upper bounds of the data transformations induced by a single self-attention layer with softmax unit and by gradient-descent on a $\ell_2$ regression loss for softmax prediction function. Our theoretical results imply that when training self-attention-only Transformers for fundamental regression tasks, the models learned by gradient-descent and Transformers show great similarity.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Adversarial Attacks on Online Learning to Rank with Click Feedback

  • Jinhang Zuo
  • Zhiyao Zhang
  • Zhiyong Wang
  • Shuai Li
  • Mohammad Hajiesmaili
  • Adam Wierman

Online learning to rank (OLTR) is a sequential decision-making problem where a learning agent selects an ordered list of items and receives feedback through user clicks. Although potential attacks against OLTR algorithms may cause serious losses in real-world applications, there is limited knowledge about adversarial attacks on OLTR. This paper studies attack strategies against multiple variants of OLTR. Our first result provides an attack strategy against the UCB algorithm on classical stochastic bandits with binary feedback, which solves the key issues caused by bounded and discrete feedback that previous works cannot handle. Building on this result, we design attack algorithms against UCB-based OLTR algorithms in position-based and cascade models. Finally, we propose a general attack strategy against any algorithm under the general click model. Each attack algorithm manipulates the learning agent into choosing the target attack item $T-o(T)$ times, incurring a cumulative cost of $o(T)$. Experiments on synthetic and real data further validate the effectiveness of our proposed attack algorithms.

PDF Details

AAMAS Conference 2023 Conference Paper

CraftEnv: A Flexible Collective Robotic Construction Environment for Multi-Agent Reinforcement Learning

  • Rui Zhao
  • Xu Liu
  • Yizheng Zhang
  • Minghao Li
  • Cheng Zhou
  • Shuai Li
  • Lei Han

CraftEnv is a flexible Collective Robotic Construction (CRC) environment for Multi-Agent Reinforcement Learning (MARL) research. CraftEnv can be used to study how artificial intelligent agents may learn to cooperate and solve complex real world tasks, such as collective construction and intelligent warehousing. The environment contains a set of collective construction tasks, which require a group of robotic vehicles to cooperate and learn to build different constructions efficiently. There are different elements in the CraftEnv, such as smartcars, blocks, and slopes. The smartcars can use the blocks and slopes to build different structures. The CraftEnv is highly flexible and simple to use, which enables creative and quick task-designs. The environment is written in python and can be rendered using PyBullet. The simulation is built based on real world robotic systems, designed with real-world constraints in mind. The learned policy can be transferred to the real world robotic system. CraftEnv is tailored for effective use by the research community and pushing forward collective intelligence and swarm technology.

PDF

IJCAI Conference 2023 Conference Paper

DPMAC: Differentially Private Communication for Cooperative Multi-Agent Reinforcement Learning

  • Canzhe Zhao
  • Yanjie Ze
  • Jing Dong
  • Baoxiang Wang
  • Shuai Li

Communication lays the foundation for cooperation in human society and in multi-agent reinforcement learning (MARL). Humans also desire to maintain their privacy when communicating with others, yet such privacy concern has not been considered in existing works in MARL. We propose the differentially private multi-agent communication (DPMAC) algorithm, which protects the sensitive information of individual agents by equipping each agent with a local message sender with rigorous (epsilon, delta)-differential privacy (DP) guarantee. In contrast to directly perturbing the messages with predefined DP noise as commonly done in privacy-preserving scenarios, we adopt a stochastic message sender for each agent respectively and incorporate the DP requirement into the sender, which automatically adjusts the learned message distribution to alleviate the instability caused by DP noise. Further, we prove the existence of a Nash equilibrium in cooperative MARL with privacy-preserving communication, which suggests that this problem is game-theoretically learnable. Extensive experiments demonstrate a clear advantage of DPMAC over baseline methods in privacy-preserving scenarios.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Efficient Explorative Key-Term Selection Strategies for Conversational Contextual Bandits

  • Zhiyong Wang
  • Xutong Liu
  • Shuai Li
  • John C. S. Lui

Conversational contextual bandits elicit user preferences by occasionally querying for explicit feedback on key-terms to accelerate learning. However, there are aspects of existing approaches which limit their performance. First, information gained from key-term-level conversations and arm-level recommendations is not appropriately incorporated to speed up learning. Second, it is important to ask explorative key-terms to quickly elicit the user's potential interests in various domains to accelerate the convergence of user preference estimation, which has never been considered in existing works. To tackle these issues, we first propose ``ConLinUCB", a general framework for conversational bandits with better information incorporation, combining arm-level and key-term-level feedback to estimate user preference in one step at each time. Based on this framework, we further design two bandit algorithms with explorative key-term selection strategies, ConLinUCB-BS and ConLinUCB-MCR. We prove tighter regret upper bounds of our proposed algorithms. Particularly, ConLinUCB-BS achieves a better regret bound than the previous result. Extensive experiments on synthetic and real-world data show significant advantages of our algorithms in learning accuracy (up to 54% improvement) and computational efficiency (up to 72% improvement), compared to the classic ConUCB algorithm, showing the potential benefit to recommender systems.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Few-Shot Composition Learning for Image Retrieval with Prompt Tuning

  • Junda Wu
  • Rui Wang
  • Handong Zhao
  • Ruiyi Zhang
  • Chaochao Lu
  • Shuai Li
  • Ricardo Henao

We study the problem of composition learning for image retrieval, for which we learn to retrieve target images with search queries in the form of a composition of a reference image and a modification text that describes desired modifications of the image. Existing models of composition learning for image retrieval are generally built with large-scale datasets, demanding extensive training samples, i.e., query-target pairs, as supervision, which restricts their application for the scenario of few-shot learning with only few query-target pairs available. Recently, prompt tuning with frozen pretrained language models has shown remarkable performance when the amount of training data is limited. Inspired by this, we propose a prompt tuning mechanism with the pretrained CLIP model for the task of few-shot composition learning for image retrieval. Specifically, we regard the representation of the reference image as a trainable visual prompt, prefixed to the embedding of the text sequence. One challenge is to efficiently train visual prompt with few-shot samples. To deal with this issue, we further propose a self-upervised auxiliary task via ensuring that the reference image can retrieve itself when no modification information is given from the text, which facilitates training for the visual prompt, while not requiring additional annotations for query-target pairs. Experiments on multiple benchmarks show that our proposed model can yield superior performance when trained with only few query-target pairs.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding

  • Junda Wu
  • Tong Yu
  • Rui Wang
  • Zhao Song
  • Ruiyi Zhang
  • Handong Zhao
  • Chaochao Lu
  • Shuai Li

Soft prompt tuning achieves superior performances across a wide range of few-shot tasks. However, the performances of prompt tuning can be highly sensitive to the initialization of the prompts. We have also empirically observed that conventional prompt tuning methods cannot encode and learn sufficient task-relevant information from prompt tokens. In this work, we develop an information-theoretic framework that formulates soft prompt tuning as maximizing the mutual information between prompts and other model parameters (or encoded representations). This novel view helps us to develop a more efficient, accurate and robust soft prompt tuning method, InfoPrompt. With this framework, we develop two novel mutual information based loss functions, to (i) explore proper prompt initialization for the downstream tasks and learn sufficient task-relevant information from prompt tokens and (ii) encourage the output representation from the pretrained language model to be more aware of the task-relevant information captured in the learnt prompts. Extensive experiments validate that InfoPrompt can significantly accelerate the convergence of the prompt tuning and outperform traditional prompt tuning methods. Finally, we provide a formal theoretical result to show that a gradient descent type algorithm can be used to train our mutual information loss.

PDF Details

EAAI Journal 2023 Journal Article

Interpretable knowledge-guided framework for modeling minimum miscible pressure of CO2-oil system in CO2-EOR projects

  • Bin Shen
  • Shenglai Yang
  • Xinyuan Gao
  • Shuai Li
  • Kun Yang
  • Jiangtao Hu
  • Hao Chen

Carbon dioxide enhanced oil recovery (CO2-EOR) is a promising application for carbon capture, utilization and storage (CCUS). Accurate modeling of CO2-oil minimum miscible pressure (MMP) is crucial for CO2-EOR projects. In this study, a knowledge-guided framework for an extreme gradient boosting machine (XGBoost) and interpretable tabular learning architecture (TabNet), called KXGB and KTabNet, respectively, are developed to model the MMP. The proposed models are strengthened using a large MMP database of 421 samples collected from literature. Domain knowledge is integrated into intelligent models to prevent data-driven models from producing predictions that violated the domain knowledge. The Shapley Additive Explanations (SHAP) method is used to explain the proposed model to ensure the credibility of petroleum engineers. To further verify the model’s effectiveness, the same experimental strategy is employed to compare the proposed models with existing machine-learning (ML) methods. The results show that KXGB is the most recommended superior solution for modeling MMP owing to its outstanding performance and simplicity of optimization. The correlation coefficient, root mean square error, and mean absolute error are 0. 9833, 0. 7637 and 0. 55, respectively. However, KTabNet has great potential. Although its accuracy is slightly lower than that of the former, its strong representation ability and decision transparency may be favorable for future research. This study also demonstrate that the proposed framework conforms to certain theoretical rules and has a reasonable domain of applicability. To the best of our knowledge, this is the first study on integration of domain knowledge into MMP modeling methods. The experience and insights obtained from this study can guide CO2-EOR projects and other tabular data modeling in the oil and gas industry.

Details DOI

NeurIPS Conference 2023 Conference Paper

K-Nearest-Neighbor Local Sampling Based Conditional Independence Testing

  • Shuai Li
  • Yingjie Zhang
  • Hongtu Zhu
  • Christina Wang
  • Hai Shu
  • Ziqi Chen
  • Zhuoran Sun
  • Yanfeng Yang

Conditional independence (CI) testing is a fundamental task in statistics and machine learning, but its effectiveness is hindered by the challenges posed by high-dimensional conditioning variables and limited data samples. This article introduces a novel testing approach to address these challenges and enhance control of the type I error while achieving high power under alternative hypotheses. The proposed approach incorporates a computationally efficient classifier-based conditional mutual information (CMI) estimator, capable of capturing intricate dependence structures among variables. To approximate a distribution encoding the null hypothesis, a $k$-nearest-neighbor local sampling strategy is employed. An important advantage of this approach is its ability to operate without assumptions about distribution forms or feature dependencies. Furthermore, it eliminates the need to derive asymptotic null distributions for the estimated CMI and avoids dataset splitting, making it particularly suitable for small datasets. The method presented in this article demonstrates asymptotic control of the type I error and consistency against all alternative hypotheses. Extensive analyses using both synthetic and real data highlight the computational efficiency of the proposed test. Moreover, it outperforms existing state-of-the-art methods in terms of type I and II errors, even in scenarios with high-dimensional conditioning sets. Additionally, the proposed approach exhibits robustness in the presence of heavy-tailed data.

PDF Details

NeurIPS Conference 2023 Conference Paper

Learning Adversarial Low-rank Markov Decision Processes with Unknown Transition and Full-information Feedback

  • Canzhe Zhao
  • Ruofeng Yang
  • Baoxiang Wang
  • Xuezhou Zhang
  • Shuai Li

In this work, we study the low-rank MDPs with adversarially changed losses in the full-information feedback setting. In particular, the unknown transition probability kernel admits a low-rank matrix decomposition \citep{REPUCB22}, and the loss functions may change adversarially but are revealed to the learner at the end of each episode. We propose a policy optimization-based algorithm POLO, and we prove that it attains the $\widetilde{O}(K^{\frac{5}{6}}A^{\frac{1}{2}}d\ln(1+M)/(1-\gamma)^2)$ regret guarantee, where $d$ is rank of the transition kernel (and hence the dimension of the unknown representations), $A$ is the cardinality of the action space, $M$ is the cardinality of the model class that contains all the plausible representations, and $\gamma$ is the discounted factor. Notably, our algorithm is oracle-efficient and has a regret guarantee with no dependence on the size of potentially arbitrarily large state space. Furthermore, we also prove an $\Omega(\frac{\gamma^2}{1-\gamma} \sqrt{d A K})$ regret lower bound for this problem, showing that low-rank MDPs are statistically more difficult to learn than linear MDPs in the regret minimization setting. To the best of our knowledge, we present the first algorithm that interleaves representation learning, exploration, and exploitation to achieve the sublinear regret guarantee for RL with nonlinear function approximation and adversarial losses.

PDF Details

AAAI Conference 2023 Conference Paper

Nearest-Neighbor Sampling Based Conditional Independence Testing

  • Shuai Li
  • Ziqi Chen
  • Hongtu Zhu
  • Christina Dan Wang
  • Wang Wen

The conditional randomization test (CRT) was recently proposed to test whether two random variables X and Y are conditionally independent given random variables Z. The CRT assumes that the conditional distribution of X given Z is known under the null hypothesis and then it is compared to the distribution of the observed samples of the original data. The aim of this paper is to develop a novel alternative of CRT by using nearest-neighbor sampling without assuming the exact form of the distribution of X given Z. Specifically, we utilize the computationally efficient 1-nearest-neighbor to approximate the conditional distribution that encodes the null hypothesis. Then, theoretically, we show that the distribution of the generated samples is very close to the true conditional distribution in terms of total variation distance. Furthermore, we take the classifier-based conditional mutual information estimator as our test statistic. The test statistic as an empirical fundamental information theoretic quantity is able to well capture the conditional-dependence feature. We show that our proposed test is computationally very fast, while controlling type I and II errors quite well. Finally, we demonstrate the efficiency of our proposed test in both synthetic and real data analyses.

PDF Details DOI

EAAI Journal 2023 Journal Article

Object detection in hospital facilities: A comprehensive dataset and performance evaluation

  • Da Hu
  • Shuai Li
  • Mengjun Wang

Detecting objects in hospital indoor environments is critical for scene understanding and can have various applications in healthcare. Deep learning algorithms have proven to be effective in object recognition from images or videos, but the availability of annotated datasets plays a crucial role in their successful application. However, there is a shortage of datasets for object detection in hospital settings, hindering the advancement of hospital indoor object detection algorithms. In this paper, we present the Hospital Indoor Object Detection (HIOD) dataset, consisting of 4, 417 images covering 56 object categories. The HIOD dataset represents the frequently encountered objects in hospitals and comprises 51, 869 annotated objects. The dataset is characterized by dense annotation, with an average of 11. 7 objects and 6. 8 object categories per image. An object detection benchmark was established using the HIOD dataset and eight state-of-the-art object detectors. The benchmark provides a comprehensive evaluation of the performance of the selected object detectors on a large and diverse set of images of objects commonly seen in hospital environments. The results of the benchmark can be used to compare and analyze the performance of different object detectors and identify their strengths and weaknesses for use in hospital environments. In the benchmark, one-stage detectors have shown superior performance compared to two-stage detectors of similar parameter sizes. In particular, YOLOv6-L was able to attain a mean Average Precision (mAP) of 51. 7% while operating at a detection speed of 255 FPS. The benchmark and dataset can serve as a valuable resource for researchers and practitioners in the field of computer vision and robotics, helping to advance the development of more effective and efficient object detection algorithms for developing automated operations in hospitals such as robotic disinfection and patient assistance.

Details DOI

NeurIPS Conference 2023 Conference Paper

Online Clustering of Bandits with Misspecified User Models

  • Zhiyong Wang
  • Jize Xie
  • Xutong Liu
  • Shuai Li
  • John C. S. Lui

The contextual linear bandit is an important online learning problem where given arm features, a learning agent selects an arm at each round to maximize the cumulative rewards in the long run. A line of works, called the clustering of bandits (CB), utilize the collaborative effect over user preferences and have shown significant improvements over classic linear bandit algorithms. However, existing CB algorithms require well-specified linear user models and can fail when this critical assumption does not hold. Whether robust CB algorithms can be designed for more practical scenarios with misspecified user models remains an open problem. In this paper, we are the first to present the important problem of clustering of bandits with misspecified user models (CBMUM), where the expected rewards in user models can be perturbed away from perfect linear models. We devise two robust CB algorithms, RCLUMB and RSCLUMB (representing the learned clustering structure with dynamic graph and sets, respectively), that can accommodate the inaccurate user preference estimations and erroneous clustering caused by model misspecifications. We prove regret upper bounds of $O(\epsilon_*T\sqrt{md\log T} + d\sqrt{mT}\log T)$ for our algorithms under milder assumptions than previous CB works, which match the lower bound asymptotically in $T$ up to logarithmic factors, and also match the state-of-the-art results in several degenerate cases. Our regret analysis is novel and different from the typical proof flow of previous CB works. The techniques in proving the regret caused by misclustering users are quite general and may be of independent interest. Experiments on both synthetic and real-world data show our outperformance over previous algorithms.

PDF Details

NeurIPS Conference 2023 Conference Paper

Online Corrupted User Detection and Regret Minimization

  • Zhiyong Wang
  • Jize Xie
  • Tong Yu
  • Shuai Li
  • John C. S. Lui

In real-world online web systems, multiple users usually arrive sequentially into the system. For applications like click fraud and fake reviews, some users can maliciously perform corrupted (disrupted) behaviors to trick the system. Therefore, it is crucial to design efficient online learning algorithms to robustly learn from potentially corrupted user behaviors and accurately identify the corrupted users in an online manner. Existing works propose bandit algorithms robust to adversarial corruption. However, these algorithms are designed for a single user, and cannot leverage the implicit social relations among multiple users for more efficient learning. Moreover, none of them consider how to detect corrupted users online in the multiple-user scenario. In this paper, we present an important online learning problem named LOCUD to learn and utilize unknown user relations from disrupted behaviors to speed up learning, and identify the corrupted users in an online setting. To robustly learn and utilize the unknown relations among potentially corrupted users, we propose a novel bandit algorithm RCLUB-WCU. To detect the corrupted users, we devise a novel online detection algorithm OCCUD based on RCLUB-WCU's inferred user relations. We prove a regret upper bound for RCLUB-WCU, which asymptotically matches the lower bound with respect to $T$ up to logarithmic factors, and matches the state-of-the-art results in degenerate cases. We also give a theoretical guarantee for the detection accuracy of OCCUD. With extensive experiments, our methods achieve superior performance over previous bandit algorithms and high corrupted user detection accuracy.

PDF Details

AAMAS Conference 2023 Conference Paper

Online Influence Maximization under Decreasing Cascade Model

  • Fang Kong
  • Jize Xie
  • Baoxiang Wang
  • Tao Yao
  • Shuai Li

We study online influence maximization (OIM) under a new model of decreasing cascade (DC). This model is a generalization of the independent cascade (IC) model by considering the common phenomenon of market saturation. In DC, the chance of an influence attempt being successful reduces with previous failures. The effect is neglected by previous OIM works under IC and linear threshold models. We propose the DC-UCB algorithm to solve this problem, which achieves a regret bound of the same order as the state-of-theart works on the IC model. Extensive experiments on both synthetic and real datasets show the effectiveness of our algorithm.

PDF

EAAI Journal 2023 Journal Article

Single-image HDR reconstruction by dual learning the camera imaging process

  • Lei She
  • Mao Ye
  • Shuai Li
  • Yu Zhao
  • Ce Zhu
  • Hu Wang

It is a very challenging problem to reconstruct a high dynamic range (HDR) image from a single exposure image. There exist three problems, i. e. , the many-to-many mapping problem between low dynamic range (LDR) images and HDR images, the image quality problem caused by the change of dynamic range and the problem of unpaired LDR–HDR training images. These problems can be solved to some extent through a dual learning framework simultaneously to learn the forward and reverse of camera imaging processes. This procedure is divided into a primary module, to reconstruct HDR from LDR, and a secondary module to reversely mapping the HDR to LDR. The secondary module guides the learning of primary module by constraining the outputs of the primary module. After that, the attention mechanism is used to solve the problem of unnatural perception caused by the change of dynamic range. In the end, with the advantage of our dual learning framework, unpaired data is further explored to train our model, which enriches the training samples. Compared with the state-of-the-art methods, a large number of quantitative and qualitative experiments confirm that our method can achieve better performance.

Details DOI

AAAI Conference 2023 Conference Paper

Understanding Representation Learnability of Nonlinear Self-Supervised Learning

  • Ruofeng Yang
  • Xiangyuan Li
  • Bo Jiang
  • Shuai Li

Self-supervised learning (SSL) has empirically shown its data representation learnability in many downstream tasks. There are only a few theoretical works on data representation learnability, and many of those focus on final data representation, treating the nonlinear neural network as a ``black box". However, the accurate learning results of neural networks are crucial for describing the data distribution features learned by SSL models. Our paper is the first to analyze the learning results of the nonlinear SSL model accurately. We consider a toy data distribution that contains two features: the label-related feature and the hidden feature. Unlike previous linear setting work that depends on closed-form solutions, we use the gradient descent algorithm to train a 1-layer nonlinear SSL model with a certain initialization region and prove that the model converges to a local minimum. Furthermore, different from the complex iterative analysis, we propose a new analysis process which uses the exact version of Inverse Function Theorem to accurately describe the features learned by the local minimum. With this local minimum, we prove that the nonlinear SSL model can capture the label-related feature and hidden feature at the same time. In contrast, the nonlinear supervised learning (SL) model can only learn the label-related feature. We also present the learning processes and results of the nonlinear SSL and SL model via simulation experiments.

PDF Details DOI

JBHI Journal 2022 Journal Article

Automatic Dental Plaque Segmentation Based on Local-to-Global Features Fused Self-Attention Network

  • Shuai Li
  • Yuting Guo
  • Zhennan Pang
  • Wenfeng Song
  • Aimin Hao
  • Bin Xia
  • Hong Qin

The accurate detection of dental plaque at an early stage will definitely prevent periodontal diseases and dental caries. However, it remains difficult for the current dental examination to accurately recognize dental plaque without using medical dyeing reagent due to the low contrast between dental plaque and healthy teeth. To combat this problem, this paper proposes a novel network enhanced by a self-attention module for intelligent dental plaque segmentation. The key motivation is to directly utilize oral endoscope images (bypassing the need for dyeing reagent) and get accurate pixel-level dental plaque segmentation results. The algorithm needs to conduct self-attention at the super-pixel level and fuse the super-pixels’ local-to-global features. Our newly-designed network architecture will afford the simultaneous fusion of multiple-scale complementary information guided by the powerful deep learning paradigm. The critical fused information includes the statistical distribution of the plaques color, the heat kernel signature (HKS) based local-to-global structure relationship, and the circle-LBP based local texture pattern in the nearby regions centering around the plaque area. To further refine the fuzed multiple-scale features, we devise an attention module based on CNN, which could focalize the regions of interest in plaque more easily, especially for many challenging cases. Extensive experiments and comprehensive evaluations confirm that, for a small-scale training dataset, our method could outperform the state-of-the-art methods. Meanwhile, the user studies verify the claim that our method is more accurate than conventional dental practice conducted by experienced dentists.

Details DOI

AAAI Conference 2022 Conference Paper

DisenCite: Graph-Based Disentangled Representation Learning for Context-Specific Citation Generation

  • Yifan Wang
  • Yiping Song
  • Shuai Li
  • Chaoran Cheng
  • Wei Ju
  • Ming Zhang
  • Sheng Wang

Citing and describing related literature are crucial to scientific writing. Many existing approaches show encouraging performance in citation recommendation, but are unable to accomplish the more challenging and onerous task of citation text generation. In this paper, we propose a novel disentangled representation based model DisenCite to automatically generate the citation text through integrating paper text and citation graph. A key novelty of our method compared with existing approaches is to generate context-specific citation text, empowering the generation of different types of citations for the same paper. In particular, we first build and make available a graph enhanced contextual citation dataset (GCite) with 25K edges in different types characterized by citation contained sections over 4. 8K research papers. Based on this dataset, we encode each paper according to both textual contexts and structure information in the heterogeneous citation graph. The resulted paper representations are then disentangled by the mutual information regularization between this paper and its neighbors in graph. Extensive experiments demonstrate the superior performance of our method comparing to state-of-the-art approaches. We further conduct ablation and case studies to reassure that the improvement of our method comes from generating the context-specific citation through incorporating the citation graph.

PDF Details

IJCAI Conference 2022 Conference Paper

KUNet: Imaging Knowledge-Inspired Single HDR Image Reconstruction

  • Hu Wang
  • Mao Ye
  • Xiatian Zhu
  • Shuai Li
  • Ce Zhu
  • Xue Li

Recently, with the rise of high dynamic range (HDR) display devices, there is a great demand to transfer traditional low dynamic range (LDR) images into HDR versions. The key to success is how to solve the many-to-many mapping problem. However, the existing approaches either do not consider constraining solution space or just simply imitate the inverse camera imaging pipeline in stages, without directly formulating the HDR image generation process. In this work, we address this problem by integrating LDR-to-HDR imaging knowledge into an UNet architecture, dubbed as Knowledge-inspired UNet (KUNet). The conversion from LDR-to-HDR image is mathematically formulated, and can be conceptually divided into recovering missing details, adjusting imaging parameters and reducing imaging noise. Accordingly, we develop a basic knowledge-inspired block (KIB) including three subnetworks corresponding to the three procedures in this HDR imaging process. The KIB blocks are cascaded in the similar way to the UNet to construct HDR image with rich global information. In addition, we also propose a knowledge inspired jump-connect structure to fit a dynamic range gap between HDR and LDR images. Experimental results demonstrate that the proposed KUNet achieves superior performance compared with the state-of-the-art methods. The code, dataset and appendix materials are available at https: //github. com/wanghu178/KUNet. git.

PDF Details DOI

AAAI Conference 2022 Conference Paper

Simultaneously Learning Stochastic and Adversarial Bandits under the Position-Based Model

  • Cheng Chen
  • Canzhe Zhao
  • Shuai Li

Online learning to rank (OLTR) interactively learns to choose lists of items from a large collection based on certain click models that describe users’ click behaviors. Most recent works for this problem focus on the stochastic environment where the item attractiveness is assumed to be invariant during the learning process. In many real-world scenarios, however, the environment could be dynamic or even arbitrarily changing. This work studies the OLTR problem in both stochastic and adversarial environments under the positionbased model (PBM). We propose a method based on the follow-the-regularized-leader (FTRL) framework with Tsallis entropy and develop a new self-bounding constraint especially designed for PBM. We prove the proposed algorithm simultaneously achieves O(log T) regret in the stochastic environment and O(m √ nT) regret in the adversarial environment, where T is the number of rounds, n is the number of items and m is the number of positions. We also provide a lower bound of order Ω(m √ nT) for adversarial PBM, which matches our upper bound and improves over the state-of-theart lower bound. The experiments show that our algorithm could simultaneously learn in both stochastic and adversarial environments and is competitive compared to existing methods that are designed for a single environment.

PDF Details

IJCAI Conference 2022 Conference Paper

Thompson Sampling for Bandit Learning in Matching Markets

  • Fang Kong
  • Junming Yin
  • Shuai Li

The problem of two-sided matching markets has a wide range of real-world applications and has been extensively studied in the literature. A line of recent works have focused on the problem setting where the preferences of one-side market participants are unknown a priori and are learned by iteratively interacting with the other side of participants. All these works are based on explore-then-commit (ETC) and upper confidence bound (UCB) algorithms, two common strategies in multi-armed bandits (MAB). Thompson sampling (TS) is another popular approach, which attracts lots of attention due to its easier implementation and better empirical performances. In many problems, even when UCB and ETC-type algorithms have already been analyzed, researchers are still trying to study TS for its benefits. However, the convergence analysis of TS is much more challenging and remains open in many problem settings. In this paper, we provide the first regret analysis for TS in the new setting of iterative matching markets. Extensive experiments demonstrate the practical advantages of the TS-type algorithm over the ETC and UCB-type baselines.

PDF Details DOI

AAAI Conference 2021 Conference Paper

Category Dictionary Guided Unsupervised Domain Adaptation for Object Detection

  • Shuai Li
  • Jianqiang Huang
  • Xian-Sheng Hua
  • Lei Zhang

Unsupervised domain adaption (UDA) is a promising solution to enhance the generalization ability of a model from a source domain to a target domain without manually annotating labels for the target data. Recent works in cross-domain object detection mostly resort to adversarial feature adaptation to match the marginal distributions of two domains. However, perfect feature alignment is hard to achieve and what’s more is likely to cause negative transfer due to the high complexity of object detection. In this paper, we take a different approach to reduce the domain gap by a selftraining paradigm, which regards the pseudo-labels as ground truth to fully exploit the unlabeled target data. In order to generate more informative pseudo labels, we further propose a category dictionary guided (CDG) UDA model for crossdomain object detection, which learns category-specific dictionaries from the source domain to represent the candidate boxes in target domain. The representation residual can be used for not only pseudo label assignment but also quality (e. g. , IoU) estimation of the candidate box. Compared with decision boundary based classifiers such as softmax, the proposed CDG scheme can select more informative and reliable pseudo-boxes. Experimental results on benchmark datasets show that the proposed CDG significantly exceeds the stateof-the-arts in cross-domain object detection.

PDF Details

ICRA Conference 2021 Conference Paper

Decentralized Circle Formation Control for Fish-like Robots in the Real-world via Reinforcement Learning

  • Tianhao Zhang
  • Yueheng Li
  • Shuai Li
  • Qiwei Ye
  • Chen Wang 0005
  • Guangming Xie

In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from simulation to reality. It is only trained in our established simulation environment, and the trained controller can be deployed to real robots without any manual tuning. Simulation results confirm that the proposed model-free robust formation control method is scalable with respect to the group size of the robots and outperforms other representative RL algorithms. Several experiments in the real world verify the effectiveness of our RL-based approach for circle formation control.

Details

NeurIPS Conference 2021 Conference Paper

Knowledge-inspired 3D Scene Graph Prediction in Point Cloud

  • Shoulong Zhang
  • Shuai Li
  • Aimin Hao
  • Hong Qin

Prior knowledge integration helps identify semantic entities and their relationships in a graphical representation, however, its meaningful abstraction and intervention remain elusive. This paper advocates a knowledge-inspired 3D scene graph prediction method solely based on point clouds. At the mathematical modeling level, we formulate the task as two sub-problems: knowledge learning and scene graph prediction with learned prior knowledge. Unlike conventional methods that learn knowledge embedding and regular patterns from encoded visual information, we propose to suppress the misunderstandings caused by appearance similarities and other perceptual confusion. At the network design level, we devise a graph auto-encoder to automatically extract class-dependent representations and topological patterns from the one-hot class labels and their intrinsic graphical structures, so that the prior knowledge can avoid perceptual errors and noises. We further devise a scene graph prediction model to predict credible relationship triplets by incorporating the related prototype knowledge with perceptual information. Comprehensive experiments confirm that, our method can successfully learn representative knowledge embedding, and the obtained prior knowledge can effectively enhance the accuracy of relationship predictions. Our thorough evaluations indicate the new method can achieve the state-of-the-art performance compared with other scene graph prediction methods.

PDF Details

AAAI Conference 2021 Conference Paper

Point Cloud Semantic Scene Completion from RGB-D Images

  • Shoulong Zhang
  • Shuai Li
  • Aimin Hao
  • Hong Qin

In this paper, we devise a novel semantic completion network, called point cloud semantic scene completion network (PCSSC-Net), for indoor scenes solely based on point clouds. Existing point cloud completion networks still suffer from their inability of fully recovering complex structures and contents from global geometric descriptions neglecting semantic hints. To extract and infer comprehensive information from partial input, we design a patch-based contextual encoder to hierarchically learn point-level, patch-level, and scene-level geometric and contextual semantic information with a divideand-conquer strategy. Consider that the scene semantics afford a high-level clue of constituting geometry for an indoor scene environment, we articulate a semantics-guided completion decoder where semantics could help cluster isolated points in the latent space and infer complicated scene geometry. Given the fact that real-world scans tend to be incomplete as ground truth, we choose to synthesize scene dataset with RGB-D images and annotate complete point clouds as ground truth for the supervised training purpose. Extensive experiments validate that our new method achieves the stateof-the-art performance, in contrast with the current methods applied to our dataset.

PDF Details

IJCAI Conference 2021 Conference Paper

Self-Guided Community Detection on Networks with Missing Edges

  • Dongxiao He
  • Shuai Li
  • Di Jin
  • Pengfei Jiao
  • Yuxiao Huang

The vast majority of community detection algorithms assume that the networks are totally observed. However, in reality many networks cannot be fully observed. On such network is edges-missing network, where some relationships (edges) between two entities are missing. Recently, several works have been proposed to solve this problem by combining link prediction and community detection in a two-stage method or in a unified framework. However, the goal of link prediction, which is to predict as many correct edges as possible, is not consistent with the requirement for predicting the important edges for discovering community structure on edges-missing networks. Thus, combining link prediction and community detection cannot work very well in terms of detecting community structure for edges-missing network. In this paper, we propose a community self-guided generative model which jointly completes the edges-missing network and identifies communities. In our new model, completing missing edges and identifying communities are not isolated but closely intertwined. Furthermore, we developed an effective model inference method that combines a nested Expectation-Maximization (EM) algorithm and Metropolis-Hastings Sampling. Extensive experiments on real-world edges-missing networks show that our model can effectively detect community structures while completing missing edges.

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

The Hardness Analysis of Thompson Sampling for Combinatorial Semi-bandits with Greedy Oracle

  • Fang Kong
  • Yueran Yang
  • Wei Chen
  • Shuai Li

Thompson sampling (TS) has attracted a lot of interest in the bandit area. It was introduced in the 1930s but has not been theoretically proven until recent years. All of its analysis in the combinatorial multi-armed bandit (CMAB) setting requires an exact oracle to provide optimal solutions with any input. However, such an oracle is usually not feasible since many combinatorial optimization problems are NP-hard and only approximation oracles are available. An example \cite{WangC18} has shown the failure of TS to learn with an approximation oracle. However, this oracle is uncommon and is designed only for a specific problem instance. It is still an open question whether the convergence analysis of TS can be extended beyond the exact oracle in CMAB. In this paper, we study this question under the greedy oracle, which is a common (approximation) oracle with theoretical guarantees to solve many (offline) combinatorial optimization problems. We provide a problem-dependent regret lower bound of order $\Omega(\log T/\Delta^2)$ to quantify the hardness of TS to solve CMAB problems with greedy oracle, where $T$ is the time horizon and $\Delta$ is some reward gap. We also provide an almost matching regret upper bound. These are the first theoretical results for TS to solve CMAB with a common approximation oracle and break the misconception that TS cannot work with approximation oracles.

PDF Details

NeurIPS Conference 2021 Conference Paper

Understanding Bandits with Graph Feedback

  • Houshuang Chen
  • zengfeng Huang
  • Shuai Li
  • Chihao Zhang

The bandit problem with graph feedback, proposed in [Mannor and Shamir, NeurIPS 2011], is modeled by a directed graph $G=(V, E)$ where $V$ is the collection of bandit arms, and once an arm is triggered, all its incident arms are observed. A fundamental question is how the structure of the graph affects the min-max regret. We propose the notions of the fractional weak domination number $\delta^*$ and the $k$-packing independence number capturing upper bound and lower bound for the regret respectively. We show that the two notions are inherently connected via aligning them with the linear program of the weakly dominating set and its dual --- the fractional vertex packing set respectively. Based on this connection, we utilize the strong duality theorem to prove a general regret upper bound $O\left(\left(\delta^*\log |V|\right)^{\frac{1}{3}}T^{\frac{2}{3}}\right)$ and a lower bound $\Omega\left(\left(\delta^*/\alpha\right)^{\frac{1}{3}}T^{\frac{2}{3}}\right)$ where $\alpha$ is the integrality gap of the dual linear program. Therefore, our bounds are tight up to a $\left(\log |V|\right)^{\frac{1}{3}}$ factor on graphs with bounded integrality gap for the vertex packing problem including trees and graphs with bounded degree. Moreover, we show that for several special families of graphs, we can get rid of the $\left(\log |V|\right)^{\frac{1}{3}}$ factor and establish optimal regret.

PDF Details

NeurIPS Conference 2020 Conference Paper

Online Influence Maximization under Linear Threshold Model

  • Shuai Li
  • Fang Kong
  • Kejie Tang
  • Qizhi Li
  • Wei Chen

Online influence maximization (OIM) is a popular problem in social networks to learn influence propagation model parameters and maximize the influence spread at the same time. Most previous studies focus on the independent cascade (IC) model under the edge-level feedback. In this paper, we address OIM in the linear threshold (LT) model. Because node activations in the LT model are due to the aggregated effect of all active neighbors, it is more natural to model OIM with the nodel-level feedback. And this brings new challenge in online learning since we only observe aggregated effect from groups of nodes and the groups are also random. Based on the linear structure in node activations, we incorporate ideas from linear bandits and design an algorithm $\ltlinucb$ that is consistent with the observed feedback. By proving group observation modulated (GOM) bounded smoothness property, a novel result of the influence difference in terms of the random observations, we provide a regret of order $\tilde{O}(\mathrm{poly}(m)\sqrt{T})$, where $m$ is the number of edges and $T$ is the number of rounds. This is the first theoretical result in such order for OIM under the LT model. In the end, we also provide an algorithm $\oimetc$ with regret bound $O(\mathrm{poly}(m)\ T^{2/3})$, which is model-independent, simple and has less requirement on online feedback and offline computation.

PDF Details

AAAI Conference 2020 Conference Paper

Stochastic Online Learning with Probabilistic Graph Feedback

  • Shuai Li
  • Wei Chen
  • Zheng Wen
  • Kwong-Sak Leung

We consider a problem of stochastic online learning with general probabilistic graph feedback, where each directed edge in the feedback graph has probability pij. Two cases are covered. (a) The one-step case, where after playing arm i the learner observes a sample reward feedback of arm j with independent probability pij. (b) The cascade case where after playing arm i the learner observes feedback of all arms j in a probabilistic cascade starting from i – for each (i, j) with probability pij, if arm i is played or observed, then a reward sample of arm j would be observed with independent probability pij. Previous works mainly focus on deterministic graphs which corresponds to one-step case with pij ∈ {0, 1}, an adversarial sequence of graphs with certain topology guarantees, or a specific type of random graphs. We analyze the asymptotic lower bounds and design algorithms in both cases. The regret upper bounds of the algorithms match the lower bounds with high probability.

PDF Details

ICML Conference 2020 Conference Paper

Towards Understanding the Regularization of Adversarial Robustness on Neural Networks

  • Yuxin Wen
  • Shuai Li
  • Kui Jia

The problem of adversarial examples has shown that modern Neural Network (NN) models could be rather fragile. Among the more established techniques to solve the problem, one is to require the model to be \emph{$\epsilon$-adversarially robust} (AR); that is, to require the model not to change predicted labels when any given input examples are perturbed within a certain range. However, it is observed that such methods would lead to standard performance degradation, i. e. , the degradation on natural examples. In this work, we study the degradation through the regularization perspective. We identify quantities from generalization analysis of NNs; with the identified quantities we empirically find that AR is achieved by regularizing/biasing NNs towards less confident solutions by making the changes in the feature space (induced by changes in the instance space) of most layers smoother uniformly in all directions; so to a certain extent, it prevents sudden change in prediction w. r. t. perturbations. However, the end result of such smoothing concentrates samples around decision boundaries, resulting in less confident solutions, and leads to worse standard performance. Our studies suggest that one might consider ways that build AR into NNs in a gentler way to avoid the problematic regularization.

Details

IROS Conference 2019 Conference Paper

Adaptive Assist-as-needed Control Based on Actor-Critic Reinforcement Learning

  • Yufeng Zhang 0003
  • Shuai Li
  • Karen J. Nolan
  • Damiano Zanotto

In robot-assisted rehabilitation, assist-as-needed (AAN) controllers have been proposed to promote subjects’ active participation, which is thought to lead to better training outcomes. Most of these AAN controllers require a patient-specific manual tuning of the parameters defining the underlying force-field, which typically results in a tedious and time-consuming process. In this paper, we propose a reinforcement-learning-based impedance controller that actively reshapes the stiffness of the force-field to the subject’s performance, while providing assistance only when needed. This adaptability is made possible by correlating the subject’s most recent performance to the ultimate control objective in real-time. In addition, the proposed controller is built upon action dependent heuristic dynamic programming using the actor-critic structure, and therefore does not require prior knowledge of the system model. The controller is experimentally validated with healthy subjects through a simulated ankle mobilization training session using a powered ankle-foot orthosis.

Details

IJCAI Conference 2019 Conference Paper

Improved Algorithm on Online Clustering of Bandits

  • Wei Chen
  • Shuai Li
  • Kwong-Sak Leung

We generalize the setting of online clustering of bandits by allowing non-uniform distribution over user frequencies. A more efficient algorithm is proposed with simple set structures to represent clusters. We prove a regret bound for the new algorithm which is free of the minimal frequency over users. The experiments on both synthetic and real datasets consistently show the advantage of the new algorithm over existing methods.

PDF Details

ICRA Conference 2019 Conference Paper

Manipulability Optimization Control of a Serial Redundant Robot for Robot-assisted Minimally Invasive Surgery

  • Hang Su 0001
  • Shuai Li
  • Jagadesh Manivannan
  • Luca Bascetta
  • Giancarlo Ferrigno
  • Elena De Momi

This paper proposes a manipulability optimization control of a 7-DoF robot manipulator for Robot-Assisted Minimally Invasive Surgery (RAMIS), which at the same time guarantees a Remote Center of Motion (RCM). The first degree of redundancy of the manipulator is used to achieve an RCM constraint, the second one is adopted for manipulability optimization. A hierarchical operational space formulation is introduced to integrate all the control components, including a Cartesian compliance control involving the main surgical task, a first null-space controller for the RCM constraint, and a second null-space controller for manipulability optimization. Experiments with virtual surgical tasks, in an augmented reality environment, were performed to validate the proposed control strategy using the KUKA LWR 4 +. The results demonstrate that end-effector accuracy and RCM constraint can be guaranteed, along with improving the manipulability of the surgical tip.

Details

JBHI Journal 2019 Journal Article

Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition

  • Wenfeng Song
  • Shuai Li
  • Ji Liu
  • Hong Qin
  • Bo Zhang
  • Shuyang Zhang
  • Aimin Hao

Thyroid ultrasonography is a widely used clinical technique for nodule diagnosis in thyroid regions. However, it remains difficult to detect and recognize the nodules due to low contrast, high noise, and diverse appearance of nodules. In today's clinical practice, senior doctors could pinpoint nodules by analyzing global context features, local geometry structure, and intensity changes, which would require rich clinical experience accumulated from hundreds and thousands of nodule case studies. To alleviate doctors’ tremendous labor in the diagnosis procedure, we advocate a machine learning approach to the detection and recognition tasks in this paper. In particular, we develop a multitask cascade convolution neural network (MC-CNN) framework to exploit the context information of thyroid nodules. It may be noted that our framework is built upon a large number of clinically confirmed thyroid ultrasound images with accurate and detailed ground truth labels. Other key advantages of our framework result from a multitask cascade architecture, two stages of carefully designed deep convolution networks in order to detect and recognize thyroid nodules in a pyramidal fashion, and capturing various intrinsic features in a global-to-local way. Within our framework, the potential regions of interest after initial detection are further fed to the spatial pyramid augmented CNNs to embed multiscale discriminative information for fine-grained thyroid recognition. Experimental results on 4309 clinical ultrasound images have indicated that our MC-CNN is accurate and effective for both thyroid nodules detection and recognition. For the correct diagnosis rate of malignant and benign thyroid nodules, its mean Average Precision (mAP) performance can achieve up to $\text{98. 2}\%$ accuracy, which outperforms the common CNNs by $\text{5}\%$ on average. In addition, we conduct rigorous user studies to confirm that our MC-CNN outperforms experienced doctors, yet only consuming roughly $\text{2}\%$ ( $1/48$ ) of doctors’ examination time on average. Therefore, the accuracy and efficiency of our new method exhibit its great potential in clinical applications.

Details DOI

IROS Conference 2019 Conference Paper

Tracking Control of Fully-Constrained Cable-Driven Parallel Robots using Adaptive Dynamic Programming

  • Shuai Li
  • Damiano Zanotto

In this paper, a new adaptive tracking controller with learning ability is proposed for fully-constrained cable-driven parallel robots (CDPRs). For these systems, the necessity of maintaining positive and bounded tensions in all cables while coping with disturbances represents a critical control requirement. To achieve this goal, we propose a control law based on adaptive dynamic programming (ADP), with an actorcritic structure. In the critic part, an artificial neural network (NN) approximates the value function which is to evaluate the system performance; in the action part, the controller’s parameters are tuned online to achieve optimal control performance. Additionally, the anti-windup (AW) technique is combined with the adaptive controller to cope with the input saturation problem. The stability of the closed-loop system with the proposed control algorithm is proved using the Lyapunov method. Numerical simulations show the effectiveness of the proposed controller.

Details

AAAI Conference 2018 Conference Paper

Online Clustering of Contextual Cascading Bandits

  • Shuai Li
  • Shengyu Zhang

We consider a new setting of online clustering of contextual cascading bandits, an online learning problem where the underlying cluster structure over users is unknown and needs to be learned from a random prefix feedback. More precisely, a learning agent recommends an ordered list of items to a user, who checks the list and stops at the first satisfactory item, if any. We propose an algorithm of CLUB-cascade for this setting and prove an n-step regret bound of order Õ( √ n). Previous work (Li et al. 2016) corresponds to the degenerate case of only one cluster, and our general regret bound in this special case also significantly improves theirs. We conduct experiments on both synthetic and real data, and demonstrate the effectiveness of our algorithm and the advantage of incorporating online clustering method.

PDF Details

NeurIPS Conference 2018 Conference Paper

TopRank: A practical algorithm for online stochastic ranking

  • Tor Lattimore
  • Branislav Kveton
  • Shuai Li
  • Csaba Szepesvari

Online learning to rank is a sequential decision-making problem where in each round the learning agent chooses a list of items and receives feedback in the form of clicks from the user. Many sample-efficient algorithms have been proposed for this problem that assume a specific click model connecting rankings and user behavior. We propose a generalized click model that encompasses many existing models, including the position-based and cascade models. Our generalization motivates a novel online learning algorithm based on topological sort, which we call TopRank. TopRank is (a) more natural than existing algorithms, (b) has stronger regret guarantees than existing algorithms with comparable generality, (c) has a more insightful proof that leaves the door open to many generalizations, (d) outperforms existing algorithms empirically.

PDF Details

ICML Conference 2016 Conference Paper

Distributed Clustering of Linear Bandits in Peer to Peer Networks

  • Nathaniel Korda
  • Balázs Szörényi
  • Shuai Li

We provide two distributed confidence ball algorithms for solving linear bandit problems in peer to peer networks with limited communication capabilities. For the first, we assume that all the peers are solving the same linear bandit problem, and prove that our algorithm achieves the optimal asymptotic regret rate of any centralised algorithm that can instantly communicate information between the peers. For the second, we assume that there are clusters of peers solving the same bandit problem within each cluster, and we prove that our algorithm discovers these clusters, while achieving the optimal asymptotic regret rate within each one. Through experiments on several real-world datasets, we demonstrate the performance of proposed algorithms compared to the state-of-the-art.

Details

TAAS Journal 2016 Journal Article

Distributed Multirobot Formation and Tracking Control in Cluttered Environments

  • Muhammad Umer Khan
  • Shuai Li
  • Qixin Wang
  • Zili Shao

In this article, we propose formation control of nonholonomic mobile robots avoiding obstacles in a distributed manner for cluttered environments. The introduction of a virtual robot restructures the formation control problem into a tracking control problem between the virtual reference robot and follower robots. A novel obstacle avoidance approach is proposed based upon the scaling of whole (partial) formation corresponding to a centralized (distributed) framework. For the distributed environment with limited communication, our approach utilized proportional-integral average consensus estimators, whereby information from each robot diffuses through the communication network. The theoretical contribution is to determine the time constant involved in the diffusion process, which can affect overall system performance. The asymptotic convergence of follower robots to the position and orientation of the reference robot is ensured using the Lyapunov function. The new technique is tested with complete, limited, and no information availability. Several simulation results are provided that demonstrate the formation control and obstacle avoidance for multirobots using the proposed scheme.

Details DOI

ICRA Conference 2014 Conference Paper

Multi-robot cooperative control for monitoring and tracking dynamic plumes

  • Shuai Li
  • Yi Guo 0004
  • Brian Bingham

We study robotic tracking of dynamic plume front modeled by the advection-diffusion equation in this paper. Different from existing work purely relying on gradient measurement, the transport model of pollution source is explicitly considered in tracking control design. We first study the problem using a single robot and solve the problem in an estimation and control framework. We then extend it to the multi-robot case in a nearest-neighbor communication structure, and have the robots take formation along the plume front. The distributed control is scalable to a large number of robots. Simulation results show satisfactory performances of the proposed method.

Details

ICML Conference 2014 Conference Paper

Online Clustering of Bandits

  • Claudio Gentile
  • Shuai Li
  • Giovanni Zappella

We introduce a novel algorithmic approach to content recommendation based on adaptive clustering of exploration-exploitation (“bandit") strategies. We provide a sharp regret analysis of this algorithm in a standard stochastic noise setting, demonstrate its scalability properties, and prove its effectiveness on a number of artificial and real-world datasets. Our experiments show a significant increase in prediction performance over state-of-the-art methods for bandit problems.

Details

ICRA Conference 2012 Conference Paper

Distributed source seeking by cooperative robots: All-to-all and limited communications

  • Shuai Li
  • Yi Guo 0004

We consider the problem of source seeking using a group of mobile robots equipped with sensors for concentration measurement (instead of the gradient). In our formulation, each robot maintains a gradient estimation, moves to the source by tracing the gradient, and all together keep a predefined formation in movement. We present two control algorithms with all-to-all and limited communications, respectively. The estimation error is taken into account to derive robust control algorithms. Comparing to existing methods, the proposed algorithm with limited communications is fully distributed. Both theoretical analysis and numerical simulations are given to validate the effectiveness of our methods.

Details
v2026.09.13