Arrow Research search

Author name cluster

Kai Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

54 papers
2 author rows

Possible papers

54

AAAI Conference 2026 Conference Paper

Decision-Driven Orthogonal Learning with Complementary Feature Mining for Robust Synthetic Image Detection

  • Kai Li
  • Wei Wang
  • Linchao Zhang
  • Siying Zhu
  • Wenqi Ren

The widespread and inconsistent compression applied by Online Social Networks severely degrades the performance of synthetic image detectors. We attribute this degradation to two main issues: 1) the model confuses forgery artifacts with compression artifacts, and 2) compression erodes crucial discriminative high-frequency details. Existing methods suppress compression features during training but overlook the overlap between compression features and forgery-related features, leading to the unintended removal of forgery traces. To address artifact confusion, we introduce a Decision-Driven Orthogonal Constraint, which defines a classification decision axis pointing from the real class centroid to the forged class centroid. This constraint enforces compression artifacts to be orthogonal to the decision axis, mitigating their interference with forgery detection without entirely removing them, thus preventing the suppression of forgery-related features. To mitigate the erosion of high-frequency details, we propose to mine complementary forgery cues from both low-frequency information and compressed high-frequency components. A bidirectional update strategy and an adaptive global-local modulator are proposed to facilitate the utilization of forgery cues. Extensive experiments demonstrate that our method achieves state-of-the-art generalization performance in challenging open-world detection scenarios.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Deep (Predictive) Discounted Counterfactual Regret Minimization

  • Hang Xu
  • Kai Li
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Counterfactual regret minimization (CFR) is a family of algorithms for effectively solving imperfect-information games. To enhance CFR's applicability in large games, researchers use neural networks to approximate its behavior. However, existing methods are mainly based on vanilla CFR and struggle to effectively integrate more advanced CFR variants. In this work, we propose an efficient model-free neural CFR algorithm, overcoming the limitations of existing methods in approximating advanced CFR variants. At each iteration, it collects variance-reduced sampled advantages based on a value network, fits cumulative advantages by bootstrapping, and applies discounting and clipping operations to simulate the update mechanisms of advanced CFR variants. Experimental results show that, compared with model-free neural algorithms, it exhibits faster convergence in typical imperfect-information games and demonstrates stronger adversarial performance in a large poker game.

PDF Details DOI

AAAI Conference 2026 Conference Paper

DegVoC: Revisiting Neural Vocoder from a Degradation Perspective

  • Andong Li
  • Tong Lei
  • Lingling Dai
  • Kai Li
  • Rilin Chen
  • Meng Yu
  • Xiaodong Li
  • Dong Yu

Existing neural vocoders have demonstrated promising performance by leveraging Mel-spectrum as an acoustic feature for conditional audio generation. Nonetheless, they remain constrained by an inherent ``performance-cost'' dilemma that significantly hinders the development of this field. This paper revisits this foundational task from a novel degradation perspective, where Mel-spectrum is regarded as a special signal degradation process from the target spectrum. Drawing inspiration from traditional sparse signal recovery problems, we propose DegVoC, a GAN-based neural vocoder with a two-step solution procedure. First, by exploiting degradation priors, we attempt to retrieve the initial spectral structure from Mel-domain representations as an initial solution via a simple linear transformation. Based on that, we introduce a deep prior solver that accounts for the heterogeneous distribution of sub-bands in the time-frequency domain. A convolution-style attention module with a large kernel size is specially devised for efficient inter-frame and inter-band contextual modeling. With 3.89 M parameters and substantially reduced inference complexity, DegVoC achieves state-of-the-art performance across objective and subjective evaluations, outperforming existing GAN-, DDPM- and flow-matching-based baselines.

PDF Details DOI

AAAI Conference 2026 Conference Paper

DIMM: Decoupled Multi-hierarchy Kalman Filter via Reinforcement Learning

  • Jirong Zha
  • Yuxuan Fan
  • Kai Li
  • Han Li
  • Chen Gao
  • Xinlei Chen

State estimation is challenging for target tracking with high maneuverability, as the target's state transition function changes rapidly, irregularly, and is unknown to the estimator. Existing work based on interacting multiple model (IMM) achieves more accurate estimation than single-filter approaches through model combination, aligning appropriate models for different motion modes of the target over time. However, two limitations of conventional IMM remain unsolved. First, the solution space of the model combination is constrained as the target's diverse kinematic properties in different directions are ignored. Second, the model combination weights calculated by the observation likelihood are not accurate enough due to the measurement uncertainty. In this paper, we propose a novel framework, DIMM, to effectively combine estimates from different motion models in each direction, thus increasing the target tracking accuracy. First, DIMM extends the model combination solution space of conventional IMM from a hyperplane to a hypercube by designing a 3D-decoupled multi-hierarchy filter bank, which describes the target's motion with various-order linear models. Second, DIMM generates more reliable combination weight matrices through a differentiable adaptive fusion network for importance allocation rather than solely relying on the observation likelihood; it contains an attention-based twin delayed deep deterministic policy gradient (TD3) method with a hierarchical reward. Experiments demonstrate that DIMM significantly improves the tracking accuracy of existing state estimation methods by 31.61%~99.23%.

PDF Details DOI

AAAI Conference 2026 Conference Paper

FGNet: Leveraging Feature-Guided Attention to Refine SAM2 for 3D EM Neuron Segmentation

  • Zhenghua Li
  • Hang Chen
  • Zihao Sun
  • Kai Li
  • Xiaolin Hu

Accurate segmentation of neural structures in Electron Microscopy (EM) images is paramount for neuroscience. However, this task is challenged by intricate morphologies, low signal-to-noise ratios, and scarce annotations, limiting the accuracy and generalization of existing methods. To address these challenges, we seek to leverage the priors learned by visual foundation models on a vast amount of natural images to better tackle this task. Specifically, we propose a novel framework that can effectively transfer knowledge from Segment Anything 2 (SAM2), which is pre-trained on natural images, to the EM domain. We first use SAM2 to extract powerful, general-purpose features. To bridge the domain gap, we introduce a Feature-Guided Attention module that leverages semantic cues from SAM2 to guide a lightweight encoder, the Fine-Grained Encoder (FGE), in focusing on these challenging regions. Finally, a dual-affinity decoder generates both coarse and refined affinity maps. Experimental results demonstrate that our method achieves performance comparable to state-of-the-art (SOTA) approaches with the SAM2 weights frozen. Upon further fine-tuning on EM data, our method significantly outperforms existing SOTA methods. This study validates that transferring representations pre-trained on natural images, when combined with targeted domain-adaptive guidance, can effectively address the specific challenges in neuron segmentation.

PDF Details DOI

AAAI Conference 2026 Conference Paper

SepPrune: Structured Pruning for Efficient Deep Speech Separation

  • Yuqi Li
  • Kai Li
  • Xin Yin
  • Zhifei Yang
  • Zeyu Dong
  • Zhengtao Yao
  • Haoyan Xu
  • Yingli Tian

Although deep learning has substantially advanced speech separation in recent years, most existing studies continue to prioritize separation quality while overlooking computational efficiency, an essential factor for low-latency speech processing in real-time applications. In this paper, we propose SepPrune, the first structured pruning framework specifically designed to compress deep speech separation models and reduce their computational cost. SepPrune begins by analyzing the computational structure of a given model to identify layers with the highest computational burden. It then introduces a differentiable masking strategy to enable gradient-driven channel selection. Based on the learned masks, SepPrune prunes redundant channels and fine-tunes the remaining parameters to recover performance. Extensive experiments demonstrate that this learnable pruning paradigm yields substantial advantages for channel pruning in speech separation models, outperforming existing methods. Notably, a model pruned with SepPrune can recover 85% of the performance of a pre-trained model (trained over hundreds of epochs) with only one epoch of fine-tuning, and achieves convergence 36x faster than training from scratch.

PDF Details DOI

AAMAS Conference 2026 Conference Paper

Towards Foresighted AI Cooperators with LLM-driven Decision-Time Planning

  • Yuheng Jing
  • Kai Li
  • Bingyun Liu
  • Ziwen Zhang
  • Zhe Wu
  • Yifan Zhang
  • Junliang Xing
  • Jian Cheng

In multi-agent systems, building agents capable of seamlessly collaborating with unknown partners is a long-standing research goal. Existing approaches primarily generate a diverse population of partners and then train an agent against this population to master various cooperation conventions. However, these approaches are often hindered in two aspects: (1) They heavily rely on taskspecific training; (2) Their trained agents lack adaptability at test time. In this paper, we investigate how to leverage Large Language Models (LLMs) to build agents capable of foresighted coordination, addressing the challenges faced by existing work. To facilitate structured reasoning mechanisms, we introduce DTPAgent, a novel LLM-driven Decision-Time Planning (DTP) framework. Within this framework, LLMs, without relying on task-specific training and solely through in-context learning, estimate the partner policy and the transition-reward function to model the full dynamics of the environment. Based on these LLM-driven modelings, DTPAgent This work is licensed under a Creative Commons Attribution International 4. 0 License. Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026), C. Amato, L. Dennis, V. Mascardi, J. Thangarajah (eds.), May 25 – 29, 2026, Paphos, Cyprus. © 2026 International Foundation for Autonomous Agents and Multiagent Systems (www. ifaamas. org). https: //doi. org/10. 65109/XALP4331 simulates a range of possible trajectories to dynamically search for the most advantageous policy at each timestep. We demonstrate on the popular benchmark, Overcooked, that DTPAgent, built with small-scale LLMs, effectively outperforms various types of baselines when faced with unseen partners. Our DTPAgent also exhibits a scalable property that existing agents lack: the ability to translate test-time computation into improved performance.

PDF

AAAI Conference 2025 Conference Paper

An Open-Ended Learning Framework for Opponent Modeling

  • Yuheng Jing
  • Kai Li
  • Bingyun Liu
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Opponent Modeling (OM) aims to enhance decision-making by modeling other agents in multi-agent environments. Existing works typically learn opponent models against a pre-designated fixed set of opponents during training. However, this will cause poor generalization when facing unknown opponents during testing, as previously unseen opponents can exhibit out-of-distribution (OOD) behaviors that the learned opponent models cannot handle. To tackle this problem, we introduce a novel Open-Ended Opponent Modeling (OEOM) framework, which continuously generates opponents with diverse strengths and styles to reduce the possibility of OOD situations occurring during testing. Founded on population-based training and information-theoretic trajectory space diversity regularization, OEOM generates a dynamic set of opponents. This set is then fed to any OM approaches to train a potentially generalizable opponent model. Upon this, we further propose a simple yet effective OM approach that naturally fits within the OEOM framework. This approach is based on in-context reinforcement learning and learns a Transformer that dynamically recognizes and responds to opponents based on their trajectories. Extensive experiments in cooperative, competitive, and mixed environments demonstrate that OEOM is an approach-agnostic framework that improves generalizability compared to training against a fixed set of opponents, regardless of OM approaches or testing opponent settings. The results also indicate that our proposed approach generally outperforms existing OM baselines.

PDF Details DOI

IROS Conference 2025 Conference Paper

Collective Behavior Clone with Visual Attention via Neural Interaction Graph Prediction

  • Kai Li
  • Zhao Ma
  • Liang Li
  • Shiyu Zhao 0002

In this paper, we propose a framework, collective behavioral cloning (CBC), to learn the underlying interaction mechanism and control policy of a swarm system. Given the trajectory data of a swarm system, we propose a graph variational autoencoder (GVAE) to learn the local interaction graph. Based on the interaction graph and swarm trajectory, we use behavioral cloning to learn the control policy of the swarm system. To demonstrate the practicality of CBC, we deploy it on a real-world decentralized vision-based robot swarm system. A visual attention network is trained based on the learned interaction graph for online neighbor selection. Experimental results show that our method outperforms previous approaches in predicting both the interaction graph and swarm actions with higher accuracy. This work offers a promising approach for understanding interaction mechanisms and swarm dynamics in future swarm robotics research. Code and data are available 6.

Details

AAAI Conference 2025 Conference Paper

Critical Forgetting-Based Multi-Scale Disentanglement for Deepfake Detection

  • Kai Li
  • Wenqi Ren
  • Jianshu Li
  • Wei Wang
  • Xiaochun Cao

Recent face forgery detection methods based on disentangled representation learning utilize paired images for cross-reconstruction, aiming to extract forgery-relevant attributes and forgery-irrelevant content. However, there still exist the following issues that may comprise the detector performance: 1) using information-dense images as the decoupling targets increases the decoupling difficulty; 2) the extracted attribute features are reconstruction-irrelevant rather than forgery-relevant, and single-scale forgery representation decoupling cannot capture sufficient discriminative information; 3) the generalization performance of decoupled attribute features is poor as the detector focuses on learning specific artifact types in the training set. To address these issues, we propose a novel disentangled representation learning framework for deepfake detection. First, we extract features by partitioning the dense information within the image, focusing independently on texture, color, or edges. These features are then used as the decoupling targets rather than the images themselves, which could mitigate the decoupling difficulty. Second, we extend reconstruction loss from image-level to feature-level, thus extending the forgery representation decoupling from single-scale to multi-scale. Third, we propose a critical forgetting mechanism that forces the detector to forget the most salient features during training, which correspond to specific forgery artifact types in the training set. Extensive experimental results validate the efficacy of the proposed method.

PDF Details DOI

UAI Conference 2025 Conference Paper

Enhanced Equilibria-Solving via Private Information Pre-Branch Structure in Adversarial Team Games

  • Chen Qiu
  • Haobo Fu
  • Kai Li
  • Jiajia Zhang
  • Xuan Wang

In ex ante coordinated adversarial team games (ATGs), a team competes against an adversary, and team members can only coordinate their strategies before the game starts. The team-maxmin equilibrium with correlation (TMECor) is a suitable solution concept for extensive-form sequential ATGs. One class of TMECor-solving methods transforms the problem into solving NE in two-player zero-sum games, leveraging well-established tools for the latter. However, existing methods are fundamentally action-based, resulting in poor generalizability and low solving efficiency due to the exponential growth in the size of the transformed game. To address the above issues, we propose an efficient game transformation method based on private information, where all team members are represented by a single coordinator. We designed a structure called private information pre-branch, which makes decisions considering all possible private information from teammates. We prove that the size of the game transformed by our method is exponentially reduced compared to the current state-of-the-art. Moreover, we demonstrate equilibria equivalence. Experimentally, our method achieves a significant speedup of 182. 89$\times$ to 694. 44$\times$ in scenarios where the current state-of-the-art method can work, such as small-scale Kuhn poker and Leduc poker. Furthermore, our method is applicable to larger games and those with dynamically changing private information, such as Goofspiel.

Details

NeurIPS Conference 2025 Conference Paper

Learned Prefix Caching for Efficient LLM Inference

  • Dongsheng Yang
  • Austin Li
  • Kai Li
  • Wyatt Lloyd

Prefix caching is a key technique for reducing Large Language Model (LLM) inference costs. However, the prevalent least-recently-used (LRU) eviction algorithm has a large gap to the optimal algorithm. This paper introduces LPC, the first learned method to perform LLM prefix cache eviction. LPC leverages conversational content analysis to provide predictive guidance for eviction, determining which conversations are likely to continue. These insights, combined with last access timestamps, inform more effective cache management. Extensive evaluations across three real-world datasets demonstrate that LPC achieves 18-47% reductions in required cache sizes for equivalent hit ratios and has an 11% improvement in LLM prefilling throughput in an emulated environment.

PDF Details

NeurIPS Conference 2025 Conference Paper

MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix

  • Ziyang Ma
  • Yinghao Ma
  • Yanqiao Zhu
  • Chen Yang
  • Yi-Wen Chao
  • Ruiyang Xu
  • Wenxi Chen
  • Yuanzhe Chen

We introduce MMAR, a new benchmark designed to evaluate the deep reasoning capabilities of Audio-Language Models (ALMs) across massive multi-disciplinary tasks. MMAR comprises 1, 000 meticulously curated audio-question-answer triplets, collected from real-world internet videos and refined through iterative error corrections and quality checks to ensure high quality. Unlike existing benchmarks that are limited to specific domains of sound, music, or speech, MMAR extends them to a broad spectrum of real-world audio scenarios, including mixed-modality combinations of sound, music, and speech. Each question in MMAR is hierarchically categorized across four reasoning layers: Signal, Perception, Semantic, and Cultural, with additional sub-categories within each layer to reflect task diversity and complexity. To further foster research in this area, we annotate every question with a Chain-of-Thought (CoT) rationale to promote future advancements in audio reasoning. Each item in the benchmark demands multi-step deep reasoning beyond surface-level understanding. Moreover, a part of the questions requires graduate-level perceptual and domain-specific knowledge, elevating the benchmark's difficulty and depth. We evaluate MMAR using a broad set of models, including Large Audio-Language Models (LALMs), Large Audio Reasoning Models (LARMs), Omni Language Models (OLMs), Large Language Models (LLMs), and Large Reasoning Models (LRMs), with audio caption inputs. The performance of these models on MMAR highlights the benchmark's challenging nature, and our analysis further reveals critical limitations of understanding and reasoning capabilities among current models. These findings underscore the urgent need for greater research attention in audio-language reasoning, including both data and algorithm innovation. We hope MMAR will serve as a catalyst for future advances in this important but little-explored area.

PDF Details

AAAI Conference 2025 Conference Paper

MultiBooth: Towards Generating All Your Concepts in an Image from Text

  • Chenyang Zhu
  • Kai Li
  • Yue Ma
  • Chunming He
  • Xiu Li

This paper introduces MultiBooth, a method that generates images from texts containing various concepts from users.Despite diffusion models bringing significant advancements for customized text-to-image generation, existing methods often struggle with multi-concept scenarios due to low concept fidelity and high inference cost. MultiBooth addresses these issues by dividing the multi-concept generation process into two phases: a single-concept learning phase and a multi-concept integration phase. During the single-concept learning phase, we employ a multi-modal image encoder and an efficient concept encoding technique to learn a concise and discriminative representation for each concept. In the multi-concept integration phase, we use bounding boxes to define the generation area for each concept within the cross-attention map. This method enables the creation of individual concepts within their specified regions, thereby facilitating the formation of multi-concept images. This strategy not only improves concept fidelity but also reduces additional inference cost. MultiBooth surpasses various baselines in both qualitative and quantitative evaluations, showcasing its superior performance and computational efficiency.

PDF Details DOI

YNICL Journal 2025 Journal Article

The relationship between balance and visuospatial attention on hemispheric stroke survivors: A study of egocentric and allocentric neural processing

  • Shijue Li
  • Kai Li
  • Ziyan Huang
  • Zhenwen Liang
  • Huaqing Chen
  • Yongping Zheng
  • Chuhuai Wang
  • Qiuhua Yu

BACKGROUND: Impaired balance and visuospatial attention are well-documented sequelae of stroke. However, the interplay between balance function and visuospatial attention, particularly within egocentric and allocentric reference frames, remains poorly understood. OBJECTIVE: This study aimed to elucidate the relationship between balance and visuospatial attention in stroke survivors with left- and right-hemisphere lesions and to investigate the underlying neural mechanisms. METHODS: Seventeen patients with right-hemisphere stroke, sixteen with left-hemisphere stroke, and eighteen age-matched healthy controls participated in this study. Balance function was evaluated using Prokin, while visuospatial attention was assessed through tasks involving egocentric and allocentric reference frames. In addition, event-related potentials of the P1, N1, and P2 components were measured during the attention tasks. RESULTS: Patients with left-hemisphere stroke exhibited superior balance and visuospatial attention performance compared to those with right-hemisphere stroke. Balance function was positively correlated with both egocentric and allocentric visuospatial attention performance in left-hemisphere stroke survivors. Attenuated P1 amplitudes and enhanced P2 amplitudes were observed during allocentric processing in right-hemisphere stroke survivors. The P2 amplitude at the O2 electrode was positively associated with medio-lateral velocity, ellipse area, and perimeter during balance tasks in left-hemisphere stroke survivors. CONCLUSIONS: Balance function is closely linked to selective attention and categorization processes in allocentric visuospatial tasks, particularly in patients with left-hemisphere stroke, suggesting that the right hemisphere may play an important role in mediating balance and visuospatial attention functions in the patients with mild to moderate stroke.

Details DOI

ICLR Conference 2025 Conference Paper

TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation

  • Mohan Xu
  • Kai Li
  • Guo Chen
  • Xiaolin Hu

In recent years, much speech separation research has focused primarily on improving model performance. However, for low-latency speech processing systems, high efficiency is equally important. Therefore, we propose a speech separation model with significantly reduced parameters and computational costs: Time-frequency Interleaved Gain Extraction and Reconstruction network (TIGER). TIGER leverages prior knowledge to divide frequency bands and compresses frequency information. We employ a multi-scale selective attention module to extract contextual features, while introducing a full-frequency-frame attention module to capture both temporal and frequency contextual information. Additionally, to more realistically evaluate the performance of speech separation models in complex acoustic environments, we introduce a dataset called EchoSet. This dataset includes noise and more realistic reverberation (e.g., considering object occlusions and material properties), with speech from two speakers overlapping at random proportions. Experimental results showed that models trained on EchoSet had better generalization ability than those trained on other datasets to the data collected in the physical world, which validated the practical value of the EchoSet. On EchoSet and real-world data, TIGER significantly reduces the number of parameters by 94.3% and the MACs by 95.3% while achieving performance surpassing state-of-the-art (SOTA) model TF-GridNet.

Details

EAAI Journal 2024 Journal Article

A reputation-aided negotiation mechanism for multi-agent society based on blockchain

  • Lei Niu
  • Qihang Cai
  • Kai Li
  • Fenghui Ren
  • Xinguo Yu

In an economic market, participants usually need to conduct negotiations to complete commercial trades. With the constant expansion of the economic scale, it has become much more difficult to select good candidates as the negotiation opponents among all participants with varied behaviors in a market. Moreover, economic markets often suffer from a monopoly problem, which creates barriers for competitors to enter the market and leading to unfair consumer practices. To overcome the above issues, this paper (1) proposes a reputation-aided negotiation mechanism for multiple participants in the market based on the blockchain technique and multi-agent system; and (2) implements a system that applies the proposed mechanism, which is a specific Artificial Intelligence (AI) application of the combination of multi-agent system and blockchain. These AI agents leverage sophisticated reputation systems to evaluate the reliability and historical behaviors of potential trade partners, thereby enriching the negotiation decision-making process. The integration of agents and reputation-based mechanisms into blockchain-based systems has the following significance: (1) market participants stand to benefit from streamlined and trustworthy negotiations; (2) pushing all participants in the market to proactively improve their behaviors to increase their profits and market shares; (3) the implementation of a novice protection method within this framework serves to counteract the detrimental effects of monopolies, fostering a more equitable and competitive market landscape for all participants. Overall, the fusion of reputation-aided negotiation mechanisms with implemented AI presents a promising approach to enhancing market dynamics, promoting fair competition, and incentivizing positive behaviors among participants in economic markets.

Details DOI

EAAI Journal 2024 Journal Article

An offer-generating strategy for multiple negotiations with mixed types of issues and issue interdependency

  • Kai Li
  • Lei Niu
  • Fenghui Ren
  • Xinguo Yu

Agent negotiation in multi-agent systems has been extensively studied, focusing on both theoretical and applied research. However, a limited number of studies have considered proposing an offer-generating strategy for agents to propose offers during the negotiation process in the multiple-negotiation situation where interdependency exist between a mixture of discrete issues and continuous issues across different negotiations. Especially, considering the above common real-life situation, there is little work of proposing such a strategy which is able to generate an approximately Pareto optimal solution. To address such challenges, this paper targets at multiple-negotiation scenarios involving interdependency between mixed types of issues across different negotiations. The contributions of this paper are threefold. Firstly, this paper addresses the research gap in mixed-type of issues in multiple negotiations. Secondly, the paper introduces a formalized negotiation model for multiple-negotiation scenarios, addressing both discrete and continuous issues, enabling automatic agents to obtain goal-aligned offers effectively. Thirdly, this paper introduces a Hybrid of PSO (Particle Swarm Optimization) and GA (Genetic Algorithm) Algorithm (i. e. , named as HPGA in this paper) as an offer-generating strategy to assist agents in achieving approximately Pareto optimization in multiple-negotiation scenarios. To support those claims, this paper presents an overall modeling framework, introduces the proposed offer-generation strategy, conducts a series of experiments to demonstrate the superiority of the proposed approach in this paper, and presents two realistic case studies. Overall, this research expands upon existing studies in agent-based negotiation by addressing the overlooked aspects of mixed types of issues and issue interdependency across multiple negotiations. The proposed modeling approach and offer-generation strategy contribute to the advancement of negotiation techniques in multi-agent systems.

Details DOI

AIJ Journal 2024 Journal Article

Automatically designing counterfactual regret minimization algorithms for solving imperfect-information games

  • Kai Li
  • Hang Xu
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing

Strategic decision-making in imperfect-information games is an important problem in artificial intelligence. Counterfactual regret minimization (CFR), a family of iterative algorithms, has been the workhorse for solving these types of games since its inception. In recent years, a series of novel CFR variants have been proposed, significantly improving the convergence rate of vanilla CFR. However, most of these new variants are hand-designed by researchers through trial and error, often based on different motivations, which generally requires a tremendous amount of effort and insight. This work proposes AutoCFR, a systematic framework that meta-learns novel CFR algorithms through evolution, easing the burden of manual algorithm design. We first design a search language that is rich enough to represent various CFR variants. We then exploit a scalable regularized evolution algorithm with a set of acceleration techniques to efficiently search over the combinatorial space of algorithms defined by this language. The learned novel CFR algorithm can generalize to new imperfect-information games not seen during training and performs on par with or better than existing state-of-the-art CFR variants. In addition to superior empirical performance, we also theoretically show that the learned algorithm converges to an approximate Nash equilibrium. Extensive experiments across diverse imperfect-information games highlight the scalability, extensibility, and generalizability of AutoCFR, establishing it as a general-purpose framework for solving imperfect-information games.

Details DOI

NeurIPS Conference 2024 Conference Paper

BitDelta: Your Fine-Tune May Only Be Worth One Bit

  • James Liu
  • Guangxuan Xiao
  • Kai Li
  • Jason D. Lee
  • Song Han
  • Tri Dao
  • Tianle Cai

Large Language Models (LLMs) are typically trained in two phases: pre-training on large internet-scale datasets, and fine-tuning for downstream tasks. Given the higher computational demand of pre-training, it is intuitive to assume that fine-tuning adds less new information to the model, and is thus more compressible. We explore this assumption by decomposing the weights of fine-tuned models into their pre-trained components and an additional delta. We introduce a simple method, BitDelta, which successfully quantizes this delta down to 1 bit without compromising performance. This interesting finding not only highlights the potential redundancy of information added during fine-tuning, but also has significant implications for the multi-tenant serving and multi-tenant storage of fine-tuned models. By enabling the use of a single high-precision base model accompanied by multiple 1-bit deltas, BitDelta dramatically reduces GPU memory requirements by more than 10x, thus reducing per-user generation latency by more than 10x in multi-tenant settings. We validate BitDelta through experiments across Llama-2, Mistral and MPT model families, and on models up to 70B parameters, showcasing minimal performance degradation in all tested settings.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Design and Analysis of Soft Hybrid-Driven Manipulator with Variable Stiffness and Multiple Motion Patterns

  • Xin Fu 0005
  • Daohui Zhang
  • Liyan Mo
  • Kai Li
  • Xingang Zhao

Soft manipulators offer the advantages of safety and adaptability. However, due to insufficient stiffness and single motion mode limitations, existing soft manipulators usually exhibit low load capacity and small working space. To address this problem, we propose a novel soft hybrid-driven manipulator with continuous stiffness control capability and multiple motion patterns (omnidirectional bending and extension). Furthermore, we develop kinematic and stiffness models based on the constant curvature assumption. The soft robot consists of a soft bellows actuator and inextensible rigid skeletons, which exhibit a high extension ratio and low drive pressure. With the antagonistic actuation of tendon-pulling and air-pushing, the robot can achieve independent control over stiffness and position in three-dimensional space. The performance associated with the designed soft hybrid-driven manipulator is experimentally verified. The robot can achieve an elongation of 198% and a maximum bending angle of up to 240°. The robot can also increase stiffness by increasing internal air pressure to resist deformation caused by external loads. Additionally, tracking experiments with various trajectories in space verify the accuracy of the kinematic model, which indicates that the soft manipulator can stabilize motion within a broad workspace.

Details

NeurIPS Conference 2024 Conference Paper

Efficient Multi-task Reinforcement Learning with Cross-Task Policy Guidance

  • Jinmin He
  • Kai Li
  • Yifan Zang
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Multi-task reinforcement learning endeavors to efficiently leverage shared information across various tasks, facilitating the simultaneous learning of multiple tasks. Existing approaches primarily focus on parameter sharing with carefully designed network structures or tailored optimization procedures. However, they overlook a direct and complementary way to exploit cross-task similarities: the control policies of tasks already proficient in some skills can provide explicit guidance for unmastered tasks to accelerate skills acquisition. To this end, we present a novel framework called Cross-Task Policy Guidance (CTPG), which trains a guide policy for each task to select the behavior policy interacting with the environment from all tasks' control policies, generating better training trajectories. In addition, we propose two gating mechanisms to improve the learning efficiency of CTPG: one gate filters out control policies that are not beneficial for guidance, while the other gate blocks tasks that do not necessitate guidance. CTPG is a general framework adaptable to existing parameter sharing approaches. Empirical evaluations demonstrate that incorporating CTPG with these approaches significantly enhances performance in manipulation and locomotion benchmarks.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Evaluating Copyright Takedown Methods for Language Models

  • Boyi Wei
  • Weijia Shi
  • Yangsibo Huang
  • Noah A. Smith
  • Chiyuan Zhang
  • Luke Zettlemoyer
  • Kai Li
  • Peter Henderson

Language models (LMs) derive their capabilities from extensive training on diverse data, including copyrighted material. These models can memorize and generate content similar to their training data, potentially risking legal issues like copyright infringement. Therefore, model creators are motivated to develop mitigation methods that prevent generating particular copyrighted content, an ability we refer to as copyright takedowns. This paper introduces the first evaluation of the feasibility and side effects of copyright takedowns for LMs. We propose CoTaEval, an evaluation framework to assess the effectiveness of copyright takedown methods, the impact on the model's ability to retain uncopyrightable factual knowledge from the copyrighted content, and how well the model maintains its general utility and efficiency. We examine several strategies, including adding system prompts, decoding-time filtering interventions, and unlearning approaches. Our findings indicate that no method excels across all metrics, showing significant room for research in this unique problem setting and indicating potential unresolved challenges for live policy proposals.

PDF Details DOI

EAAI Journal 2024 Journal Article

Fusion flow-enhanced graph pooling residual networks for Unmanned Aerial Vehicles surveillance in day and night dual visions

  • Alam Noor
  • Kai Li
  • Eduardo Tovar
  • Pei Zhang
  • Bo Wei

Recognizing unauthorized Unmanned Aerial Vehicles (UAVs) within designated no-fly zones throughout the day and night is of paramount importance, where the unauthorized UAVs pose a substantial threat to both civil and military aviation safety. However, recognizing UAVs day and night with dual-vision cameras is nontrivial, since red–green–blue (RGB) images suffer from a low detection rate under an insufficient light condition, such as on cloudy or stormy days, while black-and-white infrared (IR) images struggle to capture UAVs that overlap with the background at night. In this paper, we propose a new optical flow-assisted graph-pooling residual network (OF-GPRN), which significantly enhances the UAV detection rate in day and night dual visions. The proposed OF-GPRN develops a new optical fusion to remove superfluous backgrounds, which improves RGB/IR imaging clarity. Furthermore, OF-GPRN extends optical fusion by incorporating a graph residual split attention network and a feature pyramid, which refines the perception of UAVs, leading to a higher success rate in UAV detection. A comprehensive performance evaluation is conducted using a benchmark UAV catch dataset. The results indicate that the proposed OF-GPRN elevates the UAV mean average precision (mAP) detection rate to 87. 8%, marking a 17. 9% advancement compared to the residual graph neural network (ResGCN)-based approach.

Details DOI

IJCAI Conference 2024 Conference Paper

Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent

  • Hang Xu
  • Kai Li
  • Bingyun Liu
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Counterfactual regret minimization (CFR) is a family of algorithms for effectively solving imperfect-information games. It decomposes the total regret into counterfactual regrets, utilizing local regret minimization algorithms, such as Regret Matching (RM) or RM+, to minimize them. Recent research establishes a connection between Online Mirror Descent (OMD) and RM+, paving the way for an optimistic variant PRM+ and its extension PCFR+. However, PCFR+ assigns uniform weights for each iteration when determining regrets, leading to substantial regrets when facing dominated actions. This work explores minimizing weighted counterfactual regret with optimistic OMD, resulting in a novel CFR variant PDCFR+. It integrates PCFR+ and Discounted CFR (DCFR) in a principled manner, swiftly mitigating negative effects of dominated actions and consistently leveraging predictions to accelerate convergence. Theoretical analyses prove that PDCFR+ converges to a Nash equilibrium, particularly under distinct weighting schemes for regrets and average strategies. Experimental results demonstrate PDCFR+'s fast convergence in common imperfect-information games. The code is available at https: //github. com/rpSebastian/PDCFRPlus.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Multiagent Gumbel MuZero: Efficient Planning in Combinatorial Action Spaces

  • Xiaotian Hao
  • Jianye Hao
  • Chenjun Xiao
  • Kai Li
  • Dong Li
  • Yan Zheng

AlphaZero and MuZero have achieved state-of-the-art (SOTA) performance in a wide range of domains, including board games and robotics, with discrete and continuous action spaces. However, to obtain an improved policy, they often require an excessively large number of simulations, especially for domains with large action spaces. As the simulation budget decreases, their performance drops significantly. In addition, many important real-world applications have combinatorial (or exponential) action spaces, making it infeasible to search directly over all possible actions. In this paper, we extend AlphaZero and MuZero to learn and plan in more complex multiagent (MA) Markov decision processes, where the action spaces increase exponentially with the number of agents. Our new algorithms, MA Gumbel AlphaZero and MA Gumbel MuZero, respectively without and with model learning, achieve superior performance on cooperative multiagent control problems, while reducing the number of environmental interactions by up to an order of magnitude compared to model-free approaches. In particular, we significantly improve prior performance when planning with much fewer simulation budgets. The code and appendix are available at https://github.com/tjuHaoXiaotian/MA-MuZero.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routing

  • Jinmin He
  • Kai Li
  • Yifan Zang
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Multi-task reinforcement learning endeavors to accomplish a set of different tasks with a single policy. To enhance data efficiency by sharing parameters across multiple tasks, a common practice segments the network into distinct modules and trains a routing network to recombine these modules into task-specific policies. However, existing routing approaches employ a fixed number of modules for all tasks, neglecting that tasks with varying difficulties commonly require varying amounts of knowledge. This work presents a Dynamic Depth Routing (D2R) framework, which learns strategic skipping of certain intermediate modules, thereby flexibly choosing different numbers of modules for each task. Under this framework, we further introduce a ResRouting method to address the issue of disparate routing paths between behavior and target policies during off-policy training. In addition, we design an automatic route-balancing mechanism to encourage continued routing exploration for unmastered tasks without disturbing the routing of mastered ones. We conduct extensive experiments on various robotics manipulation tasks in the Meta-World benchmark, where D2R achieves state-of-the-art performance with significantly improved learning efficiency.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Opponent Modeling with In-context Search

  • Yuheng Jing
  • Bingyun Liu
  • Kai Li
  • Yifan Zang
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Opponent modeling is a longstanding research topic aimed at enhancing decision-making by modeling information about opponents in multi-agent environments. However, existing approaches often face challenges such as having difficulty generalizing to unknown opponent policies and conducting unstable performance. To tackle these challenges, we propose a novel approach based on in-context learning and decision-time search named Opponent Modeling with In-context Search (OMIS). OMIS leverages in-context learning-based pretraining to train a Transformer model for decision-making. It consists of three in-context components: an actor learning best responses to opponent policies, an opponent imitator mimicking opponent actions, and a critic estimating state values. When testing in an environment that features unknown non-stationary opponent agents, OMIS uses pretrained in-context components for decision-time search to refine the actor's policy. Theoretically, we prove that under reasonable assumptions, OMIS without search converges in opponent policy recognition and has good generalization properties; with search, OMIS provides improvement guarantees, exhibiting performance stability. Empirically, in competitive, cooperative, and mixed environments, OMIS demonstrates more effective and stable adaptation to opponents than other approaches. See our project website at https: //sites. google. com/view/nips2024-omis.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Real-world Image Dehazing with Coherence-based Pseudo Labeling and Cooperative Unfolding Network

  • Chengyu Fang
  • Chunming He
  • Fengyang Xiao
  • Yulun Zhang
  • Longxiang Tang
  • Yuelin Zhang
  • Kai Li
  • Xiu Li

Real-world Image Dehazing (RID) aims to alleviate haze-induced degradation in real-world settings. This task remains challenging due to the complexities in accurately modeling real haze distributions and the scarcity of paired real-world data. To address these challenges, we first introduce a cooperative unfolding network that jointly models atmospheric scattering and image scenes, effectively integrating physical knowledge into deep networks to restore haze-contaminated details. Additionally, we propose the first RID-oriented iterative mean-teacher framework, termed the Coherence-based Label Generator, to generate high-quality pseudo labels for network training. Specifically, we provide an optimal label pool to store the best pseudo-labels during network training, leveraging both global and local coherence to select high-quality candidates and assign weights to prioritize haze-free regions. We verify the effectiveness of our method, with experiments demonstrating that it achieves state-of-the-art performance on RID tasks. Code will be available at https: //github. com/cnyvfang/CORUN-Colabator.

PDF Details DOI

IROS Conference 2024 Conference Paper

Uncertainty-Aware Semi-Supervised Semantic Key Point Detection via Bundle Adjustment

  • Kai Li
  • Yin Zhang
  • Shiyu Zhao 0002

Visual relative localization is widely used in multi-robot systems. While semantic key points offer a promising solution for 6DoF pose estimation, manual data labeling for network training remains unavoidable. In this paper, we introduce a novel method that jointly estimates the semantic key point detection model and 6DoF camera pose. Our key idea is to leverage the 3D-2D projection to produce pseudo labels for detection model training while taking the key point predictions as landmarks for 6DoF camera pose estimation. Compared with state-of-the-art works, our method eliminates the need for calibration and time synchronization of multi-camera systems, requiring only a handful of manually labeled data, which significantly improves the training efficiency. The experiment validates the effectiveness and practicality of our method in public datasets and real-world robotic applications. Code and data are made available 3.

Details

AAAI Conference 2023 Conference Paper

Adversarial Alignment for Source Free Object Detection

  • Qiaosong Chu
  • Shuyan Li
  • Guangyi Chen
  • Kai Li
  • Xiu Li

Source-free object detection (SFOD) aims to transfer a detector pre-trained on a label-rich source domain to an unlabeled target domain without seeing source data. While most existing SFOD methods generate pseudo labels via a source-pretrained model to guide training, these pseudo labels usually contain high noises due to heavy domain discrepancy. In order to obtain better pseudo supervisions, we divide the target domain into source-similar and source-dissimilar parts and align them in the feature space by adversarial learning.Specifically, we design a detection variance-based criterion to divide the target domain. This criterion is motivated by a finding that larger detection variances denote higher recall and larger similarity to the source domain. Then we incorporate an adversarial module into a mean teacher framework to drive the feature spaces of these two subsets indistinguishable. Extensive experiments on multiple cross-domain object detection datasets demonstrate that our proposed method consistently outperforms the compared SFOD methods. Our implementation is available at https://github.com/ChuQiaosong.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Automatic Grouping for Efficient Cooperative Multi-Agent Reinforcement Learning

  • Yifan Zang
  • Jinmin He
  • Kai Li
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing
  • Jian Cheng

Grouping is ubiquitous in natural systems and is essential for promoting efficiency in team coordination. This paper proposes a novel formulation of Group-oriented Multi-Agent Reinforcement Learning (GoMARL), which learns automatic grouping without domain knowledge for efficient cooperation. In contrast to existing approaches that attempt to directly learn the complex relationship between the joint action-values and individual utilities, we empower subgroups as a bridge to model the connection between small sets of agents and encourage cooperation among them, thereby improving the learning efficiency of the whole team. In particular, we factorize the joint action-values as a combination of group-wise values, which guide agents to improve their policies in a fine-grained fashion. We present an automatic grouping mechanism to generate dynamic groups and group action-values. We further introduce a hierarchical control for policy learning that drives the agents in the same group to specialize in similar policies and possess diverse strategies for various groups. Experiments on the StarCraft II micromanagement tasks and Google Research Football scenarios verify our method's effectiveness. Extensive component studies show how grouping works and enhances performance.

PDF Details

EAAI Journal 2023 Journal Article

Hybrid partial-constrained learning with orthogonality regularization for unsupervised person re-identification

  • Jiazuo Yu
  • Jinjia Peng
  • Kai Li
  • Huibing Wang

Person re-identification (re-ID) aims at determining whether there is a specific person in image sets or videos via computer vision technology. State-of-the-art unsupervised re-ID methods extract image features through CNNs-based networks and store these extracted features in memory for identity matching. However, extracted global features of these methods ignore the problem of information redundancy and the influence of the constraints between the internal features. To overcome these problems, a Hybrid Partial-constrained Learning (HPcL) network with orthogonality regularization is proposed to learn a discriminative visual representation by generating hybrid features. Specifically, the hybrid features are generated by our designed Dynamic Fusion Module (DFM) to initialize the memory dictionary and match the identity, which can constrain each part of the features extracted by our proposed Multi-Scale (M-S) module and learn robust visual representations. In addition, a new orthogonal regularization method is introduced to constrain orthogonality of the kernel weights and features, which reduces the correlations among features. Extensive experimental results on Market-1501, DukeMTMC-reID, PersonX, and MSMT17 datasets demonstrate that our method is effective and superior to the state-of-the-art methods.

Details DOI

AAMAS Conference 2023 Conference Paper

Sequential Cooperative Multi-Agent Reinforcement Learning

  • Yifan Zang
  • Jinmin He
  • Kai Li
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing

Cooperative multi-agent reinforcement learning (MARL) aims to coordinate the actions of multiple agents via a shared team reward. The complex interactions among agents make this problem extremely difficult. The mainstream of MARL methods often implicitly learn an inexplicable value decomposition from the shared reward into individual utilities, failing to give insights into how well each agent acts and lacking direct policy optimization guidance. This paper presents a sequential MARL framework that factorizes and simplifies the complex interaction analysis into a sequential evaluation process for more effective and efficient learning. We explicitly formulate this factorization via a novel sequential advantage function to evaluate each agent’s actions, which achieves an explicable credit assignment and substantially facilitates policy optimization. We realize the sequential credit assignment (SeCA) by dynamically adjusting the sequence in light of agents’ contributions to the team. Extensive experimental validations on a challenging set of StarCraft II micromanagement tasks verify SeCA’s effectiveness.

PDF

NeurIPS Conference 2023 Conference Paper

Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping

  • Chunming He
  • Kai Li
  • Yachao Zhang
  • Guoxia Xu
  • Longxiang Tang
  • Yulun Zhang
  • Zhenhua Guo
  • Xiu Li

Weakly-Supervised Concealed Object Segmentation (WSCOS) aims to segment objects well blended with surrounding environments using sparsely-annotated data for model training. It remains a challenging task since (1) it is hard to distinguish concealed objects from the background due to the intrinsic similarity and (2) the sparsely-annotated training data only provide weak supervision for model learning. In this paper, we propose a new WSCOS method to address these two challenges. To tackle the intrinsic similarity challenge, we design a multi-scale feature grouping module that first groups features at different granularities and then aggregates these grouping results. By grouping similar features together, it encourages segmentation coherence, helping obtain complete segmentation results for both single and multiple-object images. For the weak supervision challenge, we utilize the recently-proposed vision foundation model, ``Segment Anything Model (SAM)'', and use the provided sparse annotations as prompts to generate segmentation masks, which are used to train the model. To alleviate the impact of low-quality segmentation masks, we further propose a series of strategies, including multi-augmentation result ensemble, entropy-based pixel-level weighting, and entropy-based image-level selection. These strategies help provide more reliable supervision to train the segmentation model. We verify the effectiveness of our method on various WSCOS tasks, and experiments demonstrate that our method achieves state-of-the-art performance on these tasks.

PDF Details

AAAI Conference 2022 Conference Paper

AlphaHoldem: High-Performance Artificial Intelligence for Heads-Up No-Limit Poker via End-to-End Reinforcement Learning

  • Enmin Zhao
  • Renye Yan
  • Jinqiu Li
  • Kai Li
  • Junliang Xing

Heads-up no-limit Texas hold’em (HUNL) is the quintessential game with imperfect information. Representative prior works like DeepStack and Libratus heavily rely on counterfactual regret minimization (CFR) and its variants to tackle HUNL. However, the prohibitive computation cost of CFR iteration makes it difficult for subsequent researchers to learn the CFR model in HUNL and apply it in other practical applications. In this work, we present AlphaHoldem, a highperformance and lightweight HUNL AI obtained with an endto-end self-play reinforcement learning framework. The proposed framework adopts a pseudo-siamese architecture to directly learn from the input state information to the output actions by competing the learned model with its different historical versions. The main technical contributions include a novel state representation of card and betting information, a multi-task self-play training loss function, and a new model evaluation and selection metric to generate the final model. In a study involving 100, 000 hands of poker, AlphaHoldem defeats Slumbot and DeepStack using only one PC with three days training. At the same time, AlphaHoldem only takes 2. 9 milliseconds for each decision-making using only a single GPU, more than 1, 000 times faster than DeepStack. We release the history data among among AlphaHoldem, Slumbot, and top human professionals in the author’s GitHub repository to facilitate further studies in this direction.

PDF Details

AAAI Conference 2022 Conference Paper

AutoCFR: Learning to Design Counterfactual Regret Minimization Algorithms

  • Hang Xu
  • Kai Li
  • Haobo Fu
  • Qiang Fu
  • Junliang Xing

Counterfactual regret minimization (CFR) is the most commonly used algorithm to approximately solving two-player zero-sum imperfect-information games (IIGs). In recent years, a series of novel CFR variants such as CFR+, Linear CFR, DCFR have been proposed and have significantly improved the convergence rate of the vanilla CFR. However, most of these new variants are hand-designed by researchers through trial and error based on different motivations, which generally requires a tremendous amount of efforts and insights. This work proposes to meta-learn novel CFR algorithms through evolution to ease the burden of manual algorithm design. We first design a search language that is rich enough to represent many existing hand-designed CFR variants. We then exploit a scalable regularized evolution algorithm with a bag of acceleration techniques to efficiently search over the combinatorial space of algorithms defined by this language. The learned novel CFR algorithm can generalize to new IIGs not seen during training and performs on par with or better than existing state-of-the-art CFR variants. The code is available at https: //github. com/rpSebastian/AutoCFR.

PDF Details

JBHI Journal 2022 Journal Article

Gating Attractor Dynamics of Frontal Cortex Under Acupuncture via Recurrent Neural Network

  • Kai Li
  • Jiang Wang
  • Zhicai Hu
  • Bin Deng
  • Haitao Yu

Acupuncture can regulate the functions of human body and improve the cognition of brain. However, the mechanism of acupuncture manipulations remains unclear. Here, we hypothesis that the frontal cortex plays a gating role in information routing of brain network under acupuncture. To that end, the gating effect of frontal cortex under acupuncture is analyzed in combination with EEG data of acupuncture at Zusanli acupoints. In addition, recurrent neural network (RNN) is used to reproduce the dynamics of frontal cortex under normal state and acupuncture state. From low-dimensional view, it is shown that the brain networks under acupuncture state can show stable attractor cycle dynamics, which may explain the regulation effect of acupuncture. Comparing with different manipulations, we find that the attractor of low-dimensional trajectory varies under different frequencies of acupuncture. Besides, a strip gated band of neural dynamics is found by changing the frequency of stimulation and excitatory-inhibitory balance of network. This reverse engineering of brain network indicates that the differences among acupuncture manipulations are caused by interaction and separation in the neural activity space between attractors that encode acupuncture function. Consequently, our results may provide help for quantitative analysis of acupuncture, and benefit for the clinical guidance of acupuncture clinicians.

Details DOI

AAMAS Conference 2022 Conference Paper

Multiagent Q-learning with Sub-Team Coordination

  • Wenhan Huang
  • Kai Li
  • Kun Shao
  • Tianze Zhou
  • Jun Luo
  • Dongge Wang
  • Hangyu Mao
  • Jianye Hao

For cooperative mutliagent reinforcement learning tasks, we propose a novel value factorization framework in the popular centralized training with decentralized execution paradigm, called multiagent Q-learning with sub-team coordination (QSCAN). This framework could flexibly exploit local coordination within sub-teams for effective factorization while honoring the individual-globalmax (IGM) condition. QSCAN encompasses the full spectrum of sub-team coordination according to sub-team size, ranging from the monotonic value function class to the entire IGM function class, with familiar methods such as QMIX and QPLEX located at the respective extremes of the spectrum. Empirical results show that QSCAN’s performance dominates state-of-the-art methods in predator-prey tasks and the Switch challenge in MA-Gym.

PDF

NeurIPS Conference 2022 Conference Paper

Multiagent Q-learning with Sub-Team Coordination

  • Wenhan Huang
  • Kai Li
  • Kun Shao
  • Tianze Zhou
  • Matthew Taylor
  • Jun Luo
  • Dongge Wang
  • Hangyu Mao

In many real-world cooperative multiagent reinforcement learning (MARL) tasks, teams of agents can rehearse together before deployment, but then communication constraints may force individual agents to execute independently when deployed. Centralized training and decentralized execution (CTDE) is increasingly popular in recent years, focusing mainly on this setting. In the value-based MARL branch, credit assignment mechanism is typically used to factorize the team reward into each individual’s reward — individual-global-max (IGM) is a condition on the factorization ensuring that agents’ action choices coincide with team’s optimal joint action. However, current architectures fail to consider local coordination within sub-teams that should be exploited for more effective factorization, leading to faster learning. We propose a novel value factorization framework, called multiagent Q-learning with sub-team coordination (QSCAN), to flexibly represent sub-team coordination while honoring the IGM condition. QSCAN encompasses the full spectrum of sub-team coordination according to sub-team size, ranging from the monotonic value function class to the entire IGM function class, with familiar methods such as QMIX and QPLEX located at the respective extremes of the spectrum. Experimental results show that QSCAN’s performance dominates state-of-the-art methods in matrix games, predator-prey tasks, the Switch challenge in MA-Gym. Additionally, QSCAN achieves comparable performances to those methods in a selection of StarCraft II micro-management tasks.

PDF Details

NeurIPS Conference 2022 Conference Paper

Recovering Private Text in Federated Learning of Language Models

  • Samyak Gupta
  • Yangsibo Huang
  • Zexuan Zhong
  • Tianyu Gao
  • Kai Li
  • Danqi Chen

Federated learning allows distributed users to collaboratively train a model while keeping each user’s data private. Recently, a growing body of work has demonstrated that an eavesdropping attacker can effectively recover image data from gradients transmitted during federated learning. However, little progress has been made in recovering text data. In this paper, we present a novel attack method FILM for federated learning of language models (LMs). For the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences. Unlike image-recovery methods that are optimized to match gradients, we take a distinct approach that first identifies a set of words from gradients and then directly reconstructs sentences based on beam search and a prior-based reordering strategy. We conduct the FILM attack on several large-scale datasets and show that it can successfully reconstruct single sentences with high fidelity for large batch sizes and even multiple sentences if applied iteratively. We evaluate three defense methods: gradient pruning, DPSGD, and a simple approach to freeze word embeddings that we propose. We show that both gradient pruning and DPSGD lead to a significant drop in utility. However, if we fine-tune a public pre-trained LM on private text without updating word embeddings, it can effectively defend the attack with minimal data utility loss. Together, we hope that our results can encourage the community to rethink the privacy concerns of LM training and its standard practices in the future. Our code is publicly available at https: //github. com/Princeton-SysML/FILM.

PDF Details

YNIMG Journal 2022 Journal Article

RT-Cloud: A cloud-based software framework to simplify and standardize real-time fMRI

  • Grant Wallace
  • Stephen Polcyn
  • Paula P. Brooks
  • Anne C. Mennen
  • Ke Zhao
  • Paul S. Scotti
  • Sebastian Michelmann
  • Kai Li

Real-time fMRI (RT-fMRI) neurofeedback has been shown to be effective in treating neuropsychiatric disorders and holds tremendous promise for future breakthroughs, both with regard to basic science and clinical applications. However, the prevalence of its use has been hampered by computing hardware requirements, the complexity of setting up and running an experiment, and a lack of standards that would foster collaboration. To address these issues, we have developed RT-Cloud (https://github.com/brainiak/rt-cloud), a flexible, cloud-based, open-source Python software package for the execution of RT-fMRI experiments. RT-Cloud uses standardized data formats and adaptable processing streams to support and expand open science in RT-fMRI research and applications. Cloud computing is a key enabling technology for advancing RT-fMRI because it eliminates the need for on-premise technical expertise and high-performance computing; this allows installation, configuration, and maintenance to be automated and done remotely. Furthermore, the scalability of cloud computing makes it easier to deploy computationally-demanding multivariate analyses in real time. In this paper, we describe how RT-Cloud has been integrated with open standards, including the Brain Imaging Data Structure (BIDS) standard and the OpenNeuro database, how it has been applied thus far, and our plans for further development and deployment of RT-Cloud in the coming years.

Details DOI

NeurIPS Conference 2021 Conference Paper

Evaluating Gradient Inversion Attacks and Defenses in Federated Learning

  • Yangsibo Huang
  • Samyak Gupta
  • Zhao Song
  • Kai Li
  • Sanjeev Arora

Gradient inversion attack (or input recovery from gradient) is an emerging threat to the security and privacy preservation of Federated learning, whereby malicious eavesdroppers or participants in the protocol can recover (partially) the clients' private data. This paper evaluates existing attacks and defenses. We find that some attacks make strong assumptions about the setup. Relaxing such assumptions can substantially weaken these attacks. We then evaluate the benefits of three proposed defense mechanisms against gradient inversion attacks. We show the trade-offs of privacy leakage and data utility of these defense methods, and find that combining them in an appropriate manner makes the attack less effective, even under the original strong assumptions. We also estimate the computation cost of end-to-end recovery of a single image under each evaluated defense. Our findings suggest that the state-of-the-art attacks can currently be defended against with minor data utility loss, as summarized in a list of potential strategies.

PDF Details

AAAI Conference 2021 Conference Paper

Exploration via State influence Modeling

  • Yongxin Kang
  • Enmin Zhao
  • Kai Li
  • Junliang Xing

This paper studies the challenging problem of reinforcement learning (RL) in hard exploration tasks with sparse rewards. It focuses on the exploration stage before the agent gets the first positive reward, in which case, traditional RL algorithms with simple exploration strategies often work poorly. Unlike previous methods using some attribute of a single state as the intrinsic reward to encourage exploration, this work leverages the social influence between different states to permit more efficient exploration. It introduces a general intrinsic reward construction method to evaluate the social influence of states dynamically. Three kinds of social influence are introduced for a state: conformity, power, and authority. By measuring the state influence, agents quickly find the focus state during the exploration process. The proposed RL framework with state influence evaluation works well in hard exploration task. Extensive experimental analyses and comparisons in Grid Maze and many hard exploration Atari 2600 games demonstrate its high exploration efficiency.

PDF Details

NeurIPS Conference 2021 Conference Paper

Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

  • Xiaolin Hu
  • Kai Li
  • Weiyi Zhang
  • Yi Luo
  • Jean-Marie Lemercier
  • Timo Gerkmann

Recent advances in the design of neural network architectures, in particular those specialized in modeling sequences, have provided significant improvements in speech separation performance. In this work, we propose to use a bio-inspired architecture called Fully Recurrent Convolutional Neural Network (FRCNN) to solve the separation task. This model contains bottom-up, top-down and lateral connections to fuse information processed at various time-scales represented by stages. In contrast to the traditional approach updating stages in parallel, we propose to first update the stages one by one in the bottom-up direction, then fuse information from adjacent stages simultaneously and finally fuse information from all stages to the bottom stage together. Experiments showed that this asynchronous updating scheme achieved significantly better results with much fewer parameters than the traditional synchronous updating scheme on speech separation. In addition, the proposed model achieved competitive or better results with high efficiency as compared to other state-of-the-art approaches on two benchmark datasets.

PDF Details

AIIM Journal 2020 Journal Article

Optimized artificial neural network based performance analysis of wheelchair movement for ALS patients

  • Kai Li
  • S. Ramkumar
  • J. Thimmiaraja
  • S. Diwakaran

Individuals with neurodegenerative attacks loose the entire motor neuron movements. These conditions affect the individual actions like walking, speaking impairment and totally make the person in to locked in state (LIS). To overcome the miserable condition the person need rehabilitation devices through a Brain Computer Interfaces (BCI) to satisfy their needs. BMI using Electroencephalogram (EEG) receives the mental thoughts from brain and converts into control signals to activate the exterior communication appliances in the absence of biological channels. To design the BCI, we conduct our study with three normal male subjects, three normal female subjects and three ALS affected individuals from the age of 20–60 with three electrode systems for four tasks. One Dimensional Local Binary Patterns (LBP) technique was applied to reduce the digitally sampled features collected from nine subjects was treated with Grey wolf optimization Neural Network (GWONN) to classify the mentally composed words. Using these techniques, we compared the three types of subjects to identify the performances. The study proves that subjects from normal male categories performance was maximum compared with the other subjects. To assess the individual performance of the subject, we conducted the recognition accuracy test in offline mode. From the accuracy test also, we obtained the best performance from the normal male subjects compared with female and ALS subjects with an accuracy of 98. 33 %, 95. 00 % and 88. 33 %. Finally our study concludes that patients with ALS attack need more training than that of the other subjects.

Details DOI

IJCAI Conference 2020 Conference Paper

Potential Driven Reinforcement Learning for Hard Exploration Tasks

  • Enmin Zhao
  • Shihong Deng
  • Yifan Zang
  • Yongxin Kang
  • Kai Li
  • Junliang Xing

Experience replay plays a crucial role in Reinforcement Learning (RL), enabling the agent to remember and reuse experience from the past. Most previous methods sample experience transitions using simple heuristics like uniformly sampling or prioritizing those good ones. Since humans can learn from both good and bad experiences, more sophisticated experience replay algorithms need to be developed. Inspired by the potential energy in physics, this work introduces the artificial potential field into experience replay and develops Potentialized Experience Replay (PotER) as a new and effective sampling algorithm for RL in hard exploration tasks with sparse rewards. PotER defines a potential energy function for each state in experience replay and helps the agent to learn from both good and bad experiences using intrinsic state supervision. PotER can be combined with different RL algorithms as well as the self-imitation learning algorithm. Experimental analyses and comparisons on multiple challenging hard exploration environments have verified its effectiveness and efficiency.

PDF Details DOI

IROS Conference 2020 Conference Paper

Vision Global Localization with Semantic Segmentation and Interest Feature Points

  • Kai Li
  • Xudong Zhang
  • Kun Li
  • Shuo Zhang

In this work, we present a vision-only global localization architecture for autonomous vehicle applications, and achieves centimeter-level accuracy and high robustness in various scenarios. We first apply pixel-wise segmentation to the front-view mono camera and extract the semantic features, e. g. pole-like objects, lane markings, and curbs, which are robust to illumination, viewing angles and seasonal changes. For the scenes without enough semantic information, we extract interest feature points on static backgrounds, such as ground surface and buildings, assisted by our semantic segmentation. We create the visual global map with semantic feature map layers extracted from LiDAR point-cloud semantic map and the point feature map layer built with a fixed-pose SFM. A lumped Levenberg-Marquardt optimization solver is then applied to minimize the cost from two types of observations. We further evaluate the accuracy and robustness of our method with road tests on Alibaba's autonomous delivery vehicles in multiple scenarios as well as a KAIST urban dataset.

Details

AAAI Conference 2019 Conference Paper

Cooperation Enforcement and Collusion Resistance in Repeated Public Goods Games

  • Kai Li
  • Dong Hao

Enforcing cooperation among substantial agents is one of the main objectives for multi-agent systems. However, due to the existence of inherent social dilemmas in many scenarios, the free-rider problem may arise during agents’ long-run interactions and things become even severer when self-interested agents work in collusion with each other to get extra benefits. It is commonly accepted that in such social dilemmas, there exists no simple strategy for an agent whereby she can simultaneously manipulate on the utility of each of her opponents and further promote mutual cooperation among all agents. Here, we show that such strategies do exist. Under the conventional repeated public goods game, we novelly identify them and find that, when confronted with such strategies, a single opponent can maximize his utility only via global cooperation and any colluding alliance cannot get the upper hand. Since a full cooperation is individually optimal for any single opponent, a stable cooperation among all players can be achieved. Moreover, we experimentally show that these strategies can still promote cooperation even when the opponents are both self-learning and collusive.

PDF Details

AAAI Conference 2018 Conference Paper

Discriminative Semi-Coupled Projective Dictionary Learning for Low-Resolution Person Re-Identification

  • Kai Li
  • Zhengming Ding
  • Sheng Li
  • Yun Fu

Person re-identification (re-ID) is a fundamental task in automated video surveillance. In real-world visual surveillance systems, a person is often captured in quite low resolutions. So we often need to perform low-resolution person re-ID, where images captured by different cameras have great resolution divergences. Existing methods cope problem via some complicated and time-consuming strategies, making them less favorable in practice, and their performances are far from satisfactory. In this paper, we design a novel Discriminative Semi-coupled Projective Dictionary Learning (DSPDL) model to effectively and efficiently solve this problem. Specifically, we propose to jointly learn a pair of dictionaries and a mapping to bridge the gap across low(er) and high(er) resolution person images. Besides, we develop a novel graph regularizer to incorporate positive and negative image pair information in a parameterless fashion. Meanwhile, we adopt the efficient and powerful projective dictionary learning technique to boost the our efficiency. Experiments on three public datasets show the superiority of the proposed method to the state-of-the-art ones.

PDF Details

AAAI Conference 2018 Conference Paper

Latent Discriminant Subspace Representations for Multi-View Outlier Detection

  • Kai Li
  • Sheng Li
  • Zhengming Ding
  • Weidong Zhang
  • Yun Fu

Identifying multi-view outliers is challenging because of the complex data distributions across different views. Existing methods cope this problem by exploiting pairwise constraints across different views to obtain new feature representations, based on which certain outlier score measurements are de- fined. Due to the use of pairwise constraint, it is complicated and time-consuming for existing methods to detect outliers from three or more views. In this paper, we propose a novel method capable of detecting outliers from any number of data views. Our method first learns latent discriminant representations for all view data and defines a novel outlier score function based on the latent discriminant representations. Specifically, we represent multi-view data by a global low-rank representation shared by all views and residual representations specific to each view. Through analyzing the view-specific residual representations of all views, we can get the outlier score for every sample. Moreover, we raise the problem of detecting a third type of multi-view outliers which are neglected by existing methods. Experiments on six datasets show our method outperforms the existing ones in identifying all types of multi-view outliers, often by large margins.

PDF Details

IJCAI Conference 2018 Conference Paper

Payoff Control in the Iterated Prisoner's Dilemma

  • Dong Hao
  • Kai Li
  • Tao Zhou

Repeated game has long been the touchstone model for agents’ long-run relationships. Previous results suggest that it is particularly difficult for a repeated game player to exert an autocratic control on the payoffs since they are jointly determined by all participants. This work discovers that the scale of a player’s capability to unilaterally influence the payoffs may have been much underestimated. Under the conventional iterated prisoner’s dilemma, we develop a general framework for controlling the feasible region where the players’ payoff pairs lie. A control strategy player is able to confine the payoff pairs in her objective region, as long as this region has feasible linear boundaries. With this framework, many well-known existing strategies can be categorized and various new strategies with nice properties can be further identified. We show that the control strategies perform well either in a tournament or against a human-like opponent.

PDF Details

TARK Conference 2017 Conference Paper

Conditional Belief, Knowledge and Probability

  • Jan van Eijck
  • Kai Li

A natural way to represent beliefs and the process of updating beliefs is presented by Bayesian probability theory, where belief of an agent a in P can be interpreted as a considering that P is more probable than not P. This paper attempts to get at the core logical notion underlying this. The paper presents a sound and complete neighbourhood logic for conditional belief and knowledge, and traces the connections with probabilistic logics of belief and knowledge. The key notion in this paper is that of an agent a believing P conditionally on having information Q, where it is assumed that Q is compatible with what a knows. Conditional neighbourhood logic can be viewed as a core system for reasoning about subjective plausibility that is not yet committed to an interpretation in terms of numerical probability. Indeed, every weighted Kripke model gives rise to a conditional neighbourhood model, but not vice versa. We show that our calculus for conditional neighbourhood logic is sound but not complete for weighted Kripke models. Next, we show how to extend the calculus to get completeness for the class of weighted Kripke models. Neighbourhood models for conditional belief are closed under model restriction (public announcement update), while earlier neighbourhood models for belief as `willingness to bet' were not. Therefore the logic we present improves on earlier neighbourhood logics for belief and knowledge. We present complete calculi for public announcement and for publicly revealing the truth value of propositions using reduction axioms. The reductions show that adding these announcement operators to the language does not increase expressive power.

Details DOI

ICRA Conference 2001 Conference Paper

Approximate Jacobian Feedback Control of Robots with Kinematic Uncertainty and its Application to Visual Servoing

  • Chien Chern Cheah
  • Kai Li
  • Suguru Arimoto
  • Sadao Kawamura

Most researches so far on robot control have assumed that the exact kinematics and Jacobian matrix of the manipulator from joint space to Cartesian space are known. Unfortunately, no physical parameters could be derived exactly. In addition, the robot is required to interact with its environment and hence the overall parameters would change according to different tasks. In the paper, simple feedback control laws are proposed for setpoint control of robots with uncertain kinematics and dynamics. We show that the end-effector's position converges to a desired position in a finite task space even when the kinematics is uncertain.

Details
v2026.09.13