Author name cluster

Kai Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

97 papers

2 author rows

EAAI Journal 2026 Journal Article

A cross-scale interaction framework combining Mamba and Convolutional Neural Networks for Arbitrary-Scale Super-Resolution of infrared images

Feiwei Qin
Xinyu Cao
Changmiao Wang
Kai Zhang
Yong Peng
Jing Bai

Infrared image super-resolution (SR) is a critical technique in aerospace and remote sensing. However, current methods frequently ignore the continuity of real-world signals in favor of discrete representations of infrared images. As a result, these methods necessitate training a separate model for each scale, leading to inefficient training and increased storage requirements. To overcome this limitation, we propose a Mamba-Convolutional Neural Network Cross-Scale Interaction Arbitrary-Scale Super-Resolution (MCASSR) framework, the first arbitrary-scale super-resolution method for infrared images capable of achieving SR across multiple continuous scales with a single model. Specifically, we design a Mamba-Convolutional Neural Network Mutual Learning Backbone (MCMLB) for deep feature extraction and a Compound Implicit Attention Upsampler (CIAU) for continuous upsampling. The MCMLB effectively integrates the long-range dependency modeling of Mamba and the local detail capturing of Convolutional Neural Network through bidirectional feature interaction. The CIAU enables adaptive continuous feature interpolation based on relative position and feature similarity while effectively leveraging cross-scale non-local texture priors in infrared images. Furthermore, we introduce a gradual residual decoder to capture high-frequency details in infrared images efficiently by alleviating the inherent spectral bias of multilayer perceptrons. Extensive experimental results demonstrate that MCASSR provides superior visual improvements across multiple infrared image datasets compared to current state-of-the-art methods.

Details DOI

TMLR Journal 2026 Journal Article

Beyond Affinity: A Benchmark of 1D, 2D, and 3D Methods Reveals Critical Trade-offs in Structure-Based Drug Design

Kangyu Zheng
Kai Zhang
Jiale Tan
Xuehan Chen
Yingzhou Lu
Zaixi Zhang
Lichao Sun
Marinka Zitnik

Currently, the field of structure-based drug design is dominated by three main types of algorithms: search-based algorithms, deep generative models, and reinforcement learning. While existing works have typically focused on comparing models within a single algorithmic category, cross-algorithm comparisons remain scarce. In this paper, to fill the gap, we establish a benchmark to evaluate the performance of fifteen models across these different algorithmic foundations by assessing the pharmaceutical properties of the generated molecules and their docking affinities and poses with specified target proteins. We highlight the unique advantages of each algorithmic approach and offer recommendations for the design of future SBDD models. We emphasize that 1D/2D ligand-centric drug design methods can be used in SBDD by treating the docking function as a black-box oracle, which is typically neglected. Our evaluation reveals distinct patterns across model categories. 3D structure-based models excel in binding affinities but show inconsistencies in chemical validity and pose quality. 1D models demonstrate reliable performance in standard molecular metrics but rarely achieve optimal binding affinities. 2D models offer balanced performance, maintaining high chemical validity while achieving moderate binding scores. Through detailed analysis across multiple protein targets, we identify key improvement areas for each model category, providing insights for researchers to combine strengths of different approaches while addressing their limitations. All the code that are used for benchmarking is available in https://github.com/zkysfls/2025-sbdd-benchmark

PDF Details

AAAI Conference 2026 Conference Paper

FantasyHSI: Video-Generation-Centric 4D Human Synthesis in Any Scene Through a Graph-Based Multi-Agent Framework

Lingzhou Mu
Wang Qiang
Fan Jiang
Mengchao Wang
Mu Xu
Kai Zhang

Human-Scene Interaction (HSI) seeks to generate realistic human behaviors within complex environments, yet it faces significant challenges in handling long-horizon, high-level tasks and generalizing to unseen scenes. To address these limitations, we introduce FantasyHSI, a novel HSI framework centered on video generation and multi-agent systems that operates without paired data. We model the complex interaction process as a dynamic directed graph, upon which we build a collaborative multi-agent system. This system comprises a scene navigator agent for environmental perception and high-level path planning, and a planning agent that decomposes long-horizon goals into atomic actions. Critically, we introduce a critic agent that establishes a closed-loop feedback mechanism by evaluating the deviation between generated actions and the planned path. This allows for the dynamic correction of trajectory drifts caused by the stochasticity of the generative model, thereby ensuring long-term logical consistency. To enhance the physical realism of the generated motions, we leverage Direct Preference Optimization (DPO) to train the action generator, significantly reducing artifacts such as limb distortion and foot-sliding. Extensive experiments on our custom SceneBench benchmark demonstrate that FantasyHSI significantly outperforms existing methods in terms of generalization, long-horizon task completion, and physical realism.

PDF Details DOI

YNIMG Journal 2026 Journal Article

Localization of the Berger effect in human posterior brain regions with simultaneous electroencephalogram (EEG) and stereo-EEG (SEEG) recordings

Chuanliang Han
Chen Yao
Yuanqing Wang
Sixian Li
Yuan Yao
Xingjian Liu
Kai Zhang
Chao Zhang

The Berger effect, characterized by a marked increase in alpha power (8-13 Hz) upon eye closure, is a fundamental neurophysiological phenomenon whose underlying neural networks and generating mechanism is not fully understood. To delineate this network, we simultaneously recorded Stereo-EEG (SEEG) across 36 brain regions alongside scalp EEG. Spectral analysis revealed significant state-dependent alpha power differences in nine distinct regions. Critically, we identified a robust Berger effect at the population level in five posterior cortical regions: the calcarine cortex, Cuneus, Lingual Gyrus, Precuneus, and Supramarginal Gyrus. At site level, certain subcortical region, such as the pulvinar, was also found to contribute partially to the effect. Furthermore, our decomposition analysis indicated that the Berger effect is predominantly driven by periodic alpha activity. A direct comparison of the SEEG and scalp EEG based on alpha-band spectra and alpha peak frequency consistency confirmed that the classic scalp-level effect primarily contributed from a posterior cortical source (specifically the calcarine cortex/Area 17). Our findings suggest the while the Berger effect arises from distributed cortical-subcortical networks, it is ultimately anchored in the visual cortex. This work provides a anatomically constrained understanding of the neural dynamics underlying posterior dominant rhythm and offers new insights for the clinical application of scalp EEG in resting state.

Details DOI

EAAI Journal 2026 Journal Article

Multi-modal large language model-based image captioning algorithm in information and communication technology: Bridging the gap between general and industry domain

Lianying Chao
Kai Zhang
Xubin Li
Linfeng Yin
Haoran Cai
Sijie Wu
DingCheng Shan

In the Information and Communications Technology (ICT) industry, training a domain-specific Large Language Model (LLM) or constructing a retrieval-augmented generation system requires a substantial amount of high-value domain knowledge. However, this knowledge resides not only in text but also in images. Traditional methods can parse text from domain documents but lack image captioning ability. Multi-Modal Large Language Models (MLLMs) understand images but lack sufficient domain expertise. To address this, this paper proposes a multi-stage progressive training strategy for a Domain-specific Image Captioning Model (DICModel) in the ICT domain and constructs a standard evaluation system. Specifically, we synthesize 7233 image-text pairs via the Mermaid tool and LLMs for the first-stage Supervised Fine-Tuning (SFT) of DICModel. Then, ICT domain experts manually annotate 2274 pairs for the second-stage SFT. Finally, experts and LLMs jointly build 1573 Visual Question Answering (VQA) data for instruction-based SFT. Experimental results indicate that our DICModel achieves state-of-the-art (SOTA) performance in both diagram parsing and VQA tasks. With only 7 billion parameters, DICModel outperforms 32 billion parameter SOTA models. In the parsing task, it increases the Bilingual Evaluation Understudy (BLEU) metric by 56. 8% and 20. 8% compared to SOTA models with 7 and 32 billion parameters, respectively. In the VQA task, DICModel surpasses 32 billion parameter MLLMs by 1% in accuracy on objective questions constructed by ICT experts. In summary, this work can efficiently extract logical text from images, promoting the development of multi-modal models in the ICT domain.

Details DOI

IS Journal 2026 Journal Article

Ordinal Prompt-Regularized Graph Optimal Transport for Image Ordinal Estimation

Xiangkai Wang
Kai Zhang
Xiaoxu Liu
Jia Jia
Maozhi Zhang

This article proposes a novel approach for image ordinal estimation, leveraging the power of optimal transport (OT) and prompt learning. Traditional ordinal regression methods primarily focus on learning a model to predict numerical scores, which may not directly reflect the intrinsic order. To address this limitation, we introduce a framework, termed ordinal prompt-regularized graph optimal transport (OPGOT), which utilizes OT to align the distribution of images and that of ordinal labels. First, we incorporate prompt learning with pretrained text encoders to construct ordinal prompts through a token-wise distance-based weighting scheme, enabling the model to capture the semantic relationships between ordinal categories. Second, OPGOT matches the graphs of image features and prompt embeddings via optimizing the OT with language-image cost. Hence, the learned transport plan reflects the intrinsic ordinal relationships. We conduct extensive evaluations on four benchmark datasets of different scenarios, demonstrating that OPGOT achieves significant improvements against existing methods.

Details DOI

AAAI Conference 2026 Conference Paper

RecCocktail: A Generalizable and Efficient Framework for LLM-Based Recommendation

Min Hou
Chenxi Bai
Le Wu
Hao Liu
Kai Zhang
Weiwen Liu
Richang Hong
Ruiming Tang

Large Language Models (LLMs) have achieved remarkable success in recent years, owing to their impressive generalization capabilities and rich world knowledge. To capitalize on the potential of using LLMs as recommender systems, mainstream approaches typically focus on two paradigms. The first paradigm designs multi-domain or multi-task instruction data for generalizable recommendation, so as to align LLMs with general recommendation areas and deal with cold-start recommendation. The second paradigm focuses on enhancing domain-specific recommendation tasks, improving performance in warm recommendation scenarios. While most previous works treat these two paradigms separately, we argue that they have complementary advantages, and combining them can yield better results. In this paper, we propose a generalizable and efficient LLM-based recommendation framework RecCocktail. Our approach begins with fine-tuning a "base spirit" LoRA module using domain-general recommendation instruction data to align LLM with recommendation knowledge. Next, given users' behavior of a specific domain, we construct a domain-specific "ingredient" LoRA module. We then provide an entropy-guided adaptive merging method to mix the "base spirit" and the "ingredient" in the weight space. Please note that, RecCocktail combines the advantages of the existing two paradigms without introducing additional time or space overhead during the inference phase. Moreover, RecCocktail is efficient with plug and play, as the "base spirit" LoRA is trained only once, and any domain-specific "ingredient" can be efficiently mixed with only domain-specific fine-tuning. Extensive experiments on multiple datasets under both warm and cold-start recommendation scenarios validate the effectiveness and generality of the proposed RecCocktail.

PDF Details DOI

TMLR Journal 2026 Journal Article

UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models

Xiaojie Gu
Ziying Huang
Jia-Chen Gu
Kai Zhang

Lifelong learning enables large language models (LLMs) to adapt to evolving information by continually updating their internal knowledge. An ideal system should support efficient, wide-ranging updates while preserving existing capabilities and ensuring reliable deployment. Model editing stands out as a promising solution for this goal, offering a focused and efficient way to revise a model’s internal knowledge. Although recent paradigms have made notable progress, they often struggle to meet the demands of practical lifelong adaptation at scale. To bridge this gap, we propose UltraEdit, a training-, subject-, and memory-free approach that is well-suited for ultra-scalable, real-world lifelong model editing. UltraEdit fundamentally differs from traditional paradigms by computing parameter shifts in one step using only a hidden state and its gradient, making the approach simple yet efficient. To improve scalability in lifelong settings, UltraEdit employs a lifelong normalization strategy that continuously updates feature statistics across turns, allowing it to adapt to distributional shifts and maintain consistency over time. UltraEdit achieves editing speeds more than 7× faster than the previous state-of-the-art method, while requiring 4× less VRAM. This makes it the only method currently capable of editing a 7B LLM on a 24GB consumer-grade GPU. Furthermore, we construct UltraEditBench, the largest dataset in the field to date with over 2M editing pairs, and demonstrate that our method supports up to 2M edits while maintaining high accuracy. Comprehensive experiments on five datasets and six models show that UltraEdit consistently achieves superior performance across diverse model editing scenarios, taking a further step towards safe and scalable lifelong learning. Our code is available at https://github.com/XiaojieGu/UltraEdit

PDF Details

NeurIPS Conference 2025 Conference Paper

4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time

Ziqiao Ma
Xuweiyi Chen
Shoubin Yu
Sai Bi
Kai Zhang
Ziwen Chen
Sihan Xu
Jianing Yang

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at some times to any view at any time? We provide an affirmative answer with 4D-LRM, the first large-scale 4D reconstruction model that takes input from unconstrained views and timestamps and renders arbitrary novel view-time combinations. Unlike prior 4D approaches, e. g. , optimization-based, geometry-based, or generative, that struggle with efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time representation and directly predicts per-pixel 4D Gaussian primitives from posed image tokens across time, enabling fast, high-quality rendering at, in principle, infinite frame rate. Our results demonstrate that scaling spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We show that 4D-LRM generalizes to novel objects, interpolates across time, and handles diverse camera setups. It reconstructs 24-frame sequences in one forward pass with less than 1. 5 seconds on a single A100 GPU.

PDF Details

NeurIPS Conference 2025 Conference Paper

A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking

Zixiang Zhao
Haowen Bai
Bingxin Ke
Yukun Cui
Lilun Deng
Yulun Zhang
Kai Zhang
Konrad Schindler

The real world is dynamic, yet most image fusion methods process static frames independently, ignoring temporal correlations in videos and leading to flickering and temporal inconsistency. To address this, we propose Unified Video Fusion (UniVF), a novel and unified framework for video fusion that leverages multi-frame learning and optical flow-based feature warping for informative, temporally coherent video fusion. To support its development, we also introduce Video Fusion Benchmark (VF-Bench), the first comprehensive benchmark covering four video fusion tasks: multi-exposure, multi-focus, infrared-visible, and medical fusion. VF-Bench provides high-quality, well-aligned video pairs obtained through synthetic data generation and rigorous curation from existing datasets, with a unified evaluation protocol that jointly assesses the spatial quality and temporal consistency of video fusion. Extensive experiments show that UniVF achieves state-of-the-art results across all tasks on VF-Bench. Project page: vfbench. github. io.

PDF Details

AAAI Conference 2025 Conference Paper

Adaptive Multimodal Fusion: Dynamic Attention Allocation for Intent Recognition

Bo Hu
Kai Zhang
Yanghai Zhang
Yuyang Ye

In recent years, deep multimodal learning has seen significant advancements. However, there remains a lack of multimodal fusion methods capable of dynamically adjusting the weighting of information both within and across modalities based on input samples. In the domain of multimodal intent recognition, the text modality often contains the most relevant information for intent detection, while the audio and visual modalities provide comparatively less critical information. There is a significant variation in the density of important information across different modalities and samples. To address this challenge, we propose a Dynamic Attention Allocation Fusion (DAF) method with an adaptive network structure that dynamically allocates attention both within individual modalities and across multiple modalities. This approach enables the model to focus more effectively on the most informative modalities and their respective internal features. Furthermore, we introduce a multi-view contrastive learning framework based on DAF (MVCL-DAF). This framework uses distinct and isolated modules to process information from various modalities, taking inspiration from the way the human brain processes multimodal information. Each modality independently infers intent using its respective module, while DAF integrates the multimodal information to produce a comprehensive global intent prediction. The text modality, functioning as the primary modality due to its rich semantic content, guides the other modules in the multi-view contrastive learning process. Extensive experiments demonstrate that our approach significantly outperforms existing state-of-the-art methods.

PDF Details DOI

IROS Conference 2025 Conference Paper

AdaToken-3D: Dynamic Spatial Gating for Efficient 3D Large Multimodal-Models Reasoning

Kai Zhang
Xingyu Chen
Xiaofeng Zhang

Large Multimodal Models (LMMs) have become a pivotal research focus in deep learning, demonstrating remarkable capabilities in 3D scene understanding. However, current 3D LMMs employing thousands of spatial tokens for multimodal reasoning suffer from critical inefficiencies: excessive computational overhead and redundant information flows. Unlike 2D VLMs processing single images, 3D LMMs exhibit inherent architectural redundancy due to the heterogeneous mechanisms between spatial tokens and visual tokens. To address this challenge, we propose AdaToken-3D, an adaptive spatial token optimization framework that dynamically prunes redundant tokens through spatial contribution analysis. Our method automatically tailors pruning strategies to different 3D LMM architectures by quantifying token-level information flows via attention pattern mining. Extensive experiments on LLaVA-3D (a 7B parameter 3D-LMM) demonstrate that AdaToken-3D achieves 21% faster inference speed and 63% FLOPs reduction while maintaining original task accuracy. Beyond efficiency gains, this work systematically investigates redundancy patterns in multimodal spatial information flows through quantitative token interaction analysis. Our findings reveal that over 60% of spatial tokens contribute minimally (<5%) to the final predictions, establishing theoretical foundations for efficient 3D multimodal learning.

Details

JBHI Journal 2025 Journal Article

An EEG-Based Seizure Prediction Model Encoding Brain Network Temporal Dynamics

Jiahui Liao
Yiyi Chen
Yihang He
Kai Zhang
Ting Ma
Yilong Wang
Xiaoqiu Shao

EEG-based seizure prediction enables timely treatment for patients, but its performance is limited by the difficulty in effectively characterizing the temporal dynamics of epileptic brain networks. Metastability, which describes recurring topographical patterns of spontaneous neural activity over time, provides a unique perspective for capturing the dynamic evolution before seizure onset. In this study, we propose a seizure prediction model that fuses consistent epileptic network processes across subjects into a higher-order latent space. Specifically, we first construct metastable transition patterns to identify the recurrent network states over time. Through adversarial feature learning, we then impose the metastability prior on the latent embedding space encoded via a variational autoencoder (VAE), while leveraging the maximum mean discrepancy measure (MMD) to further mitigate the patient gap. The latent representation, endowed with physiological priors, is ultimately utilized for patient-independent seizure prediction. We evaluate our method on two publicly available and one clinical scalp EEG datasets. Compared to the existing methods, our method has improved AUC, sensitivity, and specificity on CHB-MIT dataset by approximately 9%, 5%, and 5%, respectively. Our method shows that combining brain network-based physiological prior with deep learning for EEG representation learning is a brand-new strategy for associating seizures with complex brain network variations, enabling reliable patient-independent seizure prediction.

Details DOI

NeurIPS Conference 2025 Conference Paper

ARM: Adaptive Reasoning Model

Siye Wu
Jian Xie
Yikai Zhang
Aili Chen
Kai Zhang
Yu Su
Yanghua Xiao

While large reasoning models demonstrate strong performance on complex tasks, they lack the ability to adjust reasoning token usage based on task difficulty. This often leads to the "overthinking" problem—excessive and unnecessary reasoning—which, although potentially mitigated by human intervention to control the token budget, still fundamentally contradicts the goal of achieving fully autonomous AI. In this work, we propose Adaptive Reasoning Model (ARM), a reasoning model capable of adaptively selecting appropriate reasoning formats based on the task at hand. These formats include three efficient ones—Direct Answer, Short CoT, and Code—as well as a more elaborate format, Long CoT. To train ARM, we introduce Ada-GRPO, an adaptation of Group Relative Policy Optimization (GRPO), which addresses the format collapse issue in traditional GRPO. Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by an average of $\sim$30%, and up to $\sim$70%, while maintaining performance comparable to the model that relies solely on Long CoT. Furthermore, not only does it improve inference efficiency through reduced token generation, but it also brings a $\sim$2$\times$ speedup in training. In addition to the default Adaptive Mode, ARM supports two additional reasoning modes: 1) Instruction-Guided Mode, which allows users to explicitly specify the reasoning format via special tokens—ideal when the appropriate format is known for a batch of tasks. 2) Consensus-Guided Mode, which aggregates the outputs of the three efficient formats and resorts to Long CoT in case of disagreement, prioritizing performance with higher token usage. All the resources will be released.

PDF Details

NeurIPS Conference 2025 Conference Paper

CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic

Yuxuan Sun
Yixuan Si
Chenglu Zhu
Kai Zhang
Zhongyi Shui
Bowen Ding
Tao Lin
Lin Yang

Recent advances in computational pathology have led to the emergence of numerous foundation models. These models typically rely on general-purpose encoders with multi-instance learning for whole slide image (WSI) classification or apply multimodal approaches to generate reports directly from images. However, these models cannot emulate the diagnostic approach of pathologists, who systematically examine slides at low magnification to obtain an overview before progressively zooming in on suspicious regions to formulate comprehensive diagnoses. Instead, existing models directly output final diagnoses without revealing the underlying reasoning process. To address this gap, we introduce CPathAgent, an innovative agent-based approach that mimics pathologists' diagnostic workflow by autonomously navigating across WSI through zoom-in/out and move operations based on observed visual features, thereby generating substantially more transparent and interpretable diagnostic summaries. To achieve this, we develop a multi-stage training strategy that unifies patch-level, region-level, and WSI-level capabilities within a single model, which is essential for replicating how pathologists understand and reason across diverse image scales. Additionally, we construct PathMMU-HR², the first expert-validated benchmark for large region analysis. This represents a critical intermediate scale between patches and whole slides, reflecting a key clinical reality where pathologists typically examine several key large regions rather than entire slides at once. Extensive experiments demonstrate that CPathAgent consistently outperforms existing approaches across benchmarks at three different image scales, validating the effectiveness of our agent-based diagnostic approach and highlighting a promising direction for computational pathology.

PDF Details

IJCAI Conference 2025 Conference Paper

Decoupling and Reconstructing: A Multimodal Sentiment Analysis Framework Towards Robustness

Mingzheng Yang
Kai Zhang
Yuyang Ye
Yanghai Zhang
Runlong Yu
Min Hou

Multimodal sentiment analysis (MSA) has shown promising results but often poses significant challenges in real-world applications due to its dependence on the complete and aligned multimodal sequences. While existing approaches attempt to address missing modalities through feature reconstruction, they often neglect the complex interplay between homogeneous and heterogeneous relationships in multimodal features. To address this problem, we propose Decoupled-Adaptive Reconstruction (DAR), a novel framework that explicitly addresses these limitations through two key components: (1) a mutual information-based decoupling module that decomposes features into common and independent representations, and (2) a reconstruction module that independently processes these decoupled features before fusion for downstream tasks. Extensive experiments on two benchmark datasets demonstrate that DAR significantly outperforms existing methods in both modality reconstruction and sentiment analysis tasks, particularly in scenarios with missing or unaligned modalities. Our results show improvements of 2. 21% in bi-classification accuracy and 3. 9% in regression error compared to state-of-the-art baselines on the MOSEI dataset.

PDF Details DOI

TIST Journal 2025 Journal Article

Denoising Structure against Adversarial Attacks on Graph Representation Learning

Na Chen
Ping Li
Jincheng Huang
Kai Zhang

Despite their excellent performance in graph representation learning, graph convolutional networks have been proved to be vulnerable to adversarial perturbations on the connectivity between nodes in an unnoticed manner. In this work, by looking into the impacts of adversarial attacks on graph data, we empirically find that the dominant edge-addition attacks generally increase the heterophily between connected nodes, which will fool the transductive inference models on node classification task. To defend against such attacks, we develop a Two-Stage Denoising (TSD) method that aims at removing possible malicious edges so as to mitigate the heterophily issue introduced by attacks. In particular, after a rough removal of the links that have quite low feature similarity, our method further spots the potentially heterophilous links by predicting node labels with a multi-view labeling consensus. This design is based on assumption that if the label predictions for the same node from two different views of a graph data are consistent, then we have a high chance to acquire the reliable labeling. The experiments demonstrate that by denoising a graph this way, the robustness of graph convolutional networks on node classification task is remarkably improved, compared to several strong competitive robust graph neural network models.

Details DOI

AAAI Conference 2025 Conference Paper

Enhancing Low-Light Images: A Synthetic Data Perspective on Practical and Generalizable Solutions

Yu Long
Qinghua Lin
Zhihua Wang
Kai Zhang
Jianguo Zhang
Yuming Fang

Recently, deep neural networks (DNNs) have emerged as the leading approach for low-light image enhancement (LLIE). However, training these models generally requires large-scale paired datasets, which are challenging to obtain due to the labor-intensive and time-consuming nature of real-world data collection. To alleviate this issue, synthetic data are often combined with real-captured data for training. However, most existing low-light image synthesis methods are simply performed in the sRGB domain using Gamma correction or manual adjustments via Lightroom, which fail to incorporate the physical imaging prior through the image signal processing (ISP) pipeline and thus result in limited dataset size and degradation space. Consequently, LLIE methods trained on such data often exhibit some drawbacks in the results, such as inaccurate white balance and abnormal enhancement artifacts, which limit their practicality and generalizability. In this paper, we propose a practical low-light image synthesis pipeline capable of generating unlimited paired training data. Our pipeline starts with a reverse ISP model that converts sRGB images back to the unprocessed RAW domain, where we then simulate low-light degradation, noise degradation, and white balance adjustments. Finally, the degraded RAW images are processed through a forward ISP model to produce low-light sRGB images. The pipeline further employs multiple tone mapping curves and color correction matrices (CCMs) to expand the degradation space. Hence, trained with our proposed synthetic data, existing state-of-the-art (SOTA) LLIE deep models are expected to improve their performance. Extensive experiments across various datasets demonstrate that our synthetic data can indeed effectively enhance existing LLIE deep models, improving both their practicality and generalizability.

PDF Details DOI

EAAI Journal 2025 Journal Article

Fault diagnosis of driving gear in battery swapping system based on auditory bionics

Hang Yuan
Hao Wu
Jiacheng Li
Kai Zhang
Huijuan Zhang
Xiaowen You
Xianglong You

Rack and pinion drives (RPD) are widely used in battery swapping system (BSS) for electric heavy trucks (EHT), and due to the continuous heavy-load and high-intensity operation, along with the electric erosion, the gears in the RPD are always damaged, which causes unexpected consequences such as downtime or safety incidents. The working conditions of the RPD in BSS include uncertain noises, fluctuant and low speed, which pose steep challenges to accurate fault diagnosis. Considering the auditory resistance of interference, the low-frequency sensitivity of auditory perception, and the auditory saliency mechanism, to leverage the advantages of auditory perceptual mechanism in addressing the above challenges, as the contribution in artificial intelligence, we propose an entire vibration signal processing scheme based on auditory bionics, including some mathematical models for auditory mechanisms. For the application in engineering, the proposed scheme is employed for fault diagnosis of RPD in BSS in unique working conditions. First, adaptive resampling is used to smooth the speed fluctuation, then, Gammatone filters are employed to transform vibration signals to cochleograms, after that, based on auditory stream segregation and selective attention mechanisms, effective frequency channels and salient features are extracted from the cochleograms, besides, to improve the diagnosis accuracy, binaural features are also extracted, finally, based on (sectional) sparse representation and fusion, fault diagnosis is achieved. The effectiveness of the fault diagnosis scheme is demonstrated using a BSS prototype system.

Details DOI

AAAI Conference 2025 Conference Paper

Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation

Yuyang Ye
Zhi Zheng
Yishan Shen
Tianshu Wang
Hengruo Zhang
Peijun Zhu
Runlong Yu
Kai Zhang

Recent advances in Large Language Models (LLMs) have demonstrated significant potential in the field of Recommendation Systems (RSs). Most existing studies have focused on converting user behavior logs into textual prompts and leveraging techniques such as prompt tuning to enable LLMs for recommendation tasks. Meanwhile, research interest has recently grown in multimodal recommendation systems that integrate data from images, text, and other sources using modality fusion techniques. This introduces new challenges to the existing LLM-based recommendation paradigm which relies solely on text modality information. Moreover, although Multimodal Large Language Models (MLLMs) capable of processing multi-modal inputs have emerged, how to equip MLLMs with multi-modal recommendation capabilities remains largely unexplored. To this end, in this paper, we propose the Multimodal Large Language Model-enhanced Sequential Multimodal Recommendation (MLLM-MSR) model. To capture the dynamic user preference, we design a two-stage user preference summarization method. Specifically, we first utilize an MLLM-based item-summarizer to extract image feature given an item and convert the image into text. Then, we employ a recurrent user preference summarization generation paradigm to capture the dynamic changes in user preferences based on an LLM-based user-summarizer. Finally, to enable the MLLM for multi-modal recommendation task, we propose to fine-tune a MLLM-based recommender using Supervised Fine-Tuning (SFT) techniques. Extensive evaluations across various datasets validate the effectiveness of MLLM-MSR, showcasing its superior ability to capture and adapt to the evolving dynamics of user preferences.

PDF Details DOI

EAAI Journal 2025 Journal Article

Integration of neural networks in brain–computer interface applications: Research frontiers and trend analysis based on Python

Junming Chen
Hongyu Yin
Kai Zhang
Yangzhi Ren
Hui Zeng

Brain–computer interfaces (BCIs) establish a connection between the human brain and external devices, facilitating novel forms of communication and control. Although BCIs possess significant potential, there is a need for performance improvements. Integrating neural networks into BCIs is crucial for enhancing functionality and promoting broader adoption. This study examines 1, 867 articles from the Web of Science core database, covering the period from 1996 to 2024, to identify contemporary hotspots and trends in the application of neural networks within BCIs. This study employs bibliometric methods and Python for analysis, examining collaborative relationships, citation networks, keyword bursts, and clustering with visual representations of the findings. The results indicated that current study hotspots predominantly center on “P300, ” “Long Short-Term Memory, ” “Motor Imagery, ” “Epilepsy, ” “Emotion Recognition, ” “Feature Extraction, ” and “Transfer Learning. ” Future development directions encompass: (1) the establishment of public BCI datasets; (2) the exploration of diverse feature extraction and fusion methods; (3) the enhancement of machine learning and deep learning integration for improved performance and real-time processing; (4) the expansion of application scenarios and the development of portable devices; (5) the optimization of transfer learning algorithms to mitigate performance challenges arising from individual differences. This study provides an overview of the current research landscape and identifies potential future research directions. Furthermore, it assists practitioners in recognizing additional business opportunities and acts as a resource for the formulation of government policy.

Details DOI

AAAI Conference 2025 Conference Paper

Intent Oriented Contrastive Learning for Sequential Recommendation

Wuhong Wang
Jianhui Ma
Yuren Zhang
Kai Zhang
Junzhe Jiang
Yihui Yang
Yacong Zhou
Zheng Zhang

Sequential recommendation aims to predict the next item a user is likely to interact with based on their historical interaction sequence. Capturing user intent is crucial in this process, as each interaction is typically driven by specific intentions (e.g., buying skincare products for skin maintenance, buying makeup for cosmetic purposes, etc.). However, users often have multiple, dynamically changing intents, making it challenging for models to accurately learn these intents when relying on the entire historical sequence as input. To address this, we propose a novel framework called Intent Oriented Contrastive Learning for Sequential Recommendation (IOCLRec). This framework begins by segmenting users’ sequential behaviors into multiple subsequences, which represent the coarse-grained intents of users at different points in their interaction history. These subsequences form the basis for the three contrastive learning modules within IOCLRec. The fine-grained intent contrastive learning module uncovers detailed intent representations, while the single-intent and multi-intent contrastive learning modules utilize intent-oriented data augmentation operators to capture the diverse intents of users. These three modules work synergistically, driving comprehensive performance optimization in intricate sequential recommendation scenarios. Our method has been extensively evaluated on four public datasets, demonstrating superior effectiveness.

PDF Details DOI

TMLR Journal 2025 Journal Article

Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents

Yu Gu
Kai Zhang
Yuting Ning
Boyuan Zheng
Boyu Gou
Tianci Xue
Cheng Chang
Sanjari Srivastava

Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algorithms, e.g., tree search, is advantageous over reactive planning for web agents. However, unlike simulated sandbox environments, real-world environments such as the web are rife with irreversible actions. This undermines the feasibility of backtracking, a cornerstone of (tree) search. Overly relying on test-time search also hurts efficiency. We advocate model-based planning for web agents that employs a world model to simulate and deliberate over the outcome of each candidate action before committing to one. We systematically explore this paradigm by: (1) Proposing a model-based planning framework, WebDreamer, which employs LLMs to serve as both world models and value functions; (2) Training specialized LLMs as world models with a scalable data synthesis pipeline. Empirical results demonstrate that WebDreamers achieves substantial performance improvements over reactive baselines. It is competitive, while being - times more efficient, with tree search in sandbox environments (VisualWebArena) and also works effectively on real-world websites (Online-Mind2Web and Mind2Web-Live). Furthermore, our trained world model, Dreamer-7B, performs comparable to GPT-4o, highlighting the potential of specialized world models for efficient and effective planning in complex web environments. All code, models, and data are publicly available at https://github.com/OSU-NLP-Group/WebDreamer

PDF Details

EAAI Journal 2025 Journal Article

Knowledge-data synergy enabling zero-shot composite fault diagnosis in sucker-rod pumping systems

Xin-yan Wang
Li-ming Zhang
Kai Zhang
Cheng Cheng

Zero-Shot Learning Accurate and comprehensive fault diagnosis is critical to ensuring production efficiency and extending equipment lifespan in oilfield operations. However, most existing diagnostic methods for sucker rod pumping systems are limited to single-fault identification, while the scarcity and high acquisition cost of compound fault samples hinder the application of supervised learning. To address these challenges, this paper proposes a zero-shot compound fault diagnosis framework that integrates domain knowledge with data-driven representations, enabling the identification of unseen fault using only single-fault samples. First, a unified semantic space is constructed by jointly embedding text-based fault descriptions and capsule-encoded load features. Then, a semantics-guided weakly supervised attribute composition strategy is introduced to enhance the completeness and discriminability of the semantic space. Finally, a bidirectional contrastive learning mechanism is established between visual encodings and semantic representations, and an adaptive multi-task loss weighting strategy is employed to optimize the overall framework efficiently. Experiments on real-world oilfield data demonstrate that the proposed method achieves an F1-score of 86. 15 % under the zero-shot setting and 69. 08 % under the generalized zero-shot setting, offering an effective and scalable solution for compound fault diagnosis in oil wells.

Details DOI

AAAI Conference 2025 Conference Paper

LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers

Xuan Shen
Zhao Song
Yufa Zhou
Bo Chen
Yanyu Li
Yifan Gong
Kai Zhang
Hao Tan

Diffusion Transformers have emerged as the preeminent models for a wide array of generative tasks, demonstrating superior performance and efficacy across various applications. The promising results come at the cost of slow inference, as each denoising step requires running the whole transformer model with a large amount of parameters. In this paper, we show that performing the full computation of the model at each diffusion step is unnecessary, as some computations can be skipped by lazily reusing the results of previous steps. Furthermore, we show that the lower bound of similarity between outputs at consecutive steps is notably high, and this similarity can be linearly approximated using the inputs. To verify our demonstrations, we propose the **LazyDiT**, a lazy learning framework that efficiently leverages cached results from earlier steps to skip redundant computations. Specifically, we incorporate lazy learning layers into the model, effectively trained to maximize laziness, enabling dynamic skipping of redundant computations. Experimental results show that LazyDiT outperforms the DDIM sampler across multiple diffusion transformer models at various resolutions. Furthermore, we implement our method on mobile devices, achieving better performance than DDIM with similar latency.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge

Boyu Gou
Zanming Huang
Yuting Ning
Yu Gu
Michael Lin
Weijian Qi
Andrei Kopanev
Botao Yu

Agentic search such as Deep Research systems-where agents autonomously browse the web, synthesize information, and return comprehensive citation-backed answers-represents a major shift in how users interact with web-scale information. While promising greater efficiency and cognitive offloading, the growing complexity and open-endedness of agentic search have outpaced existing evaluation benchmarks and methodologies, which largely assume short search horizons and static answers. In this paper, we introduce Mind2Web 2, a benchmark of 130 realistic, high-quality, and long-horizon tasks that require real-time web browsing and extensive information synthesis, constructed with over 1000 hours of human labor. To address the challenge of evaluating time-varying and complex answers, we propose a novel Agent-as-a-Judge framework. Our method constructs task-specific judge agents based on a tree-structured rubric design to automatically assess both answer correctness and source attribution. We conduct a comprehensive evaluation of ten frontier agentic search systems and human performance, along with a detailed error analysis to draw insights for future development. The best-performing system, OpenAI Deep Research, can already achieve 50-70% of human performance while spending half the time, highlighting its great potential. Altogether, Mind2Web 2 provides a rigorous foundation for developing and benchmarking the next generation of agentic search systems.

PDF Details

EAAI Journal 2025 Journal Article

MIRROR: Multi-scale iterative refinement for robust chinese text recognition

Hengnian Qi
Qiuyi Xin
Jiabin Ye
Hao Yang
Kai Zhang
Chu Zhang
Qing Lang

Text recognition has become a key area of research due to its wide applications in various fields. As an important branch of computer vision, Chinese text recognition has gained increasing research and practical value. However, the existing Chinese text recognition methods are still limited. This paper proposes an innovative Chinese text recognition method, Multi-Scale Iterative Refinement for Robust Chinese Text Recognition (MIRROR). The model significantly improves the recognition accuracy of Chinese text through advanced algorithms and structural design. The MIRROR model consists of two core components: a feature extractor and a Next-Character Decoder. Specifically, this paper proposes a Spatial Local Self-Attention Module to enhance the model’s ability to model long-distance dependencies in complex character sequences, addressing the problem of complex distributions in medium-to-long distance Chinese character sequences. The Character Refinement Module effectively captures multi-scale information, handles stroke feature differences, and resolves inter-class similarity issues. By combining multi-scale feature extraction with iterative optimization for feature refinement, the model identifies common features across different styles of the same character, solves the intra-class variation problem, and improves model robustness. In addition, this paper introduces a Three-Dimensional Weight Attention Module to refine the granularity of character features. Experiments show that MIRROR significantly outperforms baseline models on Chinese benchmark datasets. On scene datasets, performance improves by 3. 08% (from 76. 90% to 79. 98%), on web datasets by 1. 46% (from 70. 43% to 71. 89%), on document datasets by 0. 38% (from 98. 72% to 99. 10%), and on handwriting datasets by 9. 29% (from 50. 26% to 59. 55%).

Details DOI

NeurIPS Conference 2025 Conference Paper

OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions

Yuanhao Cai
He Zhang
Xi Chen
Jinbo Xing
Yiwei Hu
Yuqian Zhou
Kai Zhang
Zhifei Zhang

Existing feedforward subject-driven video customization methods mainly study single-subject scenarios due to the difficulty of constructing multi-subject training data pairs. Another challenging problem that how to use the signals such as depth, mask, camera, and text prompts to control and edit the subject in the customized video is still less explored. In this paper, we first propose a data construction pipeline, VideoCus-Factory, to produce training data pairs for multi-subject customization from raw videos without labels and control signals such as depth-to-video and mask-to-video pairs. Based on our constructed data, we develop an Image-Video Transfer Mixed (IVTM) training with image editing data to enable instructive editing for the subject in the customized video. Then we propose a diffusion Transformer framework, OmniVCus, with two embedding mechanisms, Lottery Embedding (LE) and Temporally Aligned Embedding (TAE). LE enables inference with more subjects by using the training subjects to activate more frame embeddings. TAE encourages the generation process to extract guidance from temporally aligned control signals by assigning the same frame embeddings to the control and noise tokens. Experiments demonstrate that our method significantly surpasses state-of-the-art methods in both quantitative and qualitative evaluations. Project page is at https: //caiyuanhao1998. github. io/project/OmniVCus/

PDF Details

NeurIPS Conference 2025 Conference Paper

Personalized Visual Content Generation in Conversational Systems

Xianquan Wang
Zhaocheng Du
Huibo Xu
Shukang Yin
Yupeng Han
Jieming Zhu
Kai Zhang
Qi Liu

With the rapid progress of large language models (LLMs) and diffusion models, there has been growing interest in personalized content generation. However, current conversational systems often present the same recommended content to all users, falling into the dilemma of "one-size-fits-all. " To break this limitation and boost user engagement, in this paper, we introduce PCG ( P ersonalized Visual C ontent G eneration), a unified framework for personalizing item images within conversational systems. We tackle two key bottlenecks: the depth of personalization and the fidelity of generated images. Specifically, an LLM-powered Inclinations Analyzer is adopted to capture user likes and dislikes from context to construct personalized prompts. Moreover, we design a dual-stage LoRA mechanism—Global LoRA for understanding task-specific visual style, and Local LoRA for capturing preferred visual elements from conversation history. During training, we introduce the visual content condition method to ensure LoRA learns both historical visual context and maintains fidelity to the original item images. Extensive experiments on benchmark conversational datasets—including objective metrics and GPT-based evaluations—demonstrate that our framework outperforms strong baselines, which highlight its potential to redefine personalization in visual content generation for conversational scenarios like e-commerce and real-world recommendation.

PDF Details

EAAI Journal 2025 Journal Article

Plug-and-play fine grained neural cognitive diagnosis framework

Xinjie Sun
Qi Liu
Kai Zhang
Weiyin Gong
Shuanghong Shen
Fei Wang
Yan Zhuang
Meikai Bao

Cognitive diagnosis (CD) is a core task in intelligent education, which accurately assesses students’ mastery of specific knowledge concepts (KCs) by analyzing their answer records. However, existing methods mainly rely on explicit interaction data and use diagnostic models for automatic knowledge proficiency inference. These methods lack systematic optimization for fine-grained knowledge level representation, making it difficult to fully reflect students’ true learning status. To address this, this paper introduces a Plug-and-Play Fine Grained Neural Cognitive Diagnosis Framework (FNCD) with Knowledge-Level Constraint Awareness. The framework combines a knowledge proficiency evaluation module with students’ answer records and a Q-matrix to statically assess knowledge mastery. It uses a student similarity construction method based on random grouping to reveal latent learning pattern associations. Additionally, it employs a multi-scale relational learning strategy and a Top-k attention-enhanced graph network mechanism to dynamically adjust the student similarity relationship network, accurately modeling the complex learning relationships between students. Ultimately, a joint training mechanism is used to optimize the outputs of each module, significantly improving the rationality, interpretability, and accuracy of CD. The experimental results demonstrate that FNCD, as an artificial intelligence-driven plug-and-play module, can be effectively integrated into existing CD models to enhance the modeling of fine-grained knowledge mastery and improve diagnostic accuracy, showcasing the application potential of artificial intelligence in personalized education.

Details DOI

ICLR Conference 2025 Conference Paper

Problem-Parameter-Free Federated Learning

Wenjing Yan
Kai Zhang
Xiaolu Wang
Xuanyu Cao

Federated learning (FL) has garnered significant attention from academia and industry in recent years due to its advantages in data privacy, scalability, and communication efficiency. However, current FL algorithms face a critical limitation: their performance heavily depends on meticulously tuned hyperparameters, particularly the learning rate or stepsize. This manual tuning process is challenging in federated settings due to data heterogeneity and limited accessibility of local datasets. Consequently, the reliance on problem-specific parameters hinders the widespread adoption of FL and potentially compromises its performance in dynamic or diverse environments. To address this issue, we introduce PAdaMFed, a novel algorithm for nonconvex FL that carefully combines adaptive stepsize and momentum techniques. PAdaMFed offers two key advantages: 1) it operates autonomously without relying on problem-specific parameters; and 2) it manages data heterogeneity and partial participation without requiring heterogeneity bounds. Despite these benefits, PAdaMFed provides several strong theoretical guarantees: 1) It achieves state-of-the-art convergence rates with a sample complexity of $\mathcal{O}(\epsilon^{-4})$ and communication complexity of $\mathcal{O}(\epsilon^{-3})$ to obtain an accuracy of $||\nabla f\left(\boldsymbol{\theta}\right)|| \leq \epsilon$, even using constant learning rates; 2) these complexities can be improved to the best-known $\mathcal{O}(\epsilon^{-3})$ for sampling and $\mathcal{O}(\epsilon^{-2})$ for communication when incorporating variance reduction; 3) it exhibits linear speedup with respect to the number of local update steps and participating clients at each global round. These attributes make PAdaMFed highly scalable and adaptable for various real-world FL applications. Extensive empirical evidence on both image classification and sentiment analysis tasks validates the efficacy of our approaches.

Details

NeurIPS Conference 2025 Conference Paper

Results of the Big ANN: NeurIPS’23 competition

Harsha Vardhan simhadri
Martin Aumüller
Matthijs Douze
Dmitry Baranchuk
Amir Ingber
Edo Liberty
George Williams
Ben Landrum

The 2023 Big ANN Challenge, held at NeurIPS 2023, focused on advancing the state-of-the-art in indexing data structures and search algorithms for practical variants of Approximate Nearest Neighbor (ANN) search that reflect its the growing complexity and diversity of workloads. Unlike prior challenges that emphasized scaling up classical ANN search (Simhadri et al. , NeurIPS 2021), this competition addressed sparse, filtered, out-of-distribution, and streaming variants of ANNS. Participants developed and submitted innovative solutions that were evaluated on new standard datasets with constrained computational resources. The results showcased significant improvements in search accuracy and efficiency, with notable contributions from both academic and industrial teams. This paper summarizes the competition tracks, datasets, evaluation metrics, and the innovative approaches of the top-performing submissions, providing insights into the current advancements and future directions in the field of approximate nearest neighbor search.

PDF Details

NeurIPS Conference 2025 Conference Paper

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval

Di Liu
Meng Chen
Baotong Lu
Huiqiang Jiang
Zhenhua Han
Qianxi Zhang
Qi Chen
Chengruidong Zhang

Transformer-based Large Language Models (LLMs) have become increasingly important. However, scaling LLMs to longer contexts incurs slow inference speed and high GPU memory consumption for caching key-value (KV) vectors. This paper presents RetrievalAttention, a training-free approach to both accelerate the decoding phase and reduce GPU memory consumption by pre-building KV vector indexes for fixed contexts and maintaining them in CPU memory for efficient retrieval. Unlike conventional KV cache methods, RetrievalAttention integrate approximate nearest neighbor search (ANNS) indexes into attention computation. We observe that off-the-shelf ANNS techniques often fail due to the out-of-distribution (OOD) nature of query and key vectors in attention mechanisms. RetrievalAttention overcomes this with an attention-aware vector index. Our evaluation shows RetrievalAttention achieves near full attention accuracy while accessing only 1-3\% of the data, significantly reducing inference costs. Remarkably, RetrievalAttention enables LLMs with 8B parameters to handle 128K tokens on a single NVIDIA RTX4090 (24GB), achieving a decoding speed of 0. 107 seconds per token.

PDF Details

AAAI Conference 2025 Conference Paper

Spin: Diffusion-based Semantic Image Painting Through Independent Information Injection

Dantong Wu
Zhiqiang Chen
Tianjiao Du
Peipei Ran
Mengchao Bai
Kai Zhang

Diffusion models have been utilized as powerful tools for various image editing tasks, including semantic image painting (SIP), which aims to generate content within masked regions conditioned on a reference image or text. SIP, especially those using images as conditions, often suffers from three issues: semantic inconsistency, unnatural transitions, and style inconsistency, which significantly hinder its practical application. To address these challenges, we propose a novel Semantic Image Painting framework with INdependent INformation INjection (Spin). Specifically, we compute a saliency map to segregate the reference image into salient and non-salient components. We then filter out the non-salient information during the semantic embedding extraction phase and precisely inject the semantic embedding into the masked region instead of the whole image during the semantic generation phase. Furthermore, we impose an additional style guidance to promote style consistency between background and foreground. Experimental results demonstrate that Spin achieve superior semantic similarity and image coherence across various styles, including realistic, pencil drawings, cartoon, and oil painting. Additionally, Spin offers diversity and editability, and can be integrated into other models that meet our prerequisites.

PDF Details DOI

JBHI Journal 2025 Journal Article

Super-resolution Reconstruction of Fetal Brain MRI with Multi-view Interpolation Weight Learning

Shijie Huang
DengQiang Jia
Kai Zhang
Lingnan Kong
Fangmei Zhu
Zhongxiang Ding
Geng Chen
Dinggang Shen

Super-resolution reconstruction (SRR) of isotropic fetal brain MR images is critical for prenatal ex aminations but is hindered by fetal motion and misalignment of thick-slice scans. To address these challenges comprehensively, we introduce an innovative deep learning model, namely 3D-WISE, a 3D Weighted Interpolation for Super-resolution Estimation of fetal brain MRI. The model generates high-quality isotropic fetal brain MR images by learning the interpolation weights to correct misalignments between slices and volumes. These misalignments are estimated by extracting deep features from multiple motion corrupted stacks. Specifically, 3D-WISE incorporates two key components: (1) a weight learning module for multi view interpolation and (2) a feature extraction module guided by multi-type attention mechanisms. The weight learning module first maps motion-corrupted thick-slice stacks into latent feature spaces. The resulting features are then fed to an implicit decoding block to estimate interpolation weights of the surrounding points for a given coordinate. We further enhance our approach by incorporating convolutional block attention and atlas-induced cross-attention mechanisms. Extensive experiments on two benchmark datasets show that our 3D-WISE achieves remarkably improved performance compared to the widely adoptedregistration-reconstruction framework. We also ex tend the experiments on anatomical structure reconstruction and achieve promising results, highlighting the significant potential of our 3D-WISE for fetal brain MR images SRR in clinical settings. Our code is available at https://github.com/sj-huang/3D-WISE.

Details DOI

EAAI Journal 2025 Journal Article

Use deep transfer learning for efficient time-series updating of subsurface flow surrogate model

Wenhao Fu
Piyang Liu
Kai Zhang
Jinding Zhang
Xu Chen
Liming Zhang
Xia Yan

In subsurface flow, history matching is a complex high-dimensional sampling problem demanding numerous computationally intensive numerical simulations, which result in high costs. To mitigate this, various deep learning-based surrogate-assisted techniques are employed to approximate the original numerical simulation processes for more efficient computation. Despite the widespread application of deep learning-based surrogate models, significant limitations remain, particularly the need for the frequent reconstruction of surrogate models when new observed data become available. This process requires additional high-fidelity simulation data, further increasing computational burden of history matching. To tackle this problem, we introduce a transfer learning-based framework within an image-to-sequence surrogate model to enable efficient updates as new data become available. This surrogate model consists of a geological feature extraction module and a sequence data regression module. To facilitate rapid learning of new tasks while retaining previously learned knowledge, we employ a decoupled learning strategy where the backbone structure remains fixed, and only the sequence data regression module is fine-tuned. The surrogate model is subsequently used for posterior sampling of the geological model using the Randomized Maximum Likelihood (RML) method. The proposed workflow is evaluated on both two-dimensional and three-dimensional oil-water flow models. Experiments indicate that our surrogate model achieves a Root Mean Square Error (RMSE) below 0. 08 and a Coefficient of Determination (R2) above 0. 97. Moreover, by leveraging transfer learning, the model attains comparable predictive accuracy with only 40 % of the original training data. The results demonstrate the surrogate model's capability to reduce computational costs while preserving accuracy.

Details DOI

IJCAI Conference 2025 Conference Paper

WDMIR: Wavelet-Driven Multimodal Intent Recognition

Weiyin Gong
Kai Zhang
Yanghai Zhang
Qi Liu
Xinjie Sun
Junyu Lu
Linbo Zhu

Multimodal intent recognition (MIR) seeks to accurately interpret user intentions by integrating verbal and non-verbal information across video, audio and text modalities. While existing approaches prioritize text analysis, they often overlook the rich semantic content embedded in non-verbal cues. This paper presents a novel Wavelet-Driven Multimodal Intent Recognition (WDMIR) framework that enhances intent understanding through frequency-domain analysis of non-verbal information. To be more specific, we propose: (1) a wavelet-driven fusion module that performs synchronized decomposition and integration of video-audio features in the frequency domain, enabling fine-grained analysis of temporal dynamics; (2) a cross-modal interaction mechanism that facilitates progressive feature enhancement from bimodal to trimodal integration, effectively bridging the semantic gap between verbal and non-verbal information. Extensive experiments on MIntRec demonstrate that our approach achieves state-of-the-art performance, surpassing previous methods by 1. 13% on accuracy. Ablation studies further verify that the wavelet-driven fusion module significantly improves the extraction of semantic information from non-verbal sources, with a 0. 41% increase in recognition accuracy when analyzing subtle emotional cues.

PDF Details DOI

EAAI Journal 2024 Journal Article

Ada2MF: Dual-adaptive multi-fidelity neural network approach and its application in wind turbine wake prediction

Lingyu Zhan
Zhenfan Wang
Yaoran Chen
Limin Kuang
Yu Tu
Dai Zhou
Zhaolong Han
Kai Zhang

In the context of data-driven deep learning, employing multi-fidelity methods for swift and precise wake field prediction is a novel attempt. Current Multi-Fidelity Neural Networks (MFNNs) face accuracy loss due to structure theory limitations, and performance degradation from negative transfer. To address these issues and enhance wake prediction performance, we propose a Dual-Adaptive Multi-Fidelity Neural Network (Ada2MF) framework. This framework features an adaptive multi-fidelity (AMF) module that integrates three sub-networks via a learnable weighted gate, effectively capturing the linear, nonlinear, and residual characteristics of high-fidelity data and mitigating theoretical accuracy loss. Additionally, the adaptive fast weighting (AFW) module employs a dynamic loss-weighting algorithm to optimally balance multi-fidelity losses and prevent negative transfer. Initial validation on benchmark functions and further evaluation using a multi-fidelity wind turbine wake field database confirm the effectiveness of Ada2MF. Specifically, with a complete dataset, Ada2MF achieves 66% and 41% improvements in wake prediction accuracy over single-fidelity neural networks and MFNN, respectively. Even with an 80% reduction in data volume, these improvements escalate to 82% and 78%, without incurring significant accuracy loss. Such results underscore Ada2MF’s remarkable ability to improve prediction accuracy while substantially reducing the dependency on high-fidelity data.

Details DOI

IJCAI Conference 2024 Conference Paper

Cross-View Diversity Embedded Consensus Learning for Multi-View Clustering

Chong Peng
Kai Zhang
Yongyong Chen
Chenglizhao Chen
Qiang Cheng

Multi-view clustering (MVC) has garnered significant attention in recent studies. In this paper, we propose a novel MVC method, named CCL-MVC. The novel method constructs a cross-order neighbor tensor of multi-view data to recover a low-rank essential tensor, preserves noise-free, comprehensive, and complementary cross-order relationships among the samples. Furthermore, it constructs a consensus representation matrix by fusing the low-rank essential tensor with auto-adjusted cross-view diversity embedding, fully exploiting both consensus and discriminative information of the data. An effective optimization algorithm is developed, which is theoretically guaranteed to converge. Extensive experimental results confirm the effectiveness of the proposed method.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

DeltaDock: A Unified Framework for Accurate, Efficient, and Physically Reliable Molecular Docking

Jiaxian Yan
Zaixi Zhang
Jintao Zhu
Kai Zhang
Jianfeng Pei
Qi Liu

Molecular docking, a technique for predicting ligand binding poses, is crucial in structure-based drug design for understanding protein-ligand interactions. Recent advancements in docking methods, particularly those leveraging geometric deep learning (GDL), have demonstrated significant efficiency and accuracy advantages over traditional sampling methods. Despite these advancements, current methods are often tailored for specific docking settings, and limitations such as the neglect of protein side-chain structures, difficulties in handling large binding pockets, and challenges in predicting physically valid structures exist. To accommodate various docking settings and achieve accurate, efficient, and physically reliable docking, we propose a novel two-stage docking framework, DeltaDock, consisting of pocket prediction and site-specific docking. We innovatively reframe the pocket prediction task as a pocket-ligand alignment problem rather than direct prediction in the first stage. Then we follow a bi-level coarse-to-fine iterative refinement process to perform site-specific docking. Comprehensive experiments demonstrate the superior performance of DeltaDock. Notably, in the blind docking setting, DeltaDock achieves a 31\% relative improvement over the docking success rate compared with the previous state-of-the-art GDL model DiffDock. With the consideration of physical validity, this improvement increases to about 300\%.

PDF Details DOI

AAAI Conference 2024 Conference Paper

DiffRAW: Leveraging Diffusion Model to Generate DSLR-Comparable Perceptual Quality sRGB from Smartphone RAW Images

Mingxin Yi
Kai Zhang
Pei Liu
Tanli Zuo
Jingduo Tian

Deriving DSLR-quality sRGB images from smartphone RAW images has become a compelling challenge due to discernible detail disparity, color mapping instability, and spatial misalignment in RAW-sRGB data pairs. We present DiffRAW, a novel method that incorporates the diffusion model for the first time in learning RAW-to-sRGB mappings. By leveraging the diffusion model, our approach effectively learns the high-quality detail distribution of DSLR images, thereby enhancing the details of output images. Simultaneously, we use the RAW image as a diffusion condition to maintain image structure information such as contours and textures. To mitigate the interference caused by the color and spatial misalignment in training data pairs, we embed a color-position preserving condition within DiffRAW, ensuring that the output images do not exhibit color biases and pixel shift issues. To accelerate the inference process of DiffRAW, we designed the Domain Transform Diffusion Method, an efficient diffusion process with its corresponding reverse process. The Domain Transform Diffusion Method can reduce the required inference steps for diffusion model-based image restoration/enhancement algorithms while enhancing the quality of the generated images. Through evaluations on the ZRR dataset, DiffRAW consistently demonstrates state-of-the-art performance across all perceptual quality metrics (e.g., LPIPS, FID, MUSIQ), while achieving comparable results in PSNR and SSIM.

PDF Details DOI

ICML Conference 2024 Conference Paper

High-Order Contrastive Learning with Fine-grained Comparative Levels for Sparse Ordinal Tensor Completion

Yu Dai
Junchen Shen
Zijie Zhai
Danlin Liu
Jingyang Chen
Yu Sun
Ping Li
Jie Zhang

Contrastive learning is a powerful paradigm for representation learning with prominent success in computer vision and NLP, but how to extend its success to high-dimensional tensors remains a challenge. This is because tensor data often exhibit high-order mode-interactions that are hard to profile and with negative samples growing combinatorially faster than second-order contrastive learning; furthermore, many real-world tensors have ordinal entries that necessitate more delicate comparative levels. To solve the challenge, we propose High-Order Contrastive Tensor Completion (HOCTC), an innovative network to extend contrastive learning to sparse ordinal tensor data. HOCTC employs a novel attention-based strategy with query-expansion to capture high-order mode interactions even in case of very limited tokens, which transcends beyond second-order learning scenarios. Besides, it extends two-level comparisons (positive-vs-negative) to fine-grained contrast-levels using ordinal tensor entries as a natural guidance. Efficient sampling scheme is proposed to enforce such delicate comparative structures, generating comprehensive self-supervised signals for high-order representation learning. Extensive experiments show that HOCTC has promising results in sparse tensor completion in traffic/recommender applications.

Details

NeurIPS Conference 2024 Conference Paper

LRM-Zero: Training Large Reconstruction Models with Synthesized Data

Desai Xie
Sai Bi
Zhixin Shu
Kai Zhang
Zexiang Xu
Yi Zhou
Sören Pirk
Arie Kaufman

We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is automatically synthesized from simple primitive shapes with random texturing and augmentations (e. g. , height fields, boolean differences, and wireframes). Unlike previous 3D datasets (e. g. , Objaverse) which are often captured or crafted by humans to approximate real 3D data, Zeroverse completely ignores realistic global semantics but is rich in complex geometric and texture details that are locally similar to or even more intricate than real objects. We demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse, can achieve high visual quality in the reconstruction of real-world objects, competitive with models trained on Objaverse. We also analyze several critical design choices of Zeroverse that contribute to LRM-Zero's capability and training stability. Our work demonstrates that 3D reconstruction, one of the core tasks in 3D vision, can potentially be addressed without the semantics of real-world objects. The Zeroverse's procedural synthesis code and interactive visualization are available at: https: //desaixie. github. io/lrm-zero/.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

MambaSCI: Efficient Mamba-UNet for Quad-Bayer Patterned Video Snapshot Compressive Imaging

Zhenghao Pan
Haijin Zeng
Jiezhang Cao
Yongyong Chen
Kai Zhang
Yong Xu

Color video snapshot compressive imaging (SCI) employs computational imaging techniques to capture multiple sequential video frames in a single Bayer-patterned measurement. With the increasing popularity of quad-Bayer pattern in mainstream smartphone cameras for capturing high-resolution videos, mobile photography has become more accessible to a wider audience. However, existing color video SCI reconstruction algorithms are designed based on the traditional Bayer pattern. When applied to videos captured by quad-Bayer cameras, these algorithms often result in color distortion and ineffective demosaicing, rendering them impractical for primary equipment. To address this challenge, we propose the MambaSCI method, which leverages the Mamba and UNet architectures for efficient reconstruction of quad-Bayer patterned color video SCI. To the best of our knowledge, our work presents the first algorithm for quad-Bayer patterned SCI reconstruction, and also the initial application of the Mamba model to this task. Specifically, we customize Residual-Mamba-Blocks, which residually connect the Spatial-Temporal Mamba (STMamba), Edge-Detail-Reconstruction (EDR) module, and Channel Attention (CA) module. Respectively, STMamba is used to model long-range spatial-temporal dependencies with linear complexity, EDR is for better edge-detail reconstruction, and CA is used to compensate for the missing channel information interaction in Mamba model. Experiments demonstrate that MambaSCI surpasses state-of-the-art methods with lower computational and memory costs. PyTorch style pseudo-code for the core modules is provided in the supplementary materials. Code is at https: //github. com/PAN083/MambaSCI.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Neural Gaffer: Relighting Any Object via Diffusion

Haian Jin
Yuan Li
Fujun Luan
Yuanbo Xiangli
Sai Bi
Kai Zhang
Zexiang Xu
Jin Sun

Single-image relighting is a challenging task that involves reasoning about the complex interplay between geometry, materials, and lighting. Many prior methods either support only specific categories of images, such as portraits, or require special capture conditions, like using a flashlight. Alternatively, some methods explicitly decompose a scene into intrinsic components, such as normals and BRDFs, which can be inaccurate or under-expressive. In this work, we propose a novel end-to-end 2D relighting diffusion model, called Neural Gaffer, that takes a single image of any object and can synthesize an accurate, high-quality relit image under any novel environmental lighting condition, simply by conditioning an image generator on a target environment map, without an explicit scene decomposition. Our method builds on a pre-trained diffusion model, and fine-tunes it on a synthetic relighting dataset, revealing and harnessing the inherent understanding of lighting present in the diffusion model. We evaluate our model on both synthetic and in-the-wild Internet imagery and demonstrate its advantages in terms of generalization and accuracy. Moreover, by combining with other generative methods, our model enables many downstream 2D tasks, such as text-based relighting and object insertion. Our model can also operate as a strong relighting prior for 3D tasks, such as relighting a radiance field.

PDF Details DOI

AAAI Conference 2024 Conference Paper

PathAsst: A Generative Foundation AI Assistant towards Artificial General Intelligence of Pathology

Yuxuan Sun
Chenglu Zhu
Sunyi Zheng
Kai Zhang
Lin Sun
Zhongyi Shui
Yunlong Zhang
Honglin Li

As advances in large language models (LLMs) and multimodal techniques continue to mature, the development of general-purpose multimodal large language models (MLLMs) has surged, offering significant applications in interpreting natural images. However, the field of pathology has largely remained untapped, particularly in gathering high-quality data and designing comprehensive model frameworks. To bridge the gap in pathology MLLMs, we present PathAsst, a multimodal generative foundation AI assistant to revolutionize diagnostic and predictive analytics in pathology. The development of PathAsst involves three pivotal steps: data acquisition, CLIP model adaptation, and the training of PathAsst's multimodal generative capabilities. Firstly, we collect over 207K high-quality pathology image-text pairs from authoritative sources. Leveraging the advanced power of ChatGPT, we generate over 180K instruction-following samples. Furthermore, we devise additional instruction-following data specifically tailored for invoking eight pathology-specific sub-models we prepared, allowing the PathAsst to effectively collaborate with these models, enhancing its diagnostic ability. Secondly, by leveraging the collected data, we construct PathCLIP, a pathology-dedicated CLIP, to enhance PathAsst's capabilities in interpreting pathology images. Finally, we integrate PathCLIP with the Vicuna-13b and utilize pathology-specific instruction-tuning data to enhance the multimodal generation capacity of PathAsst and bolster its synergistic interactions with sub-models. The experimental results of PathAsst show the potential of harnessing AI-powered generative foundation model to improve pathology diagnosis and treatment processes. We open-source our dataset, as well as a comprehensive toolkit for extensive pathology data collection and preprocessing at https://github.com/superjamessyx/Generative-Foundation-AI-Assistant-for-Pathology.

PDF Details DOI

JBHI Journal 2024 Journal Article

Polygonal Approximation Learning for Convex Object Segmentation in Biomedical Images With Bounding Box Supervision

Wenhao Zheng
Jintai Chen
Kai Zhang
Jiahuan Yan
Jinhong Wang
Yi Cheng
Bang Du
Danny Z. Chen

As a common and critical medical image analysis task, deep learning based biomedical image segmentation is hindered by the dependence on costly fine-grained annotations. To alleviate this data dependence, in this article, a novel approach, called Polygonal Approximation Learning (PAL), is proposed for convex object instance segmentation with only bounding-box supervision. The key idea behind PAL is that the detection model for convex objects already contains the necessary information for segmenting them since their convex hulls, which can be generated approximately by the intersection of bounding boxes, are equivalent to the masks representing the objects. To extract the essential information from the detection model, a repeated detection approach is employed on biomedical images where various rotation angles are applied and a dice loss with the projection of the rotated detection results is utilized as a supervised signal in training our segmentation model. In biomedical imaging tasks involving convex objects, such as nuclei instance segmentation, PAL outperforms the known models (e. g. , BoxInst) that rely solely on box supervision. Furthermore, PAL achieves comparable performance with mask-supervised models including Mask R-CNN and Cascade Mask R-CNN. Interestingly, PAL also demonstrates remarkable performance on non-convex object instance segmentation tasks, for example, surgical instrument and organ instance segmentation.

Details DOI

IJCAI Conference 2024 Conference Paper

Pre-training General User Representation with Multi-type APP Behaviors

Yuren Zhang
Min Hou
Kai Zhang
Yuqing Yuan
Chao Song
Zhihao Ye
Enhong Chen
Yang Yu

In numerous user-centric services on mobile applications (apps), accurately mining user interests and generating effective user representations are paramount. Traditional approaches, which often involve training task-specific user representations, are becoming increasingly impractical due to their high computational costs and limited adaptability. This paper introduces a novel solution to this challenge: the Multi-type App-usage Fusion Network (MAFN). MAFN innovatively pre-trains universal user representations, leveraging multi-type app behaviors to overcome key limitations in existing methods. We address two primary challenges: 1) the varying frequency of user behaviors (ranging from low-frequency actions like (un)installations to high-frequency yet insightful app launches); and 2) the integration of multi-type behaviors to form a cohesive representation. Our approach involves the creation of novel pre-training tasks that harness self-supervised signals from diverse app behaviors, capturing both long-term and short-term user interests. MAFN's unique fusion approach effectively amalgamates these interests into a unified vector space, facilitating the development of a versatile, general-purpose user representation. With a practical workflow, extensive experiments with three typical downstream tasks on real-world datasets verify the effectiveness of our approach.

PDF Details DOI

ECAI Conference 2024 Conference Paper

Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolution

Tianyi Xu
Yijie Zhou
Xiaotao Hu
Kai Zhang
Anran Zhang
Xingye Qiu
Jun Xu

Arbitrary-scale super-resolution (ASSR) aims to learn a single model for image super-resolution at arbitrary magnifying scales. Existing ASSR networks typically comprise an off-the-shelf scale-agnostic feature extractor and an arbitrary scale upsampler. These feature extractors often use fixed network architectures to address different ASSR inference tasks, each of which is characterized by an input image and an upsampling scale. However, this overlooks the difficulty variance of super-resolution on different inference scenarios, where simple images or small SR scales could be resolved with less computational effort than difficult images or large SR scales. To tackle this difficulty variability, in this paper, we propose a Task-Aware Dynamic Transformer (TADT) as an input-adaptive feature extractor for efficient image ASSR. Our TADT consists of a multi-scale feature extraction backbone built upon groups of Multi-Scale Transformer Blocks (MSTBs) and a Task-Aware Routing Controller (TARC). The TARC predicts the inference paths within feature extraction backbone, specifically selecting MSTBs based on the input images and SR scales. The prediction of inference path is guided by a new loss function to trade-off the SR accuracy and efficiency. Experiments demonstrate that, when working with three popular arbitrary-scale upsamplers, our TADT achieves state-of-the-art ASSR performance when compared with mainstream feature extractors, but with relatively fewer computational costs. The code is available at https: //github. com/Tillyhere/TADT.

Details

EAAI Journal 2024 Journal Article

Theory guided Lagrange programming neural network for subsurface flow problems

Jian Wang
Xiaofeng Xue
Zhixue Sun
Jun Yao
El-Sayed M. El-Alfy
Kai Zhang
Witold Pedrycz
Jacek Mańdziuk

A deep learning model can perform efficient uncertainty quantification (UQ) for reservoir flow with uncertain model parameters, but usually requires large amounts of training data to ensure accuracy. However, the cost of obtaining large amounts of data is prohibitive, and the performance will deteriorate if sufficient training data is lacking. Alternatively, more interpretable neural networks with embedded physical laws have recently been used to solve partial differential equations as well as to solve UQ problems. This approach has received a lot of attention due to its low data volume requirements and its adherence to the laws of physics during the training process. In this paper, we propose a theory-guided framework based on a bilevel programming model with hard constraints to embed physical meaning in the model. Theory guided Lagrange programming neural network (TGLPNN) combines the method of Lagrange programming neural network approach where physical laws such as stochastic partial differential equations and boundary conditions are incorporated into the training process of a convolutional neural network. At the same time, the upper-level variables are iteratively optimized. The method based on Lagrange programming neural network inherently embeds physical laws in the network. Practical applications have shown that TGLPNN can provide higher prediction accuracy compared to state-of-the-art physics-driven methods and improved efficiency compared to numerical methods.

Details DOI

AAAI Conference 2024 Conference Paper

UMIE: Unified Multimodal Information Extraction with Instruction Tuning

Lin Sun
Kai Zhang
Qingyuan Li
Renze Lou

Multimodal information extraction (MIE) gains significant attention as the popularity of multimedia content increases. However, current MIE methods often resort to using task-specific model structures, which results in limited generalizability across tasks and underutilizes shared knowledge across MIE tasks. To address these issues, we propose UMIE, a unified multimodal information extractor to unify three MIE tasks as a generation problem using instruction tuning, being able to effectively extract both textual and visual mentions. Extensive experiments show that our single UMIE outperforms various state-of-the-art (SoTA) methods across six MIE datasets on three tasks. Furthermore, in-depth analysis demonstrates UMIE's strong generalization in the zero-shot setting, robustness to instruction variants, and interpretability. Our research serves as an initial step towards a unified MIE model and initiates the exploration into both instruction tuning and large language models within the MIE domain. Our code, data, and model are available at https://github.com/ZUCC-AI/UMIE.

PDF Details DOI

TCS Journal 2024 Journal Article

Updatable searchable symmetric encryption: Definitions and constructions

Xiwen Wang
Kai Zhang
Junqing Gong
Shi-Feng Sun
Jianting Ning

Searchable symmetric encryption (SSE) allows a client to search over encrypted data. To address the real threat of key compromise in practice, this work initiates the study of key rotation for SSE and introduces the notion of updatable SSE (USSE). In USSE, a client can issue a single update token that permits the server to convert existing encrypted data from the old key to the new key. In particular, • we formalize the syntax of USSE and define the security model that captures the inference of key, search token, update token and encrypted data with bi-/uni-/no-directional key updates and bi-/uni-directional encrypted data updates. • we present a USSE scheme that supports conjunctive queries with sub-linear complexity, and prove its security with no-directional key update and bi-directional encrypted data update. We also give extensions for concerning different key/encrypted data updates. • we implement our USSE schemes and evaluate the performance with real-world dataset, which illustrates that our schemes achieve practically acceptable computational overhead and communication cost. Technically, our formalization of USSE is inspired by updatable encryption (UE); our USSE schemes are obtained by a semi-generic transformation from Cash et al. 's SSE and UE. The transformation itself only relies on DL and DBDH assumptions. We believe that the transformation is of independent interest and applicable to other scenarios where the SSE systems follow the structure of Cash et al. 's work.

Details DOI

AAAI Conference 2024 Conference Paper

π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control

Yin Gu
Kai Zhang
Qi Liu
Weibo Gao
Longfei Li
Jun Zhou

The recent advancements in Deep Reinforcement Learning (DRL) have significantly enhanced the performance of adaptive Traffic Signal Control (TSC). However, DRL policies are typically represented by neural networks, which are over-parameterized black-box models. As a result, the learned policies often lack interpretability, and cannot be deployed directly in the real-world edge hardware due to resource constraints. In addition, the DRL methods often exhibit limited generalization performance, struggling to generalize the learned policy to other geographical regions. These factors limit the practical application of learning-based approaches. To address these issues, we suggest the use of an inherently interpretable program for representing the control policy. We present a new approach, Programmatic Interpretable reinforcement learning for traffic signal control (π-light), designed to autonomously discover non-differentiable programs. Specifically, we define a Domain Specific Language (DSL) and transformation rules for constructing programs, and utilize Monte Carlo Tree Search (MCTS) to find the optimal program in a discrete space. Extensive experiments demonstrate that our method consistently outperforms baseline approaches. Moreover, π-Light exhibits superior generalization capabilities compared to DRL, enabling training and evaluation across intersections from different cities. Finally, we analyze how the learned program policies can directly deploy on edge devices with extremely limited resources.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

AdaptSSR: Pre-training User Model with Augmentation-Adaptive Self-Supervised Ranking

Yang Yu
Qi Liu
Kai Zhang
Yuren Zhang
Chao Song
Min Hou
Yuqing Yuan
Zhihao Ye

User modeling, which aims to capture users' characteristics or interests, heavily relies on task-specific labeled data and suffers from the data sparsity issue. Several recent studies tackled this problem by pre-training the user model on massive user behavior sequences with a contrastive learning task. Generally, these methods assume different views of the same behavior sequence constructed via data augmentation are semantically consistent, i. e. , reflecting similar characteristics or interests of the user, and thus maximizing their agreement in the feature space. However, due to the diverse interests and heavy noise in user behaviors, existing augmentation methods tend to lose certain characteristics of the user or introduce noisy behaviors. Thus, forcing the user model to directly maximize the similarity between the augmented views may result in a negative transfer. To this end, we propose to replace the contrastive learning task with a new pretext task: Augmentation-Adaptive SelfSupervised Ranking (AdaptSSR), which alleviates the requirement of semantic consistency between the augmented views while pre-training a discriminative user model. Specifically, we adopt a multiple pairwise ranking loss which trains the user model to capture the similarity orders between the implicitly augmented view, the explicitly augmented view, and views from other users. We further employ an in-batch hard negative sampling strategy to facilitate model training. Moreover, considering the distinct impacts of data augmentation on different behavior sequences, we design an augmentation-adaptive fusion mechanism to automatically adjust the similarity order constraint applied to each sample based on the estimated similarity between the augmented views. Extensive experiments on both public and industrial datasets with six downstream tasks verify the effectiveness of AdaptSSR.

PDF Details

TCS Journal 2023 Journal Article

Bounded-collusion decentralized ABE with sublinear parameters

Jun Zhao
Minghao Yang
Junqing Gong
Kai Zhang
Haifeng Qian

In this paper, we propose a decentralized ABE scheme against bounded collusion which means the number of users in the system is a-prior bounded. The scheme enjoys public key and ciphertext of sublinear sizes in the number of users in the system while all prior constructions require linear sizes. Besides, our scheme achieves semi-adaptive security under bilateral k-Lin assumption and SXDH assumption in a pairing group. Keep the same as the previous constructions, the scheme supports monotone span program as a policy and does not rely on the random oracle. Technically, we follow Wang et al. 's “linear secret sharing scheme (LSSS) + inner-product functional encryption (IPFE)” paradigm [PKC'19] and use (an extended variant of) functional encryption for quadratic functions (QFE) in the place of IPFE. By this, we encrypt with sublinear-size random coins and later expand them to linear-size entropy for security proof. Roughly, the use of QFE requires bilateral k-Lin assumption while the entropy expansion relies on SXDH.

Details DOI

IJCAI Conference 2023 Conference Paper

Keep Skills in Mind: Understanding and Implementing Skills in Commonsense Question Answering

Meikai Bao
Qi Liu
Kai Zhang
Ye Liu
Linan Yue
Longfei Li
Jun Zhou

Commonsense Question Answering (CQA) aims to answer questions that require human commonsense. Closed-book CQA, as one of the subtasks, requires the model to answer questions without retrieving external knowledge, which emphasizes the importance of the model's problem-solving ability. Most previous methods relied on large-scale pre-trained models to generate question-related knowledge while ignoring the crucial role of skills in the process of answering commonsense questions. Generally, skills refer to the learned ability in performing a specific task or activity, which are derived from knowledge and experience. In this paper, we introduce a new approach named Dynamic Skill-aware Commonsense Question Answering (DSCQA), which transcends the limitations of traditional methods by informing the model about the need for each skill in questions and utilizes skills as a critical driver in CQA process. To be specific, DSCQA first employs commonsense skill extraction module to generate various skill representations. Then, DSCQA utilizes dynamic skill module to generate dynamic skill representations. Finally, in perception and emphasis module, various skills and dynamic skill representations are used to help question-answering process. Experimental results on two publicly available CQA datasets show the effectiveness of our proposed model and the considerable impact of introducing skills.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing

Kai Zhang
Lingbo Mo
Wenhu Chen
Huan Sun
Yu Su

Text-guided image editing is widely needed in daily life, ranging from personal use to professional applications such as Photoshop. However, existing methods are either zero-shot or trained on an automatically synthesized dataset, which contains a high volume of noise. Thus, they still require lots of manual tuning to produce desirable outcomes in practice. To address this issue, we introduce MagicBrush, the first large-scale, manually annotated dataset for instruction-guided real image editing that covers diverse scenarios: single-turn, multi-turn, mask-provided, and mask-free editing. MagicBrush comprises over 10K manually annotated triplets (source image, instruction, target image), which supports trainining large-scale text-guided image editing models. We fine-tune InstructPix2Pix on MagicBrush and show that the new model can produce much better images according to human evaluation. We further conduct extensive experiments to evaluate current image editing baselines from multiple dimensions including quantitative, qualitative, and human evaluations. The results reveal the challenging nature of our dataset and the gap between current baselines and real-world editing needs.

PDF Details

IROS Conference 2023 Conference Paper

Navigation Among Movable Obstacles Using Machine Learning Based Total Time Cost Optimization

Kai Zhang
Eric Lucet
Julien Alexandre Dit Sandretto
David Filliat

Most navigation approaches treat obstacles as static objects and choose to bypass them. However, the detour could be costly or could lead to failures in indoor environments. The recently developed navigation among movable obstacles (NAMO) methods prefer to remove all the movable obstacles blocking the way, which might be not the best choice when planning and moving obstacles takes a long time. We propose a pipeline where the robot solves the NAMO problems by optimizing the total time to reach the goal. This is achieved by a supervised learning approach that can predict the time of planning and performing obstacle motion before actually doing it if this leads to faster goal reaching. Besides, a pose generator based on reinforcement learning is proposed to decide where the robot can move the obstacle. The method is evaluated in two kinds of simulation environments and the results demonstrate its advantages compared to the classical bypass and obstacle removal strategies.

Details

EAAI Journal 2023 Journal Article

Testing and performance analysis of an integrated electromagnetic and hydraulic retarder for heavy-duty vehicles

Kai Zhang
Huichao Shang
Jing Xu
Jigao Niu
Yonggao Yue

Hydraulic retarder and eddy current retarder are important auxiliary braking component, which installed on heavy-duty vehicles. However, the hydraulic retarder has poor braking performance at low speeds, while the eddy-current retarder presents serious thermal recession at high speed. To overcome these issues and ensure safe driving during downhill conditions, a novel Integrated Electromagnetic and Hydraulic Retarder (IEHR) is proposed. The design includes a rotor with blades, double salient poles, and a coil built in a C-type stator. The electromagnetic field model of the IEHR is established and analyzed using the Finite Element Method. Moreover, the nonlinear eddy viscosity model of the flow field is generated and analyzed using the Computational Fluid Dynamics. The Blend braking control strategy is adopted to distribute the braking torque at high speed and low speed The IEHR performance was tested at a test bench. The test results show that the calculated braking characteristics of the electromagnetic and hydraulic subsystems were reasonable agreement with the measured results. The road test was carried out on a vehicle with a total mass of 49 tons. The deceleration and torque peaked at 0. 75 m/s2 and 3620 N m, respectively. These results indicated that the IEHR has better performance than conventional hydraulic retarders and eddy current retarder, and is more suitable for heavy vehicles.

Details DOI

YNICL Journal 2023 Journal Article

The altered spontaneous neural activity in patients with Parkinson's disease and its predictive value for the motor improvement of deep brain stimulation

Bowen Yang
Xiu Wang
Jiajie Mo
Zilin Li
Wenhan Hu
Chao Zhang
Baotian Zhao
Dongmei Gao

BACKGROUND: This study aims to investigate the altered spontaneous neural activity in patients with Parkinson's disease (PD) revealed by amplitudes of low-frequency fluctuations (ALFF) of resting-state fMRI, and the feasibility of using ALFF as neuroimaging predictors for motor improvement after bilateral subthalamic nucleus (STN) deep brain stimulation (DBS). METHODS: Fourty-four patients and 44 healthy controls were included in this study. First, the ALFF of patients with PD was compared with that of controls; then significant clusters were correlated with motor improvement after DBS (unified Parkinson's disease rating scale (UPDRS-III)) and other clinical variables. Second, regression and classification of the machine learning models were conducted to predict motor improvement after DBS. Receiver operating characteristic (ROC) analysis was used to evaluate the performance of the classification model. RESULTS: Compared with healthy controls, patients with PD showed increased ALFF in the bilateral motor area and decreased ALFF in the bilateral temporal cortex and cerebellum. The Hoehn-Yahr stages correlated with ALFF within the bilateral cerebellum (p = 0.021), and UPDRS-III improvement correlated with ALFF in the left (p < 0.001) and right (p = 0.005) motor areas. The regression model showed a significant correlation between the predicted and observed UPDRS-III changes (R = 0.65, p < 0.001). The ROC analysis revealed an area under the curve (AUC) of 0.94 which differentiated moderate and superior DBS responders. CONCLUSION: The results revealed altered ALFF patterns in patients with PD and their correlations with clinical variables. Both binary and continuous ALFF can potentially serve as predictive biomarkers for DBS response.

Details DOI

NeurIPS Conference 2022 Conference Paper

APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction

Bencheng Yan
Pengjie Wang
Kai Zhang
Feng Li
Hongbo Deng
Jian Xu
Bo Zheng

In many web applications, deep learning-based CTR prediction models (deep CTR models for short) are widely adopted. Traditional deep CTR models learn patterns in a static manner, i. e. , the network parameters are the same across all the instances. However, such a manner can hardly characterize each of the instances which may have different underlying distributions. It actually limits the representation power of deep CTR models, leading to sub-optimal results. In this paper, we propose an efficient, effective, and universal module, named as Adaptive Parameter Generation network (APG), which can dynamically generate parameters for deep CTR models on-the-fly based on different instances. Extensive experimental evaluation results show that APG can be applied to a variety of deep CTR models and significantly improve their performance. Meanwhile, APG can reduce the time cost by 38. 7\% and memory usage by 96. 6\% compared to a regular deep CTR model. We have deployed APG in the industrial sponsored search system and achieved 3\% CTR gain and 1\% RPM gain respectively.

PDF Details

NeurIPS Conference 2022 Conference Paper

Recurrent Video Restoration Transformer with Guided Deformable Attention

Jingyun Liang
Yuchen Fan
Xiaoyu Xiang
Rakesh Ranjan
Eddy Ilg
Simon Green
Jiezhang Cao
Kai Zhang

Video restoration aims at restoring multiple high-quality frames from multiple low-quality frames. Existing video restoration methods generally fall into two extreme cases, i. e. , they either restore all frames in parallel or restore the video frame by frame in a recurrent way, which would result in different merits and drawbacks. Typically, the former has the advantage of temporal information fusion. However, it suffers from large model size and intensive memory consumption; the latter has a relatively small model size as it shares parameters across frames; however, it lacks long-range dependency modeling ability and parallelizability. In this paper, we attempt to integrate the advantages of the two cases by proposing a recurrent video restoration transformer, namely RVRT. RVRT processes local neighboring frames in parallel within a globally recurrent framework which can achieve a good trade-off between model size, effectiveness, and efficiency. Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature. Within each clip, different frame features are jointly updated with implicit feature aggregation. Across different clips, the guided deformable attention is designed for clip-to-clip alignment, which predicts multiple relevant locations from the whole inferred clip and aggregates their features by the attention mechanism. Extensive experiments on video super-resolution, deblurring, and denoising show that the proposed RVRT achieves state-of-the-art performance on benchmark datasets with balanced model size, testing memory and runtime.

PDF Details

NeurIPS Conference 2022 Conference Paper

SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization

Chuanyang Zheng
Zheyang Li
Kai Zhang
Zhi Yang
Wenming Tan
Jun Xiao
Ye Ren
Shiliang Pu

Vision Transformers (ViTs) yield impressive performance across various vision tasks. However, heavy computation and memory footprint make them inaccessible for edge devices. Previous works apply importance criteria determined independently by each individual component to prune ViTs. Considering that heterogeneous components in ViTs play distinct roles, these approaches lead to suboptimal performance. In this paper, we introduce joint importance, which integrates essential structural-aware interactions between components for the first time, to perform collaborative pruning. Based on the theoretical analysis, we construct a Taylor-based approximation to evaluate the joint importance. This guides pruning toward a more balanced reduction across all components. To further reduce the algorithm complexity, we incorporate the interactions into the optimization function under some mild assumptions. Moreover, the proposed method can be seamlessly applied to various tasks including object detection. Extensive experiments demonstrate the effectiveness of our method. Notably, the proposed approach outperforms the existing state-of-the-art approaches on ImageNet, increasing accuracy by 0. 7% over the DeiT-Base baseline while saving 50% FLOPs. On COCO, we are the first to show that 70% FLOPs of FasterRCNN with ViT backbone can be removed with only 0. 3% mAP drop. The code is available at https: //github. com/hikvision-research/SAViT.

PDF Details

ICRA Conference 2022 Conference Paper

Self-supervised Transparent Liquid Segmentation for Robotic Pouring

Gautham Narayan Narasimhan
Kai Zhang
Ben Eisner
Xingyu Lin
David Held

Liquid state estimation is important for robotics tasks such as pouring; however, estimating the state of transparent liquids is a challenging problem. We propose a novel segmentation pipeline that can segment transparent liquids such as water from a static, RGB image without requiring any manual annotations or heating of the liquid for training. Instead, we use a generative model that is capable of translating images of colored liquids into synthetically generated transparent liquid images, trained only on an unpaired dataset of colored and transparent liquid images. Segmentation labels of colored liquids are obtained automatically using background subtraction. Our experiments show that we are able to accurately predict a segmentation mask for transparent liquids without requiring any manual annotations. We demonstrate the utility of transparent liquid segmentation in a robotic pouring task that controls pouring by perceiving the liquid height in a transparent cup. Accompanying video and supplementary materials can be found at https://sites.google.com/view/transparentliquidpouring.

Details

AAAI Conference 2022 Conference Paper

Structural Landmarking and Interaction Modelling: A “SLIM” Network for Graph Classification

Yaokang Zhu
Kai Zhang
Jun Wang
Haibin Ling
Jie Zhang
Hongyuan Zha

Graph neural networks are a promising architecture for learning and inference with graph-structured data. Yet, how to generate informative, fixed-dimensional graph-level features for graphs with varying size and topology can still be challenging. Typically, this is achieved through graph-pooling, which summarizes a graph by compressing all its nodes into a single vector after convolutional operations. Is such a “collapsing-style” graph-pooling the only choice for graph classification? From complex system’s point of view, properties of a complex system arise largely from the interaction among its components. Therefore, we speculate that preserving the interacting relation between parts, instead of pooling them together, could benefit system-level prediction. To verify this, we propose SLIM, a graph neural network model for Structural Landmarking and Interaction Modelling. The main idea is to compute a set of end-to-end optimizable sub-structure landmarks, so that any input graph can be projected onto these (spatially) local structural representatives for a faithful, global characterization. By doing this, explicit interaction between component parts of a graph can be leveraged directly in generating useful graphlevel representations despite significant topological variations. Encouraging results are observed on benchmark datasets for graph classification, demonstrating the value of interaction modelling in the design of graph neural networks.

PDF Details

NeurIPS Conference 2022 Conference Paper

WT-MVSNet: Window-based Transformers for Multi-view Stereo

Jinli Liao
Yikang Ding
Yoli Shavit
Dihe Huang
Shihao Ren
Jia Guo
Wensen Feng
Kai Zhang

Recently, Transformers have been shown to enhance the performance of multi-view stereo by enabling long-range feature interaction. In this work, we propose Window-based Transformers (WT) for local feature matching and global feature aggregation in multi-view stereo. We introduce a Window-based Epipolar Transformer (WET) which reduces matching redundancy by using epipolar constraints. Since point-to-line matching is sensitive to erroneous camera pose and calibration, we match windows near the epipolar lines. A second Shifted WT is employed for aggregating global information within cost volume. We present a novel Cost Transformer (CT) to replace 3D convolutions for cost volume regularization. In order to better constrain the estimated depth maps from multiple views, we further design a novel geometric consistency loss (Geo Loss) which punishes unreliable areas where multi-view consistency is not satisfied. Our WT multi-view stereo method (WT-MVSNet) achieves state-of-the-art performance across multiple datasets and ranks $1^{st}$ on Tanks and Temples benchmark. Code will be available upon acceptance.

PDF Details

AAAI Conference 2021 Conference Paper

Adversarial Language Games for Advanced Natural Language Intelligence

Yuan Yao
Haoxi Zhong
Zhengyan Zhang
Xu Han
Xiaozhi Wang
Kai Zhang
Chaojun Xiao
Guoyang Zeng

We study the problem of adversarial language games, in which multiple agents with conflicting goals compete with each other via natural language interactions. While adversarial language games are ubiquitous in human activities, little attention has been devoted to this field in natural language processing. In this work, we propose a challenging adversarial language game called Adversarial Taboo as an example, in which an attacker and a defender compete around a target word. The attacker is tasked with inducing the defender to utter the target word invisible to the defender, while the defender is tasked with detecting the target word before being induced by the attacker. In Adversarial Taboo, a successful attacker and defender need to hide or infer the intention, and induce or defend during conversations. This requires several advanced language abilities, such as adversarial pragmatic reasoning and goal-oriented language interactions in open domain, which will facilitate many downstream NLP tasks. To instantiate the game, we create a game environment and a competition platform. Comprehensive experiments on several baseline attack and defense strategies show promising and interesting results, based on which we discuss some directions for future research. The code and datasets of this paper can be obtained from https: //github. com/thunlp/AdversarialTaboo.

PDF Details

IS Journal 2021 Journal Article

An Efficient Solution to Detect Common Topologies in Money Launderings Based on Coupling and Connection

Jing He
Jiao Tian
Yuanyuan Wu
Xinyi Cia
Kai Zhang
Mengjiao Guo
Hui Zheng
Junfeng Wu

In recent years, money laundering has become much easier to be achieved but more challenging to be detected than before, which has enormous adversary effects on finance, military, and other related fields. In the real-time scenario, every money laundering case has a unique structure in terms of transactions. It is not sufficient to detect suspicious behavior by just following the probability theory, where usually the thresholds are given by experts. Since the crime of money laundering is more prevalent and sophisticated nowadays, it will increase the complexity of the detection if the accounts with the personal information are combined with the form of the transaction topology. Hence, the graph topology analysis could be used for antimoney laundering tools. This article proposes eight common topologies based on coupling and connection from simple to much more complicated structures to solve various kinds of problems concerning money laundering in the real world. Moreover, we also propose an efficient solution based on graph and subgraph isomorphism and distance measurement to detect money laundering behavior. In this way, the detection of money laundering behavior will be more efficient and effective for various situations while referencing the proposed eight topological structures.

Details DOI

IS Journal 2021 Journal Article

Anomalous Event Sequence Detection

Boxiang Dong
Zhengzhang Chen
Lu-An Tang
Haifeng Chen
Hui Wang
Kai Zhang
Ying Lin
Zhichun Li

Anomaly detection has been widely applied in modern data-driven security applications to detect abnormal events/entities that deviate from the majority. However, less work has been done in terms of detecting suspicious event sequences/paths, which are better discriminators than single events/entities for distinguishing normal and abnormal behaviors in complex systems such as cyber-physical systems. A key and challenging step in this endeavor is how to discover those abnormal event sequences from millions of system event records in an efficient and accurate way. To address this issue, we propose NINA, a network diffusion based algorithm for identifying anomalous event sequences. Experimental results on both static and streaming data show that NINA is efficient (processes about 2 million records per minute) and accurate.

Details DOI

IROS Conference 2021 Conference Paper

Continuous Robust Trajectory Tracking Control for Autonomous Ground Vehicles Considering Lateral and Longitudinal Kinematics and Dynamics via Recursive Backstepping

Ming Xin 0002
Yue Yin
Kai Zhang
David Lackner
Zhongchao Ren
Mark A. Minor

Maintaining lateral and longitudinal trajectory tracking accuracy is challenging for autonomous ground vehicles (AGVs). This paper considers kinematics and dynamics of longitudinal and lateral motion to form a novel composite structure considering the cross-impacts of acceleration and steering commands on tracking errors in the lateral and longitudinal directions, respectively. The multi-tiered structure uses backstepping with smooth robust control to iteratively map kinematics-based velocity and yaw rate commands to slip-yaw dynamics-based acceleration and steering commands. In kinematics, longitudinal tracking error is stabilized by sliding mode control (SMC) while variable structure control (VSC) stabilizes lateral tracking error and balances tracking accuracy and steering gracefulness. Backstepping extends these commands through vehicle dynamics to provide robust steering and acceleration commands. Cross impacts between lateral and longitudinal motion is addressed by vehicle modeling and controller designs. A robust observer is applied for sideslip estimation to reject uncertainties. Peaking from the high gain observer and robust control is addressed. Stability analysis is provided and field experiments on an open road demonstrate and validate effectiveness of the controllers.

Details

EAAI Journal 2021 Journal Article

Fault source location of wind turbine based on heterogeneous nodes complex network

Kai Zhang
Baoping Tang
Lei Deng
Xiaoxia Yu
Jing Wei

Previous studies regarding wind turbine health management based on wind turbine supervisory control and data acquisition (SCADA) signals focused on detecting incipient failure. In this case, most current SCADA alarm systems adopt a single or multi-signal threshold trigger alarm. When a wind turbine is operating in a large wind farm, it is catastrophic for maintenance personnel lied in the reason that the current SCADA system may trigger many alarms in the short term. Therefore, the fault source must be identified and the candidate fault sources sorted. This study aims to obtain the fault location of wind turbines based on a complex network, which does not require a large number of historical monitoring signals. A directed graph of the wind turbine and fault location theoretical framework is established; an improvement is achieved based on the heterogeneity of nodes abstracted by components of a wind turbine and variables of SCADA. The proposed fault location method based on heterogeneous nodes complex networks (HNCN) is improved based on degree-based mean-field theory and is verified by spreading simulated data and four SCADA alarm cases at a wind farm located in Liaoning, China.

Details DOI

AAAI Conference 2021 Conference Paper

NeuralAC: Learning Cooperation and Competition Effects for Match Outcome Prediction

Yin Gu
Qi Liu
Kai Zhang
Zhenya Huang
Runze Wu
Jianrong Tao

Match outcome prediction in group comparison setting is a challenging but important task. Existing works mainly focus on learning individual effects or mining limited interactions between teammates, which is not sufficient for capturing complex interactions between teammates as well as between opponents. Besides, the importance of interacting with different characters is still largely underexplored. To this end, we propose a novel Neural Attentional Cooperation-competition model (NeuralAC), which incorporates weighted-cooperation effects (i. e. , intra-team interactions) and weighted-competition effects (i. e. , inter-team interactions) for predicting match outcomes. Specifically, we first project individuals to latent vectors and learn complex interactions through deep neural networks. Then, we design two novel attention-based mechanisms to capture the importance of intra-team and inter-team interactions, which enhance NeuralAC with both accuracy and interpretability. Furthermore, we demonstrate NeuralAC can generalize several previous works. To evaluate the performances of NeuralAC, we conduct extensive experiments on four E-sports datasets. The experimental results clearly verify the effectiveness of NeuralAC compared with several state-of-the-art methods.

PDF Details

AAAI Conference 2021 Conference Paper

RpBERT: A Text-image Relation Propagation-based BERT Model for Multimodal NER

Lin Sun
Jiquan Wang
Kai Zhang
Yindu Su
Fangsheng Weng

Recently multimodal named entity recognition (MNER) has utilized images to improve the accuracy of NER in tweets. However, most of the multimodal methods use attention mechanisms to extract visual clues regardless of whether the text and image are relevant. Practically, the irrelevant textimage pairs account for a large proportion in tweets. The visual clues that are unrelated to the texts will exert uncertain or even negative effects on multimodal model learning. In this paper, we introduce a method of text-image relation propagation into the multimodal BERT model. We integrate soft or hard gates to select visual clues and propose a multitask algorithm to train on the MNER datasets. In the experiments, we deeply analyze the changes in visual attention before and after the use of text-image relation propagation. Our model achieves state-of-the-art performance on the MNER datasets.

PDF Details

TCS Journal 2020 Journal Article

A post-quantum hybrid encryption based on QC-LDPC codes in the multi-user setting

Luping Wang
Jie Chen
Kai Zhang
Haifeng Qian

The encryption schemes based on coding theory are one of the most accredited choices in post-quantum scenario, where QC-LDPC codes are usually employed to construct concrete schemes due to the well security and good efficiency. In this work, we introduce a new IND-CCA secure multi-instance framework for code-based hybrid encryption primitive in the random oracle model, which is derived from our new multi-instance KEM and DEM building modules. We note that previous multi-instance KEM and DEM are usually derived from single-instance KEM and DEM, and hence suffers from large parameter sizes and security loss. Nevertheless, our multi-instance KEM is a direct construction based on a key generation function and a one-way trapdoor function, and our multi-instance DEM is constructed from a standard DEM and MAC with a tag in the input to achieve a tighter security loss. Finally, we present a IND-CCA secure multi-instance hybrid encryption scheme based on QC-LDPC codes in the random oracle model, where the scheme achieves small private key size and only consumes addition and multiplication operations over F 2 [ x ].

Details DOI

ICRA Conference 2020 Conference Paper

Slip-Based Nonlinear Recursive Backstepping Path Following Controller for Autonomous Ground Vehicles

Ming Xin 0002
Kai Zhang
David Lackner
Mark A. Minor

Path following accuracy and error convergence with graceful motion in vehicle steering control is challenging due to the competing nature of these requirements, especially across a range of operating speeds. This work is founded upon slip-based kinematic and dynamic models, which allow derivation of controllers considering error due to sideslip and the mapping between steering commands and graceful lateral motion. A novel recursive backstepping steering controller is proposed that better couples yaw-rate based path following commands to steering angle and rate. Observer based sideslip estimates are combined with heading error in the kinematic controller to provide feedforward slip compensation. Path following error is compensated by a Variable Structure Controller (VSC) to balance graceful motion, path following error, and robustness. Yaw rate commands are used by a backstepping dynamic controller to generate robust steering commands. A High Gain Observer (HGO) estimates sideslip and yaw rate for output feedback control. Stability analysis is provided and peaking is addressed. Field experimental results evaluate the work and provide comparisons to MPC.

Details

JBHI Journal 2020 Journal Article

Towards Wearable and Flexible Sensors and Circuits Integration for Stress Monitoring

Ching-Mei Chen
Salzitsa Anastasova
Kai Zhang
Bruno G. Rosa
Benny P. L. Lo
Hazel E. Assender
Guang-Zhong Yang

Excessive stress is one of the main causes of mental illness. Long-term exposure of stress could affect one's physiological wellbeing (such as hypertension) and psychological condition (such as depression). Multisensory information such as heart rate variability (HRV) and pH can provide suitable information about mental and physical stress. This paper proposes a novel approach for stress condition monitoring using disposable flexible sensors. By integrating flexible amplifiers with a commercially available flexible polyvinylidene difluoride (PVDF) mechanical deformation sensor and a pH-type chemical sensor, the proposed system can detect arterial pulses from the neck and pH levels from sweat located in the back of the body. The system uses organic thin film transistor (OTFT)-based signal amplification front-end circuits with modifications to accommodate the dynamic signal ranges obtained from the sensors. The OTFTs were manufactured on a low-cost flexible polyethylene naphthalate (PEN) substrate using a coater capable of Roll-to-Roll (R2R) deposition. The proposed system can capture physiological indicators with data interrogated by Near Field Communication (NFC). The device has been successfully tested with healthy subjects, demonstrating its feasibility for real-time stress monitoring.

Details DOI

JBHI Journal 2019 Journal Article

Automated Layer Segmentation of Retinal Optical Coherence Tomography Images Using a Deep Feature Enhanced Structured Random Forests Classifier

Xiaoming Liu
Tianyu Fu
Zhifang Pan
Dong Liu
Wei Hu
Jun Liu
Kai Zhang

Optical coherence tomography (OCT) is a high-resolution and noninvasive imaging modality that has become one of the most prevalent techniques for ophthalmic diagnosis. Retinal layer segmentation is very crucial for doctors to diagnose and study retinal diseases. However, manual segmentation is often a time-consuming and subjective process. In this work, we propose a new method for automatically segmenting retinal OCT images, which integrates deep features and hand-designed features to train a structured random forests classifier. The deep convolutional features are learned from deep residual network. With the trained classifier, we can get the contour probability graph of each layer; finally, the shortest path is employed to achieve the final layer segmentation. The experimental results show that our method achieves good results with the mean layer contour error of 1. 215 pixels, whereas that of the state of the art was 1. 464 pixels, and achieves an F1-score of 0. 885, which is also better than 0. 863 that is obtained by the state of the art method.

Details DOI

TCS Journal 2019 Journal Article

Efficient public key encryption with equality test in the standard model

Kai Zhang
Jie Chen
Hyung Tae Lee
Haifeng Qian
Huaxiong Wang

Public key encryption with equality test (PKEET) is a special kind of public encryption scheme (PKE) that allows a tester to perform equality tests on ciphertexts generated by different public keys as well as the same public key. This feature enables us to apply PKEET to various scenarios in practice, such as efficient data management on encrypted databases and spam filtering in encrypted email systems. From these reasons, since Yang et al. [1] first proposed the concept of PKEET, there have been proposed many PKEET schemes to improve efficiency or to enhance functionalities. However, to the best of our knowledge, almost all existing schemes were presented under assuming the existence of random oracles, except for generic construction proposed by Lee et al. On the other hand, their generic approach for PKEET employs a 2-level hierarchical identity-based encryption and a strongly unforgeable one-time signature, which suffers from low efficiency. In this paper, we propose an efficient PKEET scheme under a specific cryptographic assumption in the standard model. To this end, we first encrypt a message and its hash value in a parallel way by following the recently proposed strategy. Then, to prevent adaptive chosen ciphertext attacks (CCA2), we give a link between them by adapting the technique which was originally proposed for identity-based encryption and previously exploited to design efficient CCA2-secure PKE schemes. We show that our proposed construction satisfies formal security requirements for PKEET under the decisional bilinear Diffie–Hellman (DBDH) assumption in the standard model. As a result, we obtain a new PKEET scheme which has shorter ciphertext and trapdoor sizes, and improves computational costs for encryption, decryption, and test algorithms, by about 60%, 77%, and 66%, respectively, compared to a PKEET instantiation obtained by the prior generic framework.

Details DOI

ICML Conference 2019 Conference Paper

Greedy Orthogonal Pivoting Algorithm for Non-Negative Matrix Factorization

Kai Zhang
Sheng Zhang
Jun Liu
Jun Wang
Jie Zhang

Non-negative matrix factorization is a powerful tool for learning useful representations in the data and has been widely applied in many problems such as data mining and signal processing. Orthogonal NMF, which can improve the locality of decomposition, has drawn considerable interest in solving clustering problems in recent years. However, imposing simultaneous non-negative and orthogonal structure can be quite difficult, and so existing algorithms can only solve it approximately. To address this challenge, we propose an innovative procedure called Greedy Orthogonal Pivoting Algorithm (GOPA). The GOPA algorithm fully exploits the sparsity of non-negative orthogonal solutions to break the global problem into a series of local optimizations, in which an adaptive subset of coordinates are updated in a greedy, closed-form manner. The biggest advantage of GOPA is that it promotes exact orthogonality and provides solid empirical evidence that stronger orthogonality does contribute favorably to better clustering performance. On the other hand, we further design randomized and parallel version of GOPA, which can further reduce the computational cost and improve accuracy, making it suitable for large data.

Details

AAAI Conference 2019 Conference Paper

Interactive Attention Transfer Network for Cross-Domain Sentiment Classification

Kai Zhang
Hefu Zhang
Qi Liu
Hongke Zhao
Hengshu Zhu
Enhong Chen

Cross-domain sentiment classification refers to utilizing useful knowledge in the source domain to help sentiment classification in the target domain which has few or no labeled data. Most existing methods mainly concentrate on extracting common features between domains. Unfortunately, they cannot fully consider the effects of the aspect (e. g. , the battery life in reviewing an electronic product) information of the sentences. In order to better solve this problem, we propose an Interactive Attention Transfer Network (IATN) for crossdomain sentiment classification. IATN provides an interactive attention transfer mechanism, which can better transfer sentiment across domains by incorporating information of both sentences and aspects. Specifically, IATN comprises two attention networks, one of them is to identify the common features between domains through domain classification, and the other aims to extract information from the aspects by using the common features as a bridge. Then, we conduct interactive attention learning for those two networks so that both the sentences and the aspects can influence the final sentiment representation. Extensive experiments on the Amazon reviews dataset and crowdfunding reviews dataset not only demonstrate the effectiveness and universality of our method, but also give an interpretable way to track the attention information for sentiment.

PDF Details

AAAI Conference 2019 Conference Paper

Learning Adaptive Random Features

Yanjun Li
Kai Zhang
Jun Wang
Sanjiv Kumar

Random Fourier features are a powerful framework to approximate shift invariant kernels with Monte Carlo integration, which has drawn considerable interest in scaling up kernel-based learning, dimensionality reduction, and information retrieval. In the literature, many sampling schemes have been proposed to improve the approximation performance. However, an interesting theoretic and algorithmic challenge still remains, i. e. , how to optimize the design of random Fourier features to achieve good kernel approximation on any input data using a low spectral sampling rate? In this paper, we propose to compute more adaptive random Fourier features with optimized spectral samples (wj’s) and feature weights (pj’s). The learning scheme not only significantly reduces the spectral sampling rate needed for accurate kernel approximation, but also allows joint optimization with any supervised learning framework. We establish generalization bounds using Rademacher complexity, and demonstrate advantages over previous methods. Moreover, our experiments show that the empirical kernel approximation provides effective regularization for supervised learning.

PDF Details

TCS Journal 2019 Journal Article

Public key encryption with equality test via hash proof system

Ming Zeng
Jie Chen
Kai Zhang
Haifeng Qian

Public key encryption with equality test (PKEET) allows a tester to know whether ciphertexts are the encryptions of a same message or not by using the trapdoors issued from their owners, which is a useful cryptographic primitive can be deployed in many applications, such as in the mechanism of searching over encrypted data. Based on Hash Proof System (HPS) introduced by Cramer and Shoup, this paper presents an oversimplified paradigm for constructing PKEET in the standard model. Compared with the previous works that use identity-based encryption, strongly unforgeable one-time signature or other strong cryptographic primitives, our paradigm requires only the universal2 property of HPS and provides an efficient way to obtain concrete PKEET schemes based on different assumptions in the standard model, since HPS has been shown can be easily realized from a board range of NP languages (e. g. , DLIN-based, DCR-based, Lattice-based and so on). Moreover, to demonstrate the practicality of the proposed paradigm, we instantiate it based on two kinds of NP languages respectively, one is based on the decisional Diffie-Hellman (DDH) assumption, the other one is based on the decisional composite residuosity (DCR) assumption, which results in the first concrete PKEET schemes that in the standard model without using pairing operations, and the schemes' security are also based on the standard DDH assumption and the standard DCR assumption respectively.

Details DOI

YNIMG Journal 2018 Journal Article

Neural and genetic determinants of creativity

ZhaoWen Liu
Jie Zhang
Xiaohua Xie
Edmund T. Rolls
Jiangzhou Sun
Kai Zhang
Zeyu Jiao
Qunlin Chen

Creative thinking plays a vital role in almost all aspects of human life. However, little is known about the neural and genetic mechanisms underlying creative thinking. Based on a cross-validation based predictive framework, we searched from the whole-brain connectome (34, 716 functional connectivities) and whole genome data (309, 996 SNPs) in two datasets (all collected by Southwest University, Chongqing) consisting of altogether 236 subjects, for a better understanding of the brain and genetic underpinning of creativity. Using the Torrance Tests of Creative Thinking score, we found that high figural creativity is mainly related to high functional connectivity between the executive control, attention, and memory retrieval networks (strong top-down effects); and to low functional connectivity between the default mode network, the ventral attention network, and the subcortical and primary sensory networks (weak bottom-up processing) in the first dataset (consisting of 138 subjects). High creativity also correlates significantly with mutations of genes coding for both excitatory and inhibitory neurotransmitters. Combining the brain connectome and the genomic data we can predict individuals' creativity scores with an accuracy of 78. 4%, which is significantly better than prediction using single modality data (gene or functional connectivity), indicating the importance of combining multi-modality data. Our neuroimaging prediction model built upon the first dataset was cross-validated by a completely new dataset of 98 subjects (r = 0. 267, p = 0. 0078) with an accuracy of 64. 6%. In addition, the creativity–related functional connectivity network we identified in the first dataset was still significantly correlated with the creativity score in the new dataset (p< 10 − 3 ). In summary, our research demonstrates that strong top-down control versus weak bottom-up processes underlie creativity, which is modulated by competition between the glutamate and GABA neurotransmitter systems. Our work provides the first insights into both the neural and the genetic bases of creativity.

Details DOI

AAAI Conference 2018 Conference Paper

Substructure Assembling Network for Graph Classification

Xiaohan Zhao
Bo Zong
Ziyu Guan
Kai Zhang
Wei Zhao

Graphs are natural data structures adopted to represent realworld data of complex relationships. In recent years, a surge of interest has been received to build predictive models over graphs, with prominent examples in chemistry, computational biology, and social networks. The overwhelming complexity of graph space often makes it challenging to extract interpretable and discriminative structural features for classiﬁcation tasks. In this work, we propose a novel neural network structure called Substructure Assembling Network (SAN) to extract graph features and improve the generalization performance of graph classiﬁcation. The key innovation of our work is a uniﬁed substructure assembling unit, which is a variant of Recurrent Neural Network (RNN) designed to hierarchically assemble useful pieces of graph components so as to fabricate discriminative substructures. SAN adopts a sequential, probabilistic decision process, and therefore it can tune substructure features in a ﬁner granularity. Meanwhile, the parameterized soft decisions can be continuously improved with supervised learning through back-propagation, leading to optimizable search trajectories. Overall, SAN embraces both the ﬂexibility of combinatorial pattern search and the strong optimizability of deep learning, and delivers promising results as well as interpretable structural features in graph classiﬁcation against state-of-the-art techniques.

PDF Details

IROS Conference 2017 Conference Paper

A regularized on-line sequential extreme learning machine with forgetting property for fast dynamic hysteresis modeling

Zelong Wu
Hui Tang 0003
Sifeng He
Jian Gao 0002
Xin Chen 0005
Chengqiang Cui
Yunbo He
Kai Zhang

Piezoelectric ceramics(PZT)actuator has been widely used in flexure-guided nanopositioning stage because of their high resolution. However, it is quite hard to achieve high-rate precision positioning control because of the complex hysteresis nonlinearity effect of PZT actuator. Thus, an online RELM algorithm with forgetting property(FReOS-ELM) is proposed to handle this issue. Firstly, we adopt regularized extreme learning machine(RELM)to build an intelligent hysteresis model. The training of the algorithm is completed only in one step, which avoids the shortcomings of the traditional hysteresis model based on artificial neural network(ANN) that slow training speed and easy to fall into the local minimum. Then, based on the regularized on-line sequential extreme learning machine(ReOS-ELM), an on-line RELM algorithm with forgetting property(FReOS-ELM) is designed, which can avoid the computational load of ReOS-ELM in the process of adding new data for learning on-line. In the experiment, a real-time voltage signal with varying frequencies and amplitudes is adopted, and the output displacement data of the nanopositioning stage is also acquired and analyzed. The results powerfully verify that the performance of the established hysteresis model based on the proposed FReOS-ELM is satisfactory, which can be used to improve the practical positioning performance for flexure nanopositioning stage.

Details

YNIMG Journal 2017 Journal Article

JIVE integration of imaging and behavioral data

Qunqun Yu
Benjamin B. Risk
Kai Zhang
J.S. Marron

A major goal in neuroscience is to understand the neural pathways underlying human behavior. We introduce the recently developed Joint and Individual Variation Explained (JIVE) method to the neuroscience community to simultaneously analyze imaging and behavioral data from the Human Connectome Project. Motivated by recent computational and theoretical improvements in the JIVE approach, we simultaneously explore the joint and individual variation between and within imaging and behavioral data. In particular, we demonstrate that JIVE is an effective and efficient approach for integrating task fMRI and behavioral variables using three examples: one example where task variation is strong, one where task variation is weak and a reference case where the behavior is not directly related to the image. These examples are provided to visualize the different levels of signal found in the joint variation including working memory regions in the image data and accuracy and response time from the in-task behavioral variables. Joint analysis provides insights not available from conventional single block decomposition methods such as Singular Value Decomposition. Additionally, the joint variation estimated by JIVE appears to more clearly identify the working memory regions than Partial Least Squares (PLS), while Canonical Correlation Analysis (CCA) gives grossly overfit results. The individual variation in JIVE captures the behavior unrelated signals such as a background activation that is spatially homogeneous and activation in the default mode network. The information revealed by this individual variation is not examined in traditional methods such as CCA and PLS. We suggest that JIVE can be used as an alternative to PLS and CCA to improve estimation of the signal common to two or more datasets and reveal novel insights into the signal unique to each dataset.

Details DOI

AIJ Journal 2017 Journal Article

Low-rank decomposition meets kernel learning: A generalized Nyström method

Liang Lan
Kai Zhang
Hancheng Ge
Wei Cheng
Jun Liu
Andreas Rauber
Xiao-li Li
Jun Wang

Low-rank matrix decomposition and kernel learning are two useful techniques in building advanced learning systems. Low-rank decomposition can greatly reduce the computational cost of manipulating large kernel matrices. However, existing approaches are mostly unsupervised and do not incorporate side information such as class labels, making the decomposition less effective for a specific learning task. On the other hand, kernel learning techniques aim at constructing kernel matrices whose structure is well aligned with the learning target, which improves the generalization performance of kernel methods. However, most kernel learning approaches are computationally very expensive. To obtain the advantages of both techniques and address their limitations, in this paper we propose a novel kernel low-rank decomposition formulation called the generalized Nyström method. Our approach inherits the linear time and space complexity via matrix decomposition, while at the same time fully exploits (partial) label information in computing task-dependent decomposition. In addition, the resultant low-rank factors can generalize to arbitrary new samples, rendering great flexibility in inductive learning scenarios. We further extend the algorithm to a multiple kernel learning setup. The experimental results on semi-supervised classification demonstrate the usefulness of the proposed method.

Details DOI

NeurIPS Conference 2016 Conference Paper

Distributed Flexible Nonlinear Tensor Factorization

Shandian Zhe
Kai Zhang
Pengyuan Wang
Kuang-chih Lee
Zenglin Xu
Yuan Qi
Zoubin Ghahramani

Tensor factorization is a powerful tool to analyse multi-way data. Recently proposed nonlinear factorization methods, although capable of capturing complex relationships, are computationally quite expensive and may suffer a severe learning bias in case of extreme data sparsity. Therefore, we propose a distributed, flexible nonlinear tensor factorization model, which avoids the expensive computations and structural restrictions of the Kronecker-product in the existing TGP formulations, allowing an arbitrary subset of tensor entries to be selected for training. Meanwhile, we derive a tractable and tight variational evidence lower bound (ELBO) that enables highly decoupled, parallel computations and high-quality inference. Based on the new bound, we develop a distributed, key-value-free inference algorithm in the MapReduce framework, which can fully exploit the memory cache mechanism in fast MapReduce systems such as Spark. Experiments demonstrate the advantages of our method over several state-of-the-art approaches, in terms of both predictive performance and computational efficiency.

PDF Details

IJCAI Conference 2016 Conference Paper

Entity Embedding-Based Anomaly Detection for Heterogeneous Categorical Events

Ting Chen
Lu-An Tang
Yizhou Sun
Zhengzhang Chen
Kai Zhang

Anomaly detection plays an important role in modern data-driven security applications, such as detecting suspicious access to a socket from a process. In many cases, such events can be described as a collection of categorical values that are considered as entities of different types, which we call heterogeneous categorical events. Due to the lack of intrinsic distance measures among entities, and the exponentially large event space, most existing work relies heavily on heuristics to calculate abnormal scores for events. Different from previous work, we propose a principled and unified probabilistic model APE (Anomaly detection via Probabilistic pairwise interaction and Entity embedding) that directly models the likelihood of events. In this model, we embed entities into a common latent space using their observed co-occurrence in different events. More specifically, we first model the compatibility of each pair of entities according to their embeddings. Then we utilize the weighted pairwise interactions of different entity types to define the event probability. Using Noise-Contrastive Estimation with "context-dependent" noise distribution, our model can be learned efficiently regardless of the large event space. Experimental results on real enterprise surveillance data show that our methods can accurately detect abnormal events compared to other state-of-the-art abnormal detection techniques.

PDF Details

EAAI Journal 2014 Journal Article

A hybrid approach combining extreme learning machine and sparse representation for image classification

Minxia Luo
Kai Zhang

Two well-known techniques, extreme learning machine (ELM) and sparse representation based classification (SRC) method, have attracted significant attention due to their respective performance characteristics in computer vision and pattern recognition. In general, ELM has speed advantage and SRC has accuracy advantage. However, there also remain drawbacks that limit their practical application. Actually, in the field of image classification, ELM performs extremely fast while it cannot handle noise well, whereas SRC shows notable robustness to noise while it suffers high computational cost. In order to incorporate their respective advantages and also overcome their respective drawbacks, this work proposes a novel hybrid approach combining ELM and SRC for image classification. The new approach is applied to handwritten digit classification and face recognition, experiments results demonstrate that it not only outperforms ELM in classification accuracy but also has much less computational complexity than SRC.

Details DOI

AAAI Conference 2014 Conference Paper

Improving Semi-Supervised Target Alignment via Label-Aware Base Kernels

Qiaojun Wang
Kai Zhang
Guofei Jiang
Ivan Maric

Semi-supervised kernel design is an essential step for obtaining good predictive performance in semi-supervised learning tasks. In the current literatures, a large family of algorithms builds the new kernel by using the weighted average of predefined base kernels. While optimal weighting schemes have been studied extensively, the choice of base kernels received much less attention. Many methods simply adopt the empirical kernel matrices or its eigenvectors. Such base kernels are computed irrespective of class labels and may not always reflect useful structures in the data. As a result, in case of poor base kernels, the generalization performance can be degraded however hard their weights are tuned. In this paper, we propose to construct high-quality base kernels with the help of label information to globally improve the final target alignment. In particular, we devise label-aware kernel eigenvectors under the framework of semi-supervised eigenfunction extrapolation, which span base kernels that are more useful for learning. Such base kernels are individually better aligned to the learning target, so their mixture will more likely generate a good classifier. Our approach is computationally efficient, and demonstrates encouraging performance in semisupervised classification and regression.

PDF Details

YNIMG Journal 2012 Journal Article

Alteration of brain default network in subacute phase of injury in concussed individuals: Resting-state fMRI study

Brian Johnson
Kai Zhang
Michael Gay
Silvina Horovitz
Mark Hallett
Wayne Sebastianelli
Semyon Slobounov

There are a number of symptoms, both neurological and behavioral, associated with a single episode of r mild traumatic brain injury (mTBI). Neuropsychological testing and conventional neuroimaging techniques are not sufficiently sensitive to detect these changes, which adds to the complexity and difficulty in relating symptoms from mTBI to their underlying structural or functional deficits. With the inability of traditional brain imaging techniques to properly assess the severity of brain damage induced by mTBI, there is hope that more advanced neuroimaging applications will be more sensitive, as well as specific, in accurately assessing mTBI. In this study, we used resting state functional magnetic resonance imaging to evaluate the default mode network (DMN) in the subacute phase of mTBI. Fourteen concussed student-athletes who were asymptomatic based upon clinical symptoms resolution and clearance for aerobic exercise by medical professionals were scanned using resting state functional magnetic resonance imaging. Nine additional asymptomatic yet not medically cleared athletes were recruited to investigate the effect of a single episode of mTBI versus multiple mTBIs on the resting state DMN. In concussed individuals the resting state DMN showed a reduced number of connections and strength of connections in the posterior cingulate and lateral parietal cortices. An increased number of connections and strength of connections was seen in the medial prefrontal cortex. Connections between the left dorso-lateral prefrontal cortex and left lateral parietal cortex showed a significant reduction in magnitude as the number of concussions increased. Regression analysis also indicated an overall loss of connectivity as the number of mTBI episodes increased. Our findings indicate that alterations in the brain resting state default mode network in the subacute phase of injury may be of use clinically in assessing the severity of mTBI and offering some insight into the pathophysiology of the disorder.

Details DOI

ICRA Conference 2012 Conference Paper

An efficient stochastic clustering auction for heterogeneous robot teams

Kai Zhang
Emmanuel G. Collins Jr.
Adrian Barbu

Stochastic Clustering Auctions (SCAs) constitute a class of cooperative auction methods that enable improvement of the global cost of the task allocations obtained with fast greedy algorithms. Prior research had developed Contracts Sequencing Algorithms (CSAs) that are deterministic and enable transfers, swaps, and other types of contracts between team members. In contrast to CSAs, SCAs use stochastic transfers or swaps between the task clusters assigned to each team member and have algorithm parameters that can enable tradeoffs between optimality and computational and communication requirements. The first SCA was based on a “Gibbs Sampler” and constrained the stochastic cluster reallocations to simple single transfers or swaps; it is applicable to heterogeneous teams. Subsequently, a more efficient SCA was developed, based on the generalized Swendsen-Wang method; it achieves the increased efficiency by connecting tasks that appear to be synergistic and then stochastically reassigning these connected tasks, hence enabling more complex and efficient movements between clusters than the first SCA. However, its application was limited to homogeneous teams. The contribution of this work is to present an efficient SCA for heterogeneous teams; it is based on a modified Swendsen-Wang method. For centralized auctioning and homogeneous teams, extensive numerical experiments were used to provide a comparison in terms of costs and computational and communication requirements of the three SCAs and a baseline CSA. It was seen that the new SCA maintains the efficiency of the second SCA and can yield similar performance to the baseline CSA in far fewer iterations. The same metrics were used to evaluate the performance of the new SCA for heterogeneous teams. A distributed version of the new SCA was also evaluated in numerical experiments. The results show that, as expected, the distributed SCA continually improves the global performance with each iteration, but converges to a lower cost solution than the centralized SCA.

Details

TAAS Journal 2012 Journal Article

Centralized and distributed task allocation in multi-robot teams via a stochastic clustering auction

Kai Zhang
Emmanuel G. Collins
Dongqing Shi

This article considers the problem of optimal task allocation for heterogeneous teams, for example, teams of heterogeneous robots or human-robot teams. It is well-known that this problem is NP-hard and hence computationally feasible approaches must develop an approximate solution. Here, we propose a solution via a Stochastic Clustering Auction (SCA) that uses a Markov chain search process along with simulated annealing. This is the first stochastic auction method used in conjunction with global optimization. It is based on stochastic transfer and swap moves between the clusters of tasks assigned to the various robots and considers not only downhill movements, but also uphill movements, which can avoid local minima. A novel feature of this algorithm is that, by tuning the annealing suite and turning the uphill movements on and off, the global team performance after algorithm convergence can slide in the region between the global optimal performance and the performance associated with a random allocation. Extensive numerical experiments are used to evaluate the performance of SCA in terms of costs and computational and communication requirements. For centralized auctioning, the SCA algorithm is compared to fast greedy auction algorithms. Distributed auctioning is then compared with centralized SCA.

Details DOI

IROS Conference 2010 Conference Paper

A novel Stochastic Clustering Auction for task allocation in multi-robot teams

Kai Zhang
Emmanuel G. Collins Jr.
Adrian Barbu

It will be shown that the global cost of the task allocations obtained with fast greedy algorithms can be improved upon by using a class of auction methods called Stochastic Clustering Auctions (SCAs). SCAs use stochastic transfers or swaps between the task clusters assigned to each team member, allow both uphill and downhill cost movements, and rely on simulated annealing. The choice of a key annealing parameter and turning the uphill movements on and off enables the converged solution of a SCA to slide in the region between the global optimal performance and the performance associated with a random allocation. The first SCA, called GSSCA, was based on a Gibbs Sampler, which constrained the stochastic cluster reallocations to simple single transfers or swaps. This paper presents a new and more efficient SCA, called SWSCA, based on the generalized Swendsen-Wang method that enables more complex and efficient movements between clusters by connecting tasks that appear to be synergistic and then stochastically reassigning these connected tasks. For centralized auctioning, extensive numerical experiments are used to compare the performance of SWSCA with GSSCA in terms of costs and computational and communication requirements. Distributed SWSCA is then compared with centralized SWSCA using communication links between robots that were motivated by a generic topology called a “scale free network”.

Details

TIME Conference 2006 Conference Paper

Efficient Heuristics for Solving Probabilistic Interval Algebra Networks

Kai Zhang
André Trudel

A probabilistic interval algebra (PIA) network is an interval algebra network with probabilities associated with the labels on an edge. The probabilities on each edge sum to 1. A solution is a consistent scenario where the product of the probabilities associated with each unique edge label is maximized. In this paper we investigate previous PIA network solution algorithms, and propose new ones. Our first algorithm is based on best first search and guarantees to output the optimal solution. However, this algorithm is only feasible for toy problems. We augment the algorithm with three heuristics. Although our proposed algorithm does not guarantee an optimal solution, it is very useful in practice. Good solutions can be generated quickly

Details

NeurIPS Conference 2006 Conference Paper

Simplifying Mixture Models through Function Approximation

Kai Zhang
James Kwok

Finite mixture model is a powerful tool in many statistical learning problems. In this paper, we propose a general, structure-preserving approach to reduce its model complexity, which can bring signiﬁcant computational beneﬁts in many applications. The basic idea is to group the original mixture components into compact clusters, and then minimize an upper bound on the approximation error between the original and simpliﬁed models. By adopting the L2 norm as the dis- tance measure between mixture models, we can derive closed-form solutions that are more robust and reliable than using the KL-based distance measure. Moreover, the complexity of our algorithm is only linear in the sample size and dimensional- ity. Experiments on density estimation and clustering-based image segmentation demonstrate its outstanding performance in terms of both speed and accuracy.

PDF Details