Arrow Research search

Author name cluster

Rui Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

131 papers
2 author rows

Possible papers

131

EAAI Journal 2026 Journal Article

Acoustic based leak location detection for water supply pipelines in urban areas via multi-task deep learning

  • Rui Zhang
  • Ali Fares
  • Ibrahim A. Tijani
  • Tarek Zayed
  • Zeren Jin
  • Abdul-Mugis Yussif

Expeditious and precise localization of leaks holds paramount importance for water utility providers to ensure the timely rectification of damaged pipes. This study introduces an innovative approach to address challenges for leak detection and localization, capitalizing on noise loggers to capture acoustic emissions from real field and forming the bedrock of the database for deep learning. First, the discrete wavelet transform-based denoising technique is applied to acoustic signals captured by noise loggers, mitigating susceptibility to noise interference. Second, a novel multi-task deep learning framework is devised to enhance the efficacy and accuracy of leak detection and localization, incorporating a variational autoencoder to obtain latent representations housing essential yet compact information. Finally, to enhance generalization in data-scarce scenarios, transfer learning is invoked to capitalize on the acquired latent representations to ensure optimal performance. Upon evaluation against an independent test, best leak detection accuracies of 100 % and 98. 5 % are achieved for non-metallic and metallic water supply pipelines, respectively, with corresponding leak localization errors (MAE, Mean Absolute Error) being 0. 273 m and 0. 096 m.

Details DOI

EAAI Journal 2026 Journal Article

An electroencephalogram signal analysis method based on dual self-supervised graph diffusion recurrent network

  • Sunan Ge
  • Shuang Wang
  • Rui Zhang
  • Xueqing Zhao
  • Xinshi
  • Meng Wang
  • Tao Wu

Diagnosis of neurological diseases and emotion recognition analyzing based on electroencephalogram (EEG) signals have been widely applied in numerous fields by revealing the complex operational mechanisms of the human brain. However, existing EEG signal analysis methods are hindered by label noise and the scale of labeled data samples, making it difficult to effectively learn the distribution characteristics of the data and identify the heterogeneity of EEG signals. Therefore, this paper proposes a dual self-supervised graph diffusion recurrent network (DSGDRN) method for representation learning of unlabeled EEG signals, reducing biases and noise effects caused by manual annotation and improving the ability to recognize individual differences. First, to capture the natural geometric features of EEG signals and the dynamic connection information within the brain, distance graph structures and correlation graph structures are respectively used for feature expression. A dual self-supervised algorithm is employed to represent hidden states as a learnable function, enhancing the expressive power of the graph recurrent diffusion network and its ability to recognize contextual information. Finally, during the testing process, a dual learning strategy with continuous adaptive adjustment of hidden state parameters is adopted to improve the application capability of EEG signals in real-world scenarios. Experimental results demonstrate that compared with existing methods, the proposed method exhibits superior performance in neurological disease diagnosis and emotion detection, indicating its effective representation learning capabilities in fields such as neurological disease analysis and emotion recognition.

Details DOI

AAAI Conference 2026 Conference Paper

ConfGuard: A Simple and Effective Backdoor Detection for Large Language Models

  • Zihan Wang
  • Rui Zhang
  • Hongwei Li
  • Wenshu Fan
  • Wenbo Jiang
  • Qingchuan Zhao
  • Guowen Xu

Backdoor attacks pose a significant threat to Large Language Models (LLMs), where adversaries can embed hidden triggers to manipulate LLM's outputs. Most existing defense methods, primarily designed for classification tasks, are ineffective against the autoregressive nature and vast output space of LLMs, thereby suffering from poor performance and high latency. To address these limitations, we investigate the behavioral discrepancies between benign and backdoored LLMs in output space. We identify a critical phenomenon which we term sequence lock: a backdoored model generates the target sequence with abnormally high and consistent confidence compared to benign generation. Building on this insight, we propose ConfGuard, a lightweight and effective detection method that monitors a sliding window of token confidences to identify sequence lock. Extensive experiments demonstrate ConfGuard achieves a near 100% true positive rate (TPR) and a negligible false positive rate (FPR) in the vast majority of cases. Crucially, the ConfGuard enables real-time detection almost without additional latency, making it a practical backdoor defense for real-world LLM deployments.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Data-Centric Sequential Recommendation with Relation-Augmented Generation

  • Yichen Li
  • Yichen Tan
  • Yijing Shan
  • Haozhao Wang
  • Rui Zhang
  • Imran Razzak
  • Ruixuan Li

Data-Centric Sequential Recommendation (DaCSR) has emerged as a promising technique that enhances dataset quality to better capture user preferences without increasing training complexity. However, mining item relations to improve data quality remains challenging due to the intricate nature of interaction sequences. Existing methods predominantly either: 1) optimize models to learn such item relations from fixed datasets at significant training cost, or 2) employ generative models to adaptively learn only interaction patterns, which lack interpretability and cannot guarantee effective data quality enhancement. In this paper, we pioneer a relation-guided dataset augmentation and regeneration framework for sequential recommendation called \textbf{RaSR}. This framework can significantly improve model performance on original datasets while maintaining training efficiency without modifying the model architecture. Specifically, we first preprocess user interactions to construct standardized sequential data and extract semantic representations via a Large Language Model (LLM). We then build a multi-relation graph with manually predefined metrics and semantic representations to generate augmented datasets. Finally, a relation-aware generator can produce regenerated datasets with both the multi-relation graph and the augmented dataset. To verify the effectiveness of RaSR, we conduct experiments on various backbone models and datasets, and achieve significant performance improvement compared to training the model only on the original dataset.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Efficient Diffusion Planning with Temporal Diffusion

  • Jiaming Guo
  • Rui Zhang
  • Zerun Li
  • Yunkai Gao
  • Shaohui Peng
  • Siming Lan
  • Xing Hu
  • Zidong Du

Diffusion planning is a promising method for learning high-performance policies from offline data. To avoid the impact of discrepancies between planning and reality on performance, previous works generate new plans at each time step. However, this incurs significant computational overhead and leads to lower decision frequencies, and frequent plan switching may also affect performance. In contrast, humans might create detailed short-term plans and more general, sometimes vague, long-term plans, and adjust them over time. Inspired by this, we propose the Temporal Diffusion Planner (TDP) which improves decision efficiency by distributing the denoising steps across the time dimension. TDP begins by generating an initial plan that becomes progressively more vague over time. At each subsequent time step, rather than generating an entirely new plan, TDP updates the previous one with a small number of denoising steps. This reduces the average number of denoising steps, improving decision efficiency. Additionally, we introduce an automated replanning mechanism to prevent significant deviations between the plan and reality. Experiments on D4RL show that, compared to previous works that generate new plans every time step, TDP significantly improves the decision-making frequency by 11-24.8 times while achieving higher or comparable performance.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Flexible Concept Bottleneck Model

  • Xingbo Du
  • Qiantong Dou
  • Lei Fan
  • Rui Zhang

Concept bottleneck models (CBMs) improve neural network interpretability by introducing an intermediate layer that maps human-understandable concepts to predictions. Recent work has explored the use of vision-language models (VLMs) to automate concept selection and annotation. However, existing VLM-based CBMs typically require full model retraining when new concepts are involved, which limits their adaptability and flexibility in real-world scenarios, especially considering the rapid evolution of vision-language foundation models. To address these issues, we propose Flexible Concept Bottleneck Model (FCBM), which supports dynamic concept adaptation, including complete replacement of the original concept set. Specifically, we design a hypernetwork that generates prediction weights based on concept embeddings, allowing seamless integration of new concepts without retraining the entire model. In addition, we introduce a modified sparsemax module with a learnable temperature parameter that dynamically selects the most relevant concepts, enabling the model to focus on the most informative features. Extensive experiments on five public benchmarks demonstrate that our method achieves accuracy comparable to state-of-the-art baselines with a similar number of effective concepts. Moreover, the model generalizes well to unseen concepts with just a single epoch of fine-tuning, demonstrating its strong adaptability and flexibility.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Learning to Optimize Job Shop Scheduling Under Structural Uncertainty

  • Rui Zhang
  • Jianwei Niu
  • Xuefeng Liu
  • Shaojie Tang
  • Jing Yuan

The Job-Shop Scheduling Problem (JSSP), under various forms of manufacturing uncertainty, has recently attracted considerable research attention. Most existing studies focus on parameter uncertainty, such as variable processing times, and typically adopt the actor-critic framework. In this paper, we explore a different but prevalent form of uncertainty in JSSP: structural uncertainty. Structural uncertainty arises when a job may follow one of several routing paths, and the selection is determined not by policy, but by situational factors (e.g., the quality of intermediate products) that cannot be known in advance. Existing methods struggle to address this challenge due to incorrect credit assignment: a high-quality action may be unfairly penalized if it is followed by a time-consuming path. To address this problem, we propose a novel method named UP-AAC. In contrast to conventional actor-critic methods, UP-AAC employs an asymmetric architecture. While its actor receives a standard stochastic state, the critic is crucially provided with a deterministic state reconstructed in hindsight. This design allows the critic to learn a more accurate value function, which in turn provides a lower-variance policy gradient to the actor, leading to more stable learning. In addition, we design an attention-based Uncertainty Perception Model (UPM) to enhance the actor's scheduling decisions. Extensive experiments demonstrate that our method outperforms existing approaches in reducing makespan on benchmark instances.

PDF Details DOI

AAAI Conference 2026 Conference Paper

MPMA: Preference Manipulation Attack Against Model Context Protocol

  • Zihan Wang
  • Rui Zhang
  • Yu Liu
  • Wenshu Fan
  • Wenbo Jiang
  • Qingchuan Zhao
  • Hongwei Li
  • Guowen Xu

Model Context Protocol (MCP) standardizes interface mapping for large language models (LLMs) to access external data and tools, which revolutionizes the paradigm of tool selection and facilitates the rapid expansion of the LLM agent tool ecosystem. However, as the MCP is increasingly adopted, third-party customized versions of the MCP server expose potential security vulnerabilities. In this paper, we first introduce a novel security threat, which we term the MCP Preference Manipulation Attack (MPMA). An attacker deploys a customized MCP server to manipulate LLMs, causing them to prioritize it over other competing MCP servers. This can result in economic benefits for attackers, such as revenue from paid MCP services or advertising income generated from free servers. To achieve MPMA, we first design a Direct Preference Manipulation Attack (DPMA) that achieves significant effectiveness by inserting the manipulative word and phrases into the tool name and description. However, such a direct modification is obvious to users and lacks stealthiness. To address these limitations, we further propose Genetic-based Advertising Preference Manipulation Attack (GAPMA). GAPMA employs four commonly used strategies to initialize descriptions and integrates a Genetic Algorithm (GA) to enhance stealthiness. The experiment results demonstrate that GAPMA balances high effectiveness and stealthiness. Our study reveals a critical vulnerability of the MCP in open ecosystems, highlighting an urgent need for robust defense mechanisms to ensure the fairness of the MCP ecosystem.

PDF Details DOI

AAAI Conference 2026 Conference Paper

PIMRL: Physics-Informed Multi-Scale Recurrent Learning for Burst-Sampled Spatiotemporal Dynamics

  • Han Wan
  • Qi Wang
  • Yuan Mi
  • Rui Zhang
  • Hao Sun

Deep learning has shown strong potential in modeling complex spatiotemporal dynamics. However, most existing methods depend on densely and uniformly sampled data, which is often unavailable in practice due to sensor and cost limitations. In many real-world settings, such as mobile sensing and physical experiments, data are burst-sampled with short high-frequency segments followed by long gaps, making it difficult to learn accurate dynamics from sparse observations. To address this issue, we propose Physics-Informed Multi-Scale Recurrent Learning (PIMRL), a novel framework specifically designed for burst-sampled spatiotemporal data. PIMRL combines macro-scale latent dynamics inference with micro-scale adaptive refinement guided by incomplete prior information from partial differential equations (PDEs). It further introduces a temporal message-passing mechanism to effectively propagate information across burst intervals. This multi-scale architecture enables PIMRL to model complex systems accurately even under severe data scarcity. We evaluate our approach on five benchmark datasets involving 1D to 3D multi-scale PDEs. The results show that PIMRL consistently outperforms state-of-the-art baselines, achieving substantial improvements and reducing errors by up to 80\% in the most challenging settings, which demonstrates the clear advantage of our model. Our work demonstrates the effectiveness of physics-informed recurrent learning for accurate and efficient modeling of sparse spatiotemporal systems.

PDF Details DOI

AAAI Conference 2026 Conference Paper

PPFL: A Parameter Behavior-Driven Plug-in Personalization Engine for Federated Learning

  • Qianyue Cao
  • Zongwei Zhu
  • Zirui Lian
  • Rui Zhang
  • Boyu Li
  • Yi Xiong
  • Xuehai Zhou

Personalized Federated Learning (PFL) customizes models for each client to mitigate challenges from non-IID data, wherein a dominant strategy is model decoupling that partitions models into shared and personalized parts based on architectural priors (e.g., backbone vs. head). However, we reveal a critical flaw in this strategy: it induces "intrinsic drift," a performance degradation often more severe than the well-known client drift, which limits final accuracy. We trace this drift to a steep cliff of high loss emerging from the naive stitching of shared and personalized parts. To address this, we shift from architectural partitioning to a parameter behavior-driven paradigm. We introduce PPFL, an approach that employs a novel soft-fusion strategy guided by parameter-wise behavioral perception. PPFL dynamically infers each parameter's functional role—whether it behaves more like a 'personalist' or a 'generalist' in the current context—by synthesizing its multifaceted behavior observed during local training. Extensive experiments on image, text, and multimodal classification benchmarks show that PPFL outperforms eight state-of-the-art baselines by up to 5.3%. Moreover, it can function as a plug-in module, boosting the accuracy of vanilla FedAvg with a 16.82% absolute gain.

PDF Details DOI

AAAI Conference 2026 Conference Paper

QiMeng-CRUX: Narrowing the Gap Between Natural Language and Verilog via Core Refined Understanding eXpression

  • Lei Huang
  • Rui Zhang
  • Jiaming Guo
  • Yang Zhang
  • Di Huang
  • Shuyao Cheng
  • Pengwei Jin
  • Chongxiao Li

Large language models (LLMs) have shown promising capabilities in hardware description language (HDL) generation. However, existing approaches often rely on free-form natural language descriptions that are often ambiguous, redundant, and unstructured, which poses significant challenges for downstream Verilog code generation. We treat hardware code generation as a complex transformation from an open-ended natural language space to a domain-specific, highly constrained target space. To bridge this gap, we introduce Core Refined Understanding eXpression (CRUX), a structured intermediate space that captures the essential semantics of user intent while organizing the expression for precise Verilog code generation. We further design a two-stage training framework, comprising Joint Expression Modeling and Dual-Space Optimization, to enhance the quality of both CRUX and Verilog code. Experiments across multiple Verilog generation benchmarks demonstrate that our model, QiMeng-CRUX, achieves state-of-the-art performance among general models, particularly under challenging design tasks. Furthermore, the CRUX space proves transferable and beneficial when used as input prompts for other code models, highlighting its effectiveness in narrowing the gap between free-form natural language descriptions and precise Verilog generation.

PDF Details DOI

EAAI Journal 2026 Journal Article

Research on personalized federated learning algorithm based on Kolmogorov–Smirnov test clustering

  • Rui Zhang
  • Qingao Liu
  • Siyan Yang
  • Yan Wang
  • Guodong You

Federated learning enables multiple clients to perform federated modeling to collaboratively solve machine learning tasks while protecting data privacy. However, studies have shown that when multiple clients have different data feature distributions, the performance of the obtained global model degrades and lacks generalization ability. To address this problem, this paper proposes a personalized federated learning algorithm based on Kolmogorov–Smirnov test fuzzy clustering. The algorithm adaptively clusters and tests clients with similar data feature distributions based on their local model parameter updates when learning the global model. Compared with existing methods, the proposed algorithm can dynamically estimate the number of clusters to which each client belongs and the fuzzy boundaries between clusters. We conducted experiments under conditions of independent and identically distributed, Dirichlet, and mixed triple distributions. The experiments show that compared with existing state-of-the-art clustering federated algorithms, the proposed algorithm demonstrates higher accuracy and stability in image classification tasks, and compensates for the drawbacks of the traditional federated learning algorithms, which are ineffective in directly aggregating data with differing feature distributions across clients, resulting in degraded performance.

Details DOI

AAAI Conference 2026 Conference Paper

Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation

  • Yu Zhong
  • Zihao Zhang
  • Rui Zhang
  • Lingdong Huang
  • Haihan Gao
  • Shuo Wang
  • Da Li
  • Ruijian Han

Vision-and-Language Navigation (VLN) requires an agent to dynamically explore complex 3D environments following human instructions. Recent research underscores the potential of harnessing large language models (LLMs) for VLN, given their commonsense knowledge and general reasoning capabilities. Despite their strengths, a substantial gap in task completion performance persists between LLM-based approaches and domain experts, as LLMs inherently struggle to comprehend real-world spatial correlations precisely; additionally, LLM inference can make the decision-making process considerably inefficient. To address these issues, we propose a novel dual-process thinking framework dubbed R3, integrating LLMs' generalization capabilities with VLN-specific expertise in a zero-shot manner. The framework comprises three core modules: Runner, Ruminator, and Regulator. The Runner is a lightweight transformer-based expert model that ensures efficient and accurate navigation under regular circumstances. The Ruminator employs a multimodal LLM as the backbone and adopts chain-of-thought (CoT) prompting to elicit structured reasoning from the LLM. The Regulator monitors the navigation progress and controls the appropriate thinking mode according to three criteria, integrating Runner and Ruminator harmoniously. Experimental results illustrate that R3 significantly outperforms other state-of-the-art methods, exceeding 3.28% and 3.30% in SPL and RGSPL respectively on the REVERIE benchmark, highlighting the effectiveness of our method in handling challenging VLN tasks.

PDF Details DOI

AAAI Conference 2026 Conference Paper

StepFun-Formalizer: Unlocking the Autoformalization Potential of LLMs Through Knowledge-Reasoning Fusion

  • Yutong Wu
  • Di Huang
  • Ruosi Wan
  • Yue Peng
  • Shijie Shang
  • Chenrui Cao
  • Lei Qi
  • Rui Zhang

Autoformalization aims to translate natural-language mathematical statements into a formal language. While LLMs have accelerated progress in this area, existing methods still suffer from low accuracy. We identify two key abilities for effective autoformalization: comprehensive mastery of formal-language domain knowledge, and reasoning capability of natural language problem understanding and informal-formal alignment. Without the former, a model cannot identify the correct formal objects; without the latter, it struggles to interpret real-world contexts and map them precisely into formal expressions. To address these gaps, we introduce ThinkingF, a data synthesis and training pipeline that improves both abilities. First, we construct two datasets: one by distilling and selecting large-scale examples rich in formal knowledge, and another by generating informal-to-formal reasoning trajectories guided by expert-designed templates. We then apply SFT and RLVR with these datasets to further fuse and refine the two abilities. The resulting 7B and 32B models exhibit both comprehensive formal knowledge and strong informal-to-formal reasoning. Notably, StepFun-Formalizer-32B achieves SOTA BEq@1 scores of 40.5% on FormalMATH-Lite and 26.7% on ProverBench, surpassing all prior general-purpose and specialized models.

PDF Details DOI

TMLR Journal 2026 Journal Article

Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models

  • Sarkar Snigdha Sarathi Das
  • Palash Goyal
  • Mihir Parmar
  • Yiwen Song
  • Long Le
  • Lesly Miculicich
  • Jinsung Yoon
  • Rui Zhang

Pre-trained Time Series Foundational Models (TSFMs) represent a significant advance, capable of forecasting diverse time series with complex characteristics, including varied seasonalities, trends, and long-range dependencies. Despite their primary goal of universal time series forecasting, their efficacy is far from uniform; divergent training protocols and data sources cause individual TSFMs to exhibit highly variable performance across different forecasting tasks, domains, and horizons. Leveraging this complementary expertise by arbitrating existing TSFM outputs presents a compelling strategy, yet this remains a largely unexplored area of research. In this paper, we conduct a thorough examination of how different TSFMs exhibit specialized performance profiles across various forecasting settings, and how we can effectively leverage this behavior in arbitration between different time series models. We specifically analyze how factors such as model selection and forecast horizon distribution can influence the efficacy of arbitration strategies. Based on this analysis, we propose Synapse, a novel arbitration framework for TSFMs. Synapse is designed to dynamically leverage a pool of TSFMs, assign and adjust predictive weights based on their relative, context-dependent performance, and construct a robust forecast distribution by adaptively sampling from the output quantiles of constituent models. Experimental results demonstrate that Synapse consistently outperforms other popular ensembling techniques as well as individual TSFMs, demonstrating Synapse's efficacy in time series forecasting.

PDF Details

AAAI Conference 2026 Conference Paper

Think-J: Learning to Think for Generative LLM-as-a-Judge

  • Hui Huang
  • Yancheng He
  • Hongli Zhou
  • Rui Zhang
  • Wei Liu
  • Weixun Wang
  • Jiaheng Liu
  • Wenbo Su

LLM-as-a-Judge refers to the automatic modeling of preferences for responses generated by Large Language Models (LLMs), which is of significant importance for both LLM evaluation and reward modeling. Although generative LLMs have made substantial progress in various tasks, their performance as LLM-Judge still falls short of expectations. In this work, we propose Think-J, which improves generative LLM-as-a-Judge by learning how to think. We first utilized a small amount of curated data to develop the model with initial judgment thinking capabilities. Subsequently, we optimize the judgment thinking traces based on reinforcement learning (RL). We propose two methods for judgment thinking optimization, based on offline and online RL, respectively. The offline method requires training a critic model to construct positive and negative examples for learning. The online method defines rule-based reward as feedback for optimization. Experimental results showed that our approach can significantly enhance the evaluation capability of generative LLM-Judge, surpassing both generative and classifier-based LLM-Judge without requiring extra human annotations.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Unbiased Rectification for Sequential Recommender Systems Under Fake Orders

  • Qiyu Qin
  • Yichen Li
  • Haozhao Wang
  • Cheng Wang
  • Rui Zhang
  • Ruixuan Li

Fake orders pose increasing threats to sequential recommender systems by misleading recommendation results through artificially manipulated interactions, including click farming, context-irrelevant substitutions, and sequential perturbations. Unlike injecting carefully designed fake users to influence recommendation performance, fake orders embedded within genuine user sequences aim to disrupt user preferences and mislead recommendation results, thereby manipulating exposure rates of specific items to gain competitive advantages. To protect users' authentic interest preferences and eliminate misleading information, this paper aims to perform precise and efficient rectification on compromised sequential recommender systems while avoiding the enormous computational and time costs of retraining existing models. Specifically, we identify that fake orders are not absolutely harmful—in certain cases, partial fake orders can even have a data augmentation effect. Based on this insight, we propose Dual-view Identification and Targeted Rectification (DITaR), which primarily identifies harmful samples to achieve unbiased rectification of the system. The core idea of this method is to obtain differentiated representations from collaborative and semantic views for precise detection, and then filters detected suspicious fake orders to select truly harmful ones for targeted rectification with gradient ascent. This ensures that useful information in fake orders is not removed while preventing bias residue. Moreover, it maintains the original data volume and sequence structure, thus protecting system performance and trustworthiness to achieve optimal unbiased rectification. Extensive experiments on three datasets demonstrate that DITaR achieves superior performance compared to state-of-the-art methods in terms of recommendation quality, computational efficiency, and system robustness.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Unsupervised Single-Channel Audio Separation with Diffusion Source Priors

  • Runwu Shi
  • Chang Li
  • Jiang Wang
  • Rui Zhang
  • Nabeela Khan
  • Benjamin Yen
  • Takeshi Ashizawa
  • Kazuhiro Nakadai

Single-channel audio separation aims to separate individual sources from a single-channel mixture. Most existing methods rely on supervised learning with synthetically generated paired data. However, obtaining high-quality paired data in real-world scenarios is often difficult. This data scarcity can degrade model performance under unseen conditions and limit generalization ability. To this end, in this work, we approach this problem from an unsupervised perspective, framing it as a probabilistic inverse problem. Our method requires only diffusion priors trained on individual sources. Separation is then achieved by iteratively guiding an initial state toward the solution through reconstruction guidance. Importantly, we introduce an advanced inverse problem solver specifically designed for separation, which mitigates gradient conflicts caused by interference between the diffusion prior and reconstruction guidance during inverse denoising. This design ensures high-quality and balanced separation performance across individual sources. Additionally, we find that initializing the denoising process with an augmented mixture instead of pure Gaussian noise provides an informative starting point that significantly improves the final performance. To further enhance audio prior modeling, we design a novel time–frequency attention-based network architecture that demonstrates strong audio modeling capability. Collectively, these improvements lead to significant performance gains, as validated across speech–sound event, sound event, and speech separation tasks.

PDF Details DOI

AAAI Conference 2025 Conference Paper

3DMambaIPF: A State Space Model for Iterative Point Cloud Filtering via Differentiable Rendering

  • Qingyuan Zhou
  • Weidong Yang
  • Ben Fei
  • Jingyi Xu
  • Rui Zhang
  • Keyi Liu
  • Yeqi Luo
  • Ying He

Noise is an inevitable aspect of point cloud acquisition, necessitating filtering as a fundamental task within the realm of 3D vision. Existing learning-based filtering methods have shown promising capabilities on commonly used datasets. Nonetheless, the effectiveness of these methods is constrained when dealing with a substantial quantity of point clouds. This limitation primarily stems from their limited denoising capabilities for dense and large-scale point clouds and their inclination to generate noisy outliers after denoising. To deal with this challenge, we introduce 3DMambaIPF, for the first time, exploiting Selective State Space Models (SSMs) architecture to handle highly-dense and large-scale point clouds, capitalizing on its strengths in selective input processing and large context modeling capabilities. Additionally, we present a robust and fast differentiable rendering loss to constrain the noisy points around the surface. In contrast to previous methodologies, this differentiable rendering loss enhances the visual realism of denoised geometric structures and aligns point cloud boundaries more closely with those observed in real-world objects. Extensive evaluations on commonly used datasets (typically with up to 50K points) demonstrate that 3DMambaIPF achieves state-of-the-art results. Moreover, we showcase the superior scalability and efficiency of 3DMambaIPF on highly dense and large-scale point clouds with up to 500K points compared to off-the-shelf methods.

PDF Details DOI

EAAI Journal 2025 Journal Article

An unsupervised anatomy-aware dual-constraint cascade network for lung computed tomography deformable image registration

  • Wenbin Wu
  • Yifan Gao
  • Xin Jin
  • Rui Zhang
  • Yuemei Pan
  • Xin Gao

Deformable image registration (DIR) of lung computed tomography (CT) facilitates precise radiotherapy for lung cancer by aligning images from different respiratory states or time points. Due to the high resolution of lung CT image data, the large amplitude of lung motion, and significant differences in image intensity, achieving fast and accurate DIR of lung CT is challenging. In this study, we propose an unsupervised Anatomy-aware Dual-constraint Cascade network, called ADC-Net. We construct an efficient single-level network based on fully convolutional networks, into which a tri-directional deformation capture (TDC) block is designed and integrated. The TDC block calculates voxel correlations along three spatial dimensions separately, comprehensively capturing deformations in different directions, particularly large deformations along the lung's vertical axis. To further enhance the awareness of large deformations as well as internal anatomical details, we propose a dual-constraint cascade mode. In this mode, contour-enhanced and vessel-enhanced images used as constraints, along with the original images, are fed into three single-level networks, which enables accurate registration of large deformations while maintaining precise alignment of internal details. Compared to the state-of-the-art cascade method, MANet, ADC-Net achieves a registration accuracy improvement of 3 % and 4 % on the Dir-lab and ThoraxCBCT datasets, respectively, with a notable accuracy increase of 16 % for images with large deformations. Additionally, the low-computational-cost upsampling structure adopted by the single-level network ensures efficiency and makes ADC-Net 21 % faster than MANet. ADC-Net's high accuracy, rapid processing speed, and robust applicability in DIR of lung CT suggest its potential value for clinical engineering applications.

Details DOI

IJCAI Conference 2025 Conference Paper

Automated Superscalar Processor Design by Learning Data Dependencies

  • Shuyao Cheng
  • Rui Zhang
  • Wenkai He
  • Pengwei Jin
  • Chongxiao Li
  • Zidong Du
  • Xing Hu
  • Yifan Hao

Automated processor design, which can significantly reduce human efforts and accelerate design cycles, has received considerable attention. While recent advancements have automatically designed single-cycle processors that execute one instruction per cycle, their performance cannot compete with modern superscalar processors that execute multiple instructions per cycle. Previous methods fail on superscalar processor design because they cannot address inter-instruction data dependencies, leading to inefficient sequential instruction execution. This paper proposes a novel approach to automatically designing superscalar processors using a hardware-friendly model called the Stateful Binary Speculation Diagram (State-BSD). We observe that processor parallelism can be enhanced through on-the-fly inter-instruction dependent data predictors, reusing the processor's internal states to learn the data dependency. To meet the challenge of both hardware-resource limitation and design functional correctness, State-BSD consists of two components: 1) a lightweight state-selector trained by simulated annealing method to detect the most reusable processor states and store them in a small buffer; and 2) a highly precise state-speculator trained by BSD expansion method to predict the inter-instruction dependent data using the selected states. It is the first work to achieve the automated superscalar processor design, i. e. QiMeng-CPU-v2, which improves the performance by about 380x than the state-of-the-art automated design and is comparable to human-designed superscalar processors such as ARM Cortex A53.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

ChatbotID: Identifying Chatbots with Granger Causality Test

  • Xiaoquan Yi
  • Haozhao Wang
  • Yining Qi
  • Wenchao Xu
  • Rui Zhang
  • Yuhua Li
  • Ruixuan Li

With the increasing sophistication of Large Language Models (LLMs), it is crucial to develop reliable methods to accurately identify whether an interlocutor in real-time dialogue is human or chatbot. However, existing detection methods are primarily designed for analyzing full documents, not the unique dynamics and characteristics of dialogue. These approaches frequently overlook the nuances of interaction that are essential in conversational contexts. This work identifies two key patterns in dialogues: (1) Human-Human (H-H) interactions exhibit significant bidirectional sentiment influence, while (2) Human-Chatbot (H-C) interactions display a clear asymmetric pattern. We propose an innovative approach named ChatbotID, which applies the Granger Causality Test (GCT) to extract a novel set of interactional features that capture the evolving, predictive relationships between conversational attributes. By synergistically fusing these GCT-based interactional features with contextual embeddings, and optimizing the model through a meticulous loss function. Experimental results across multiple datasets and detection models demonstrate the effectiveness of our framework, with significant improvements in accuracy for distinguishing between H-H and H-C dialogues.

PDF Details

AAAI Conference 2025 Conference Paper

Clean-Label Graph Backdoor Attack in the Node Classification Task

  • Hui Xia
  • Xiangwei Zhao
  • Rui Zhang
  • Shuo Xu
  • Luming Wang

Graph neural networks (GNNs) have achieved impressive results in various graph learning tasks. Backdoor attacks pose a significant threat to GNNs, with a focus on dirty-label attacks. However, these attacks often necessitate the inclusion of blatantly incorrect inputs into the training set, rendering them easily detectable through simple filtering. In response to this challenge, we introduce Clean-Label Graph Backdoor Attack (CGBA). The majority of features in the generated poisoned nodes align with their true labels, significantly enhancing the difficulty of detecting the attack. Firstly, leveraging the uncertainty inherent in the GNNs, we develop a low-budget strategy for selecting poisoned nodes. This approach focuses on nodes in the target class with uncertain and low-degree classifications, allowing for efficient attacks within a limited budget while mitigating the impact on other clean nodes. Secondly, we present an innovative strategy for generating feature triggers. By boosting the confidence of poisoned samples in the target class, this tactic establishes a robust association between the trigger and the target class, even without modifying the labels of poisoned nodes. Additionally, we incorporate two constraints to reduce disruption to the graph structure. In conclusion, comprehensive experimental results unequivocally showcase CGBA's exceptional attack performance across three benchmark datasets and four GNNs models. Notably, the attack targeting the GraphSAGE model attains a 100% success rate, accompanied by a marginal benign accuracy drop of no more than 0.5%.

PDF Details DOI

TMLR Journal 2025 Journal Article

Combining Machine Learning Defenses without Conflicts

  • Vasisht Duddu
  • Rui Zhang
  • N. Asokan

Machine learning (ML) models require protection against various risks to security, privacy, and fairness. Real-life ML models need simultaneous protection against multiple risks, necessitating combining multiple defenses effectively, without incurring significant drop in the effectiveness of the constituent defenses. We present a systematization of existing work based on how defenses are combined, and how they interact. We then identify unexplored combinations, and evaluate combination techniques to identify their limitations. Using these insights, we present, Def\Con, a combination technique which is (a) accurate (correctly identifies whether a combination is effective or not), (b) scalable (allows combining multiple defenses), (c) non-invasive (allows combining existing defenses without modification), and (d) general (is applicable to different types of defenses). We show that Def\Con achieves 90% accuracy on eight combinations from prior work, and 86% in 30 unexplored combinations evaluated empirically.

PDF Details

EAAI Journal 2025 Journal Article

Dual-branch crack segmentation network with multi-shape kernel based on convolutional neural network and Mamba

  • Jianming Zhang
  • Dianwen Li
  • Zhigao Zeng
  • Rui Zhang
  • Jin Wang

Cracks are one of the most common pavement diseases. If not promptly repaired, they will hasten the deterioration of the road. Semantic segmentation is the most convenient pavement crack detection method to assess the damage level. Convolutional neural networks (CNN) excel at extracting local spatial information, but they have limitations in capturing global contextual information. Therefore, a dual-branch crack segmentation network (DBCNet) with Mamba and multi-shape convolutional kernels is proposed. First, a dual-branch encoder is employed to extract both spatial and contextual information, consisting of the spatial branch and the context branch. The cross-like block (CrossBlock) that excels in extracting spatial information horizontally and vertically from cracks is proposed. Multiple CrossBlocks are stacked to construct a lightweight network as a spatial branch. The improved Visual State Space Model (VMamba) serves as a context branch for modeling long-range dependencies for more accurate pixel-by-pixel segmentation. Second, the Feature Fusion Module (FFM), based on squeeze-and-excitation attention, is constructed to dynamically fuse the features from the two branches layer by layer. Third, a Cross-aware Mamba Module (CMM) with the hybrid CNN-Mamba architecture is proposed to compose the decoder. Fourth, comprehensive evaluations were conducted on three public datasets. Performs on multiple metrics achieved considerable progress, outperforming the seven state-of-the-art models. The mean intersection over union (mIoU) on Deepcrack, CrackTree 260, and CFD reached 87. 87%, 85. 34%, and 81. 35%, respectively. Code and data will be available at https: //github. com/name191/DBCNet.

Details DOI

YNIMG Journal 2025 Journal Article

Dynamic grouping of ongoing activity in V1 hypercolumns

  • Rui Zhang
  • Jiayu Wang
  • Xingya Cai
  • Rendong Tang
  • Haidong D. Lu

Neurons' spontaneous activity provides rich information about the brain. A single neuron's activity has close relationships with the local network. In order to understand such relationships, we studied the spontaneous activity of thousands of neurons in macaque V1 and V2 with two-photon calcium imaging. In V1, the ongoing activity was dominated by global fluctuations in which the activity of majority of neurons were correlated. Neurons' activity also relied on their relative locations within the local functional architectures, including ocular dominance, orientation, and color maps. Neurons with similar preferences dynamically grouped into co-activating ensembles and exhibited spatial patterns resembling the local functional maps. Different ensembles had different strengths and frequencies. This observation was consistent across all hypercolumn-sized V1 locations we examined. In V2, different imaging sites had different orientation and color features. However, the spontaneous activity in the sampled regions also correlated with the underlying functional architectures. These results indicate that functional architectures play an essential role in influencing neurons' spontaneous activity.

Details DOI

AAAI Conference 2025 Conference Paper

EvdCLIP: Improving Vision-Language Retrieval with Entity Visual Descriptions from Large Language Models

  • GuangHao Meng
  • Sunan He
  • Jinpeng Wang
  • Tao Dai
  • Letian Zhang
  • Jieming Zhu
  • Qing Li
  • Gang Wang

Vision-language retrieval (VLR) has attracted significant attention in both academia and industry, which involves using text (or images) as queries to retrieve corresponding images (or text). However, existing methods often neglect the rich visual semantics knowledge of entities, thus leading to incorrect retrieval results. To address this problem, we propose the Entity Visual Description enhanced CLIP (EvdCLIP), designed to leverage the visual knowledge of entities to enrich queries. Specifically, since humans recognize entities through visual cues, we employ a large language model (LLM) to generate Entity Visual Descriptions (EVDs) as alignment cues to complement textual data. These EVDs are then integrated into raw queries to create visually-rich, EVD-enhanced queries. Furthermore, recognizing that EVD-enhanced queries may introduce noise or low-quality expansions, we develop a novel, trainable EVD-aware Rewriter (EaRW) for vision-language retrieval tasks. EaRW utilizes EVD knowledge and the generative capabilities of the language model to effectively rewrite queries. With our specialized training strategy, EaRW can generate high-quality and low-noise EVD-enhanced queries. Extensive quantitative and qualitative experiments on image-text retrieval benchmarks validate the superiority of EvdCLIP on vision-language retrieval tasks.

PDF Details DOI

JBHI Journal 2025 Journal Article

Fine-Grained Spatial-Frequency-Time Framework for Motor Imagery Brain–Computer Interface

  • Guoyang Liu
  • Rui Zhang
  • Lan Tian
  • Weidong Zhou

The Motor Imagery Brain–Computer Interfaces (MI-BCIs) have shown considerable promise for applications in neural rehabilitation. However, improving the practicality and interpretability of MI-BCIs remains a critical challenge. Unlike previous methods that focus generally on either spatial, frequency, or temporal domains with coarse-grained segmentation schemes, this study proposes a novel fine-grained spatial-frequency-time (FGSFT) framework, aiming to enhance the efficiency and reliability of MI-BCIs. Multi-channel MI EEG recordings are firstly processed through multiscale time-frequency segmentation and spatial segmentation schemes, yielding fine-grained spatial-frequency-time segments (SFTSs). The key SFTSs are then selected with a tailored wrapper-based feature selection approach. Discriminative MI EEG features are extracted using a divergence-based common spatial pattern algorithm with intra-class regularization and classified using an efficient linear support vector machine (SVM). The proposed framework was evaluated on the BCI IV IIa and SDU-MI datasets, demonstrating state-of-the-art performance in terms of information transfer rate (ITR) Meanwhile, the proposed spatial segmentation strategy can significantly improve the performance of MI-BCIs when using a larger number of electrodes. Additionally, the fine-grained Motor Imagery Time-Frequency Reaction Map (MI-TFRM) and time-frequency topographical map can be obtained with the proposed framework enabling visualization of the subject-specific dynamic neural process during motor imagery tasks, facilitating the devising of personalized MI-BCIs. The FGSFT framework significantly advances the accuracy, ITR, and interoperability of MI-BCIs, paving the way for future neuroscientific research and clinical applications in neural rehabilitation and assistive technologies.

Details DOI

YNIMG Journal 2025 Journal Article

Identifying individuals with high susceptibility to mental fatigue: A functional connectivity study

  • Lingyun Gao
  • Mengru Xu
  • Linze Qian
  • Rui Zhang
  • Mingming Chen
  • Yeting Hu
  • Chuantao Li
  • Yu Sun

Substantial inter-individual differences in behavioral performance were repeatedly revealed during prolonged time-on-task (TOT), indicating complex neural mechanisms underlying mental fatigue. In this work, we provide a comprehensive investigation to identify individuals with high susceptibility to mental fatigue and to reveal its influence on brain network reorganization. Specifically, behavioral data and EEG signals were collected from 95 participants when they performed a 20-min psychomotor vigilance task (PVT). A composite index ( F i n d e x ) was introduced, based upon which the participants were categorized into the fatigue-susceptible (FS, corresponding to top third F i n d e x value) and the fatigue-resistant (FR, corresponding to bottom third F i n d e x value) groups ( N F S / N F R = 30/30). Functional connectivity was then estimated and set as input for the following analyses. As expect, significant impairment of behavioral performance was showed in the FS group, while the performance of the FR group remained relatively stable. Subsequent brain network analyses showed frequency-dependent reorganizations in both groups, whereas the FR group exhibited greater stability and higher integrity than the FS group. Further classification analyses revealed satisfactory accuracy for FS identification (95. 61%) and the prominent centro-parietal distribution of contributing nodal features. In sum, this study provides further evidence to support the notion of substantial individual differences in fatigue susceptibility and provides a practical approach to identify the individuals whose performance is particularly prone to performance decline.

Details DOI

AAAI Conference 2025 Conference Paper

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

  • Yutong Wu
  • Di Huang
  • Wenxuan Shi
  • Wei Wang
  • Yewen Pu
  • Lingzhe Gao
  • Shihao Liu
  • Ziyuan Nan

Recent advancements in open-source code large language models (LLMs) have been driven by fine-tuning on the data generated from powerful closed-source LLMs, which are expensive to obtain. This paper explores whether it is possible to use a fine-tuned open-source model to generate additional data to augment its instruction-tuning dataset. We make two observations: (1) A code snippet can serve as the response to different instructions. (2) Instruction-tuned code LLMs perform better at translating code into instructions than the reverse. Based on these observations, we propose Inverse-Instruct, a data augmentation technique that uses a fine-tuned LLM to generate additional instructions of code responses from its own training dataset. The additional instruction-response pairs are added to the original dataset, and a stronger code LLM can be obtained by fine-tuning on the augmented dataset. We empirically validate Inverse-Instruct on a range of open-source code models (e.g. CodeLlama-Python and DeepSeek-Coder) and benchmarks (e.g., HumanEval(+), MBPP(+), DS-1000 and MultiPL-E), showing it consistently improves the base models.

PDF Details DOI

IROS Conference 2025 Conference Paper

Motion Control of a Hybrid Self-Reconfigurable Wheel-Legged Dual-Arm Robot

  • Rui Zhang
  • Hong Du
  • Peng Qiu
  • Yi Yang 0009
  • Wenjie Song 0001

Current wheeled bipedal robots face significant mobility challenges when traversing discontinuous terrain such as gaps and step-like obstacles, and suffer from substantial dynamic inefficiencies. This paper presents a hybrid self-reconfigurable wheel-legged dual-arm robot equipped with an active docking mechanism, enabling transitions between wheeled bipedal and multi-wheel-legged configurations. Based on a self-developed robotic platform, this work addresses key control challenges in articulated multi-wheel-legged mode and proposes a novel distributed operation paradigm for wheeled bipedal robots. Each module utilizes its manipulators for stable grasping of elevated objects and collaborative tasks, while the multi-unit system achieves efficient, high-load, and stable locomotion. To manage the control complexities in multimodal operation, we develop a unified modular control architecture integrating Virtual Model Control (VMC) and Linear Quadratic Regulator (LQR). For the articulated multi-wheel-legged mode, a body-posture controller regulates global body configuration, and a turning controller adjusts the wheelbase and roll angle via distributed actuation to manage the passive degrees of freedom (DoF) at the articulation points. Experimental validation using a physical prototype confirms the effectiveness and practicality of the proposed approach.

Details

EAAI Journal 2025 Journal Article

Paying more attention to local contrast: Improving infrared small target detection performance via prior knowledge

  • Peichao Wang
  • Jiabao Wang
  • Yao Chen
  • Rui Zhang
  • Yang Li
  • Zhuang Miao

The data-driven methods for InfraRed Small Target Detection (IRSTD) have achieved promising results. However, these methods typically incorporate modules with high computational complexity, which enhance performance at the expense of computational efficiency. Utilizing human expert knowledge to assist data-driven methods in better learning with less costs is worthy of exploration. To effectively guide the model to focus on targets’ spatial features, this paper proposes the Local Contrast Attention Enhanced infrared small target detection Network (LCAE-Net), combining prior knowledge with data-driven deep learning methods. LCAE-Net is a U-shaped neural network model which consists of two developed modules: a Local Contrast Enhancement (LCE) module and a Channel Attention Enhancement (CAE) module. The LCE module takes advantage of prior knowledge, leveraging handcrafted convolution operators to acquire Local Contrast Attention (LCA), which could realize background suppression while enhancing the potential target region, thus guiding the neural network to pay more attention to potential infrared small targets’ location information. To effectively utilize the response information throughout the downsampling progresses, the CAE module is proposed to achieve the information fusion among feature maps’ different channels. Experimental results indicate that our LCAE-Net outperforms comparison methods on the three public datasets, and its detection speed could reach up to 70 Frames Per Second (FPS). Meanwhile, our model has a parameter count and Floating-Point Operations (FLOPs) of 1. 945 Million (M) and 4. 862 Giga (G) respectively, which is suitable for deployment on edge devices. Our code will be available at https: //github. com/boa2004plaust/LCAENet.

Details DOI

IJCAI Conference 2025 Conference Paper

PeSANet: Physics-encoded Spectral Attention Network for Simulating PDE-Governed Complex Systems

  • Han Wan
  • Rui Zhang
  • Qi Wang
  • Yang Liu
  • Hao Sun

Accurately modeling and forecasting complex systems governed by partial differential equations (PDEs) is crucial in various scientific and engineering domains. However, traditional numerical methods struggle in real-world scenarios due to incomplete or unknown physical laws. Meanwhile, machine learning approaches often fail to generalize effectively when faced with scarce observational data and the challenge of capturing local and global features. To this end, we propose the Physics-encoded Spectral Attention Network (PeSANet), which integrates local and global information to forecast complex systems with limited data and incomplete physical priors. The model consists of two key components: a physics-encoded block that uses hard constraints to approximate local differential operators from limited data, and a spectral-enhanced block that captures long-range global dependencies in the frequency domain. Specifically, we introduce a novel spectral attention mechanism to model inter-spectrum relationships and learn long-range spatial features. Experimental results demonstrate that PeSANet outperforms existing methods across all metrics, particularly in long-term forecasting accuracy, providing a promising solution for simulating complex systems with limited data and incomplete physics.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation

  • Yaoyu Zhu
  • Di Huang
  • Hanqi Lyu
  • Xiaoyun Zhang
  • Chongxiao Li
  • Wenxuan Shi
  • Yutong Wu
  • Jianan Mu

Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scarcity of high-quality NL-code pairs, and the prohibitive computation cost of RLVR. To this end, we introduce CodeV-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust equivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM-generated NL descriptions, verifies code–NL–code consistency via the generated testbench, and filters out inequivalent examples to yield a high-quality dataset. Third, we employ a two-stage "distill-then-RL" training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, our novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, CodeV-R1-7B, achieves 68. 6 \% and 72. 9 \% pass@1 on VerilogEval v2 and RTLLM v1. 1, respectively, surpassing prior state-of-the-art by 12$\sim$20 \%, while even exceeding the performance of 671B DeepSeek-R1 on RTLLM. We have released our model, training code, and dataset to facilitate research in EDA and LLM communities.

PDF Details

NeurIPS Conference 2025 Conference Paper

QiMeng-MuPa: Mutual-Supervised Learning for Sequential-to-Parallel Code Translation

  • Changxin Ke
  • Rui Zhang
  • Shuo Wang
  • Li Ding
  • Guangli Li
  • Yuanbo Wen
  • Shuoming Zhang
  • Ruiyuan Xu

The rise of GPU-based high-performance computing (HPC) has driven the widespread adoption of parallel programming models such as CUDA. Yet, the inherent complexity of parallel programming creates a demand for the automated sequential-to-parallel approaches. However, data scarcity poses a significant challenge for machine learning-based sequential-to-parallel code translation. Although recent back-translation methods show promise, they still fail to ensure functional equivalence in the translated code. In this paper, we propose \textbf{QiMeng-MuPa}, a novel \textbf{Mu}tual-Supervised Learning framework for Sequential-to-\textbf{Pa}rallel code translation, to address the functional equivalence issue. QiMeng-MuPa consists of two models, a Translator and a Tester. Through an iterative loop consisting of Co-verify and Co-evolve steps, the Translator and the Tester mutually generate data for each other and improve collectively. The Tester generates unit tests to verify and filter functionally equivalent translated code, thereby evolving the Translator, while the Translator generates translated code as augmented input to evolve the Tester. Experimental results demonstrate that QiMeng-MuPa significantly enhances the performance of the base models: when applied to Qwen2. 5-Coder, it not only improves Pass@1 by up to 28. 91\% and boosts Tester performance by 68. 90\%, but also outperforms the previous state-of-the-art method CodeRosetta by 1. 56 and 6. 92 in BLEU and CodeBLEU scores, while achieving performance comparable to DeepSeek-R1 and GPT-4. 1. Our code is available at \url{https: //github. com/kcxain/mupa}.

PDF Details

NeurIPS Conference 2025 Conference Paper

QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code

  • Hainan Fang
  • Yuanbo Wen
  • Jun Bi
  • Yihan Wang
  • Tonghui He
  • Yanlin Tang
  • Di Huang
  • Jiaming Guo

Compilers, while essential, are notoriously complex systems that demand prohibitively expensive human expertise to develop and maintain. The recent advancements in Large Language Models (LLMs) offer a compelling new paradigm: Neural Compilation, which could potentially simplify compiler development for new architectures and facilitate the discovery of innovative optimization techniques. However, several critical obstacles impede its practical adoption. Firstly, a significant lack of dedicated benchmarks and robust evaluation methodologies hinders objective assessment and tracking of progress in the field. Secondly, systematically enhancing the reliability and performance of LLM-generated assembly remains a critical challenge. Addressing these challenges, this paper introduces NeuComBack, a novel benchmark dataset specifically designed for IR-to-assembly compilation. Leveraging this dataset, we first define a foundational Neural Compilation workflow and conduct a comprehensive evaluation of the capabilities of recent frontier LLMs on Neural Compilation, establishing new performance baselines. We further propose a self-evolving prompt optimization method that enables LLMs to iteratively evolve their internal prompt strategies by extracting insights from prior self-debugging traces, thereby enhancing their neural compilation capabilities. Experiments demonstrate that our method significantly improves both the functional correctness and the performance of LLM-generated assembly code. Compared to baseline prompts, the functional correctness rates improved from 44% to 64% on x86 64 and from 36% to 58% on aarch64, respectively. More significantly, among the 16 correctly generated x86 64 programs using our method, 14 (87. 5%) surpassed clang-O3 performance. These consistent improvements across diverse architectures (x86_64 and aarch64) and program distributions (NeuComBack L1 and L2) validate our method's superiority over conventional approaches and its potential for broader adoption in low-level neural compilation.

PDF Details

NeurIPS Conference 2025 Conference Paper

QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation

  • Yang Zhang
  • Rui Zhang
  • Jiaming Guo
  • Huang Lei
  • Di Huang
  • Yunpu Zhao
  • Shuyao Cheng
  • Pengwei Jin

The remarkable progress of Large Language Models (LLMs) presents promising opportunities for Verilog code generation which is significantly important for automated circuit design. The lacking of meaningful functional rewards hinders the preference optimization based on Reinforcement Learning (RL) for producing functionally correct Verilog code. In this paper, we propose Signal-Aware Learning for Verilog code generation (QiMeng-SALV) by leveraging code segments of functionally correct output signal to optimize RL training. Considering Verilog code specifies the structural interconnection of hardware gates and wires so that different output signals are independent, the key insight of QiMeng-SALV is to extract verified signal-aware implementations in partially incorrect modules, so as to enhance the extraction of meaningful functional rewards. Roughly, we verify the functional correctness of signals in generated module by comparing with that of reference module in the training data. Then abstract syntax tree (AST) is employed to identify signal-aware code segments which can provide meaningful functional rewards from erroneous modules. Finally, we introduce signal-aware DPO which is optimized on the correct signal-level code segments, thereby preventing noise and interference from incorrect signals. The proposed QiMeng-SALV underscores the paradigm shift from conventional module-level to fine-grained signal-level optimization in Verilog code generation, addressing the issue of insufficient functional rewards. Experiments demonstrate that our method achieves state-of-the-art performance on VerilogEval and RTLLM, with a 7B parameter model matching the performance of the DeepSeek v3 671B model and significantly outperforming the leading open-source model CodeV trained on the same dataset.

PDF Details

JBHI Journal 2025 Journal Article

Real-Time Epileptic Seizure Prediction Method With Spatio-Temporal Information Transfer Learning

  • Kunying Meng
  • Denghai Wang
  • Donghui Zhang
  • Kunlin Guo
  • Kai Lu
  • Junfeng Lu
  • Renping Yu
  • Lipeng Zhang

Despite numerous studies aimed at improving accuracy, the accurate prediction of epileptic seizures remains a challenge in clinical practice due to the high computational cost, poor real-time performance, and over-reliance on labelled data. To address these issues, a real-time seizure prediction method with spatio-temporal information transfer learning (RTSPM-STITL) has been proposed in this study. In the RTSPM-STITL method, the human brain is regarded as a time-varying high-dimensional neurodynamic system, in which epileptic seizures are viewed as state transitions caused by time-varying system parameters. Specifically, the spatio-temporal information transfer (STIT) model is firstly constructed by the recurrent neural network (RNN) and trained by the Force Learning (a real-time learning mechanism). Then the STIT model is utilized to transform the high-dimensional neurodynamic data into low-dimensional time series to capture the dynamic features of epileptic seizures. Also, the critical slowing down effect (CSD) of seizure dynamics is used to detect warning signals. The experimental results demonstrate that the proposed method can achieve higher accuracy and sensitivity without labeled data on both the CHB-MIT and Siena scalp EEG databases. Especially, the parameters of the STIT model can be updated in real-time based on patient data, without iterative training. More importantly, the STIT model can maintain high sensitivity and accuracy with only 48400 parameters, which is reduced by more than 91% compared with contrast models in this experiment. Therefore, the proposed method can significantly reduce the computational cost and accurately predict epileptic seizures, as well as with high real-time, practicality, applicability, and interpretability.

Details DOI

NeurIPS Conference 2025 Conference Paper

Resource-Constrained Federated Continual Learning: What Does Matter?

  • Yichen Li
  • Yuying Wang
  • Jiahua Dong
  • Haozhao Wang
  • Yining Qi
  • Rui Zhang
  • Ruixuan Li

Federated Continual Learning (FCL) aims to enable sequential privacy-preserving model training on streams of incoming data that vary in edge devices by preserving previous knowledge while adapting to new data. Current FCL literature focuses on restricted data privacy and access to previously seen data while imposing no constraints on the training overhead. This is unreasonable for FCL applications in real-world scenarios, where edge devices are primarily constrained by resources such as storage, computational budget, and label rate. We revisit this problem with a large-scale benchmark and analyze the performance of state-of-the-art FCL approaches under different resource-constrained settings. Various typical FCL techniques and six datasets in two incremental learning scenarios (Class-IL and Domain-IL) are involved in our experiments. Through extensive experiments amounting to a total of over 1, 000+ GPU hours, we find that, under limited resource-constrained settings, existing FCL approaches, with no exception, fail to achieve the expected performance. Our conclusions are consistent in the sensitivity analysis. This suggests that most existing FCL methods are particularly too resource-dependent for real-world deployment. Moreover, we study the performance of typical FCL techniques with resource constraints and shed light on future research directions in FCL.

PDF Details

JBHI Journal 2025 Journal Article

UKF-Based Model Parameter Estimation to Localize the Seizure Onset Zone in ECoG

  • Junfeng Lu
  • Donghui Zhang
  • Kunlin Guo
  • Kunying Meng
  • Denghai Wang
  • Kai Lu
  • Renping Yu
  • Lifang Yang

Drug-resistant epilepsy (DRE) patients typically require surgical intervention or neurostimulation. Therefore, accurate localization of the seizure onset zone (SOZ) is essential for effective clinical intervention. Although some physiologically meaningful parameters of neural computational models show substantial differences across brain regions during seizures, few studies pay attention to applying these model parameters to SOZ localization. To investigate whether the parameter can be used for accurate SOZ localization, the unscented kalman filter (UKF) is employed to estimate the excitatory-inhibitory balance parameter c from the Z6 neural computational model using DRE patients' electrocorticography (ECoG). The results indicate that this parameter follows a unimodal distribution during the pre-ictal period and the post-ictal period, while exhibiting a bimodal distribution during the ictal period. Then, the distribution of this parameter is combined with machine learning methods, and a bagged tree classifier is constructed to localize the SOZ. The classification results demonstrate that the classifier based on parameter distributions exhibits excellent performance, particularly during the post-ictal period, with an average accuracy of 91. 60%. Interestingly, SOZ localization is more accurate when no lesions are detected on magnetic resonance imaging (MRI) compared to when lesions are present. Finally, the model parameter distributions of the SOZs are utilized to predict the outcome of epilepsy surgery. Of note, the results demonstrate that the parameter distribution accurately predicts surgical outcomes with an average accuracy of 92. 56%. These findings suggest that the distribution of neural computational model parameters may serve as biomarkers for SOZ localization and epilepsy surgery outcome prediction, providing valuable support and assistance for clinical decision-making.

Details DOI

AAAI Conference 2025 Conference Paper

Watch Out for Your Guidance on Generation! Exploring Conditional Backdoor Attacks against Large Language Models

  • Jiaming He
  • Wenbo Jiang
  • Guanyu Hou
  • Wenshu Fan
  • Rui Zhang
  • Hongwei Li

Mainstream backdoor attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of backdoor activation, we present a new poisoning paradigm against LLMs triggered by specifying generation conditions, which are commonly adopted strategies by users during model inference. The poisoned model performs normally for output under normal/other generation conditions, while becomes harmful for output under target generation conditions. To achieve this objective, we introduce BrieFool, an efficient attack framework. It leverages the characteristics of generation conditions by efficient instruction sampling and poisoning data generation, thereby influencing the behavior of LLMs under target conditions. Our attack can be generally divided into two types with different targets: Safety unalignment attack and Ability degradation attack. Our extensive experiments demonstrate that BrieFool is effective across safety domains and ability domains, achieving higher success rates than baseline methods, with 94.3% on GPT-3.5-turbo.

PDF Details DOI

EAAI Journal 2024 Journal Article

Acoustic localization approach for urban water distribution networks using machine learning method

  • Rui Zhang
  • Abdul-Mugis Yussif
  • Ibrahim Tijani
  • Ali Fares
  • Salman Tariq
  • Tarek Zayed

Leakage in water pipelines causes the loss of valuable amounts of water and can result in considerable environmental damage. This study presents a real-time leak localization method using signal processing and machine learning (ML) techniques, addressing challenges in accuracy, cost, and real-world applicability. Firstly, a novel two-stage localization method is proposed, streamlining the traditional three-stage approach, for reducing cost and time by utilizing only noise loggers and correlators while maintaining effectiveness. To address the disparity between laboratory-based models and real-world applications, a real-life database is established in this study. This continuously updating database incorporates diverse real leak data scenarios from Hong Kong over a period exceeding one year, ensuring model accuracy and relevance in practical settings. Furthermore, various decomposition techniques are employed to extract relevant features, with statistical methods used to select optimal feature combinations. Notably, the fast Fourier transform (FFT) proves effective for acoustic-based leak localization, enhancing model accuracy. Lastly, a novel data-driven leak pinpointing method is introduced, leveraging machine learning regression techniques and a continuously updated database. The Decision Tree algorithm emerges as a robust choice for pinpointing leaks, offering high predictive accuracy across diverse leak scenarios. In new case studies for validating the proposed method, the selected model can predict the leakage point within 1 m with 96. 7% accuracy, and the prediction reduces the potential leakage scope to within 0. 3 m.

Details DOI

JBHI Journal 2024 Journal Article

An EEG Study on $\boldsymbol{\upbeta\!\!-\!\!\upgamma}$ Phase-Amplitude Coupling-Based Functional Brain Network in Epilepsy Patients

  • Junfeng Lu
  • Anyu Li
  • Kaijie Li
  • Renping Yu
  • Yuxia Hu
  • Rui Zhang
  • Lipeng Zhang
  • Hong Wan

Epilepsy, a chronic neuropsychiatric brain disorder characterized with recurrent seizures, is closely associated with abnormal neural communications within the brain. Despite that the phase-amplitude coupling (PAC) has been suggested to offer a new way to observe neural interactions during epilepsy, however, few studies pay attention to alterations of the epileptic functional brain network based on PAC, especially on the $\boldsymbol{\upbeta\! \! -\! \! \upgamma}$ PAC. Therefore, we use scalp electroencephalography (EEG) data of epileptic patients and the $\boldsymbol{\upbeta\! \! -\! \! \upgamma}$ PAC modulation index (MI) to construct functional brain networks to examine variations of neural interactions during different epileptic phases. Statistically, the findings show that between-channel MI values in the post-ictal period significantly increase compared to that in the pre-ictal period, and the between-channel MI value has a close association with the information of phase and amplitude provided by the channels. Importantly, in both the phase-amplitude and amplitude-phase functional brain networks, the average node degree is remarkably higher in the post-ictal period than that in the pre-ictal period, whereas the characteristic path length in the ictal and post-ictal periods is significantly lower than that in the pre-ictal period. Besides, the average betweenness centrality in the post-ictal period is remarkably higher than that in the ictal period. Interestingly, the positive correlations between within-channel MI values and between-channel MI values can be observed during the pre-ictal, ictal and post-ictal periods. These findings suggest that the $\boldsymbol{\upbeta\! \! -\! \! \upgamma}$ PAC-based functional brain network may provide a novel perspective to understanding alterations of neural interactions during the epileptic evolution, and may contribute to effectively controlling the spread of epileptic seizures.

Details DOI

IJCAI Conference 2024 Conference Paper

Automated CPU Design by Learning from Input-Output Examples

  • Shuyao Cheng
  • Pengwei Jin
  • Qi Guo
  • Zidong Du
  • Rui Zhang
  • Xing Hu
  • Yongwei Zhao
  • Yifan Hao

Designing a central processing unit (CPU) requires intensive manual work of talented experts to implement the circuit logic from design specifications. Although considerable progress has been made in electronic design automation (EDA) to relieve human efforts, all existing EDA tools require hand-crafted formal program codes (e. g. , Verilog, Chisel, or C) as the input. To automate the CPU design without human programming, we are motivated to learn the CPU design from only input-output (IO) examples. The key challenge is that the learned CPU design should have almost zero tolerance for inaccuracy, which makes well-known approximate algorithms such as neural networks ineffective. We propose a new AI approach to generate the CPU design in the form of a large-scale Boolean function, from only external IO examples instead of formal program code. This approach employs a novel graph structure called Binary Speculative Diagram (BSD) to approximate the CPU-scale Boolean function accurately. We propose an efficient BSD expansion method based on Boolean Distance, a new metric to quantitatively measure the structural similarity between Boolean functions, gradually increasing the design accuracy up to 100%. Our approach generates an industrial-scale RISC-V CPU design within 5 hours, reducing the design cycle by about 1000x without human involvement. The taped-out chip, Enlightenment-1, the world's first CPU designed by AI, successfully runs the Linux operating system and performs comparably against the human-design Intel 80486SX CPU. Our approach even autonomously discovers human knowledge of the von Neumann architecture.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Chain of Agents: Large Language Models Collaborating on Long-Context Tasks

  • Yusen Zhang
  • Ruoxi Sun
  • Yanfei Chen
  • Tomas Pfister
  • Rui Zhang
  • Sercan Ö. Arık

Addressing the challenge of effectively processing long contexts has become a critical issue for Large Language Models (LLMs). Two common strategies have emerged: 1) reducing the input length, such as retrieving relevant chunks by Retrieval-Augmented Generation (RAG), and 2) expanding the context window limit of LLMs. However, both strategies have drawbacks: input reduction has no guarantee of covering the part with needed information, while window extension struggles with focusing on the pertinent information for solving the task. To mitigate these limitations, we propose Chain-of-Agents (CoA), a novel framework that harnesses multi-agent collaboration through natural language to enable information aggregation and context reasoning across various LLMs over long-context tasks. CoA consists of multiple worker agents who sequentially communicate to handle different segmented portions of the text, followed by a manager agent who synthesizes these contributions into a coherent final output. CoA processes the entire input by interleaving reading and reasoning, and it mitigates long context focus issues by assigning each agent a short context. We perform a comprehensive evaluation of CoA on a wide range of long-context tasks in question answering, summarization, and code completion, demonstrating significant improvements by up to 10% over strong baselines of RAG, Full-Context, and multi-agent LLMs.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection

  • Haochen Li
  • Rui Zhang
  • Hantao Yao
  • Xin Zhang
  • Yifan Hao
  • Xinkai Song
  • Xiaqing Li
  • Yongwei Zhao

Domain adaptive object detection (DAOD) aims to generalize detectors trained on an annotated source domain to an unlabelled target domain. As the visual-language models (VLMs) can provide essential general knowledge on unseen images, freezing the visual encoder and inserting a domain-agnostic adapter can learn domain-invariant knowledge for DAOD. However, the domain-agnostic adapter is inevitably biased to the source domain. It discards some beneficial knowledge discriminative on the unlabelled domain, \ie domain-specific knowledge of the target domain. To solve the issue, we propose a novel Domain-Aware Adapter (DA-Ada) tailored for the DAOD task. The key point is exploiting domain-specific knowledge between the essential general knowledge and domain-invariant knowledge. DA-Ada consists of the Domain-Invariant Adapter (DIA) for learning domain-invariant knowledge and the Domain-Specific Adapter (DSA) for injecting the domain-specific knowledge from the information discarded by the visual encoder. Comprehensive experiments over multiple DAOD tasks show that DA-Ada can efficiently infer a domain-aware visual encoder for boosting domain adaptive object detection. Our code is available at https: //github. com/Therock90421/DA-Ada.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptative Residual Module

  • Jingbo Zhou
  • Yixuan Du
  • Ruqiong Zhang
  • Jun Xia
  • Zhizhi Yu
  • Zelin Zang
  • Di Jin
  • Carl Yang

Graph Neural Networks (GNNs), a type of neural network that can learn from graph-structured data through neighborhood information aggregation, have shown superior performance in various downstream tasks. However, as the number of layers increases, node representations becomes indistinguishable, which is known as over-smoothing. To address this issue, many residual methods have emerged. In this paper, we focus on the over-smoothing issue and related residual methods. Firstly, we revisit over-smoothing from the perspective of overlapping neighborhood subgraphs, and based on this, we explain how residual methods can alleviate over-smoothing by integrating multiple orders neighborhood subgraphs to avoid the indistinguishability of the single high-order neighborhood subgraphs. Additionally, we reveal the drawbacks of previous residual methods, such as the lack of node adaptability and severe loss of high-order neighborhood subgraph information, and propose a \textbf{Posterior-Sampling-based, Node-Adaptive Residual module (PSNR)}. We theoretically demonstrate that PSNR can alleviate the drawbacks of previous residual methods. Furthermore, extensive experiments verify the superiority of the PSNR module in fully observed node classification and missing feature scenarios. Our codeis available at \href{https: //github. com/jingbo02/PSNR-GNN}{https: //github. com/jingbo02/PSNR-GNN}.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Denoising-Aware Contrastive Learning for Noisy Time Series

  • Shuang Zhou
  • Daochen Zha
  • Xiao Shen
  • Xiao Huang
  • Rui Zhang
  • Korris Chung

Time series self-supervised learning (SSL) aims to exploit unlabeled data for pre-training to mitigate the reliance on labels. Despite the great success in recent years, there is limited discussion on the potential noise in the time series, which can severely impair the performance of existing SSL methods. To mitigate the noise, the de facto strategy is to apply conventional denoising methods before model training. However, this pre-processing approach may not fully eliminate the effect of noise in SSL for two reasons: (i) the diverse types of noise in time series make it difficult to automatically determine suitable denoising methods; (ii) noise can be amplified after mapping raw data into latent space. In this paper, we propose denoising-aware contrastive learning (DECL), which uses contrastive learning objectives to mitigate the noise in the representation and automatically selects suitable denoising methods for every sample. Extensive experiments on various datasets verify the effectiveness of our method. The code is open-sourced.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Emergent Communication for Numerical Concepts Generalization

  • Enshuai Zhou
  • Yifan Hao
  • Rui Zhang
  • Yuxuan Guo
  • Zidong Du
  • Xishan Zhang
  • Xinkai Song
  • Chao Wang

Research on emergent communication has recently gained significant traction as a promising avenue for the linguistic community to unravel human language's origins and explore artificial intelligence's generalization capabilities. Current research has predominantly concentrated on recognizing qualitative patterns of object attributes(e.g., shape and color) and paid little attention to the quantitative relationship among object quantities which is known as the part of numerical concepts. The ability to generalize numerical concepts, i.e., counting and calculations with unseen quantities, is essential, as it mirrors humans' foundational abstract reasoning abilities. In this work, we introduce the NumGame, leveraging the referential game framework, forcing agents to communicate and generalize the numerical concepts effectively. Inspired by the human learning process of numbers, we present a two-stage training approach that sequentially fosters a rudimentary numerical sense followed by the ability of arithmetic calculation, ultimately aiding agents in generating semantically stable and unambiguous language for numerical concepts. The experimental results indicate the impressive generalization capabilities to unseen quantities and regularity of the language emergence from communication.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

FastSurvival: Hidden Computational Blessings in Training Cox Proportional Hazards Models

  • Jiachang Liu
  • Rui Zhang
  • Cynthia Rudin

Survival analysis is an important research topic with applications in healthcare, business, and manufacturing. One essential tool in this area is the Cox proportional hazards (CPH) model, which is widely used for its interpretability, flexibility, and predictive performance. However, for modern data science challenges such as high dimensionality (both $n$ and $p$) and high feature correlations, current algorithms to train the CPH model have drawbacks, preventing us from using the CPH model at its full potential. The root cause is that the current algorithms, based on the Newton method, have trouble converging due to vanishing second order derivatives when outside the local region of the minimizer. To circumvent this problem, we propose new optimization methods by constructing and minimizing surrogate functions that exploit hidden mathematical structures of the CPH model. Our new methods are easy to implement and ensure monotonic loss decrease and global convergence. Empirically, we verify the computational efficiency of our methods. As a direct application, we show how our optimization methods can be used to solve the cardinality-constrained CPH problem, producing very sparse high-quality models that were not previously practical to construct. We list several extensions that our breakthrough enables, including optimization opportunities, theoretical questions on CPH's mathematical structure, as well as other CPH-related applications.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Hypothesis, Verification, and Induction: Grounding Large Language Models with Self-Driven Skill Learning

  • Shaohui Peng
  • Xing Hu
  • Qi Yi
  • Rui Zhang
  • Jiaming Guo
  • Di Huang
  • Zikang Tian
  • Ruizhi Chen

Large language models (LLMs) show their powerful automatic reasoning and planning capability with a wealth of semantic knowledge about the human world. However, the grounding problem still hinders the applications of LLMs in the real-world environment. Existing studies try to fine-tune the LLM or utilize pre-defined behavior APIs to bridge the LLMs and the environment, which not only costs huge human efforts to customize for every single task but also weakens the generality strengths of LLMs. To autonomously ground the LLM onto the environment, we proposed the Hypothesis, Verification, and Induction (HYVIN) framework to automatically and progressively ground the LLM with self-driven skill learning. HYVIN first employs the LLM to propose the hypothesis of sub-goals to achieve tasks and then verify the feasibility of the hypothesis via interacting with the underlying environment. Once verified, HYVIN can then learn generalized skills with the guidance of these successfully grounded subgoals. These skills can be further utilized to accomplish more complex tasks that fail to pass the verification phase. Verified in the famous instruction following task set, BabyAI, HYVIN achieves comparable performance in the most challenging tasks compared with imitation learning methods that cost millions of demonstrations, proving the effectiveness of learned skills and showing the feasibility and efficiency of our framework.

PDF Details DOI

EAAI Journal 2024 Journal Article

Image super-resolution reconstruction using Swin Transformer with efficient channel attention networks

  • Zhenxi Sun
  • Jin Zhang
  • Ziyi Chen
  • Lu Hong
  • Rui Zhang
  • Weishi Li
  • Haojie Xia

Image super-resolution reconstruction (SR) is an important ill-posed problem in low-level vision, which aims to reconstruct high-resolution images from low-resolution images. Although current state-of-the-art methods exhibit impressive performance, their recovery of image detail information and edge information is still unsatisfactory. To address this problem, this paper proposes a shifted window Transformer (Swin Transformer) with an efficient channel attention network (S-ECAN), which combines the attention based on convolutional neural networks and the self-attention of the Swin Transformer to combine the advantages of both and focuses on learning high-frequency features of images. In addition, to solve the problem of Convolutional Neural Network (CNN) based channel attention consumes a large number of parameters to achieve good performance, this paper proposes the Efficient Channel Attention Block (ECAB), which only involves a handful of parameters while bringing clear performance gain. Extensive experimental validation shows that the proposed model can recover more high-frequency details and texture information. The model is validated on Set5, Set14, B100, Urban100, and Manga109 datasets, where it outperforms the state-of-the-art methods by 0. 03–0. 13 dB, 0. 04–0. 09 dB, 0. 01–0. 06 dB, 0. 13–0. 20 dB, and 0. 06–0. 17 dB respectively in terms of objective metrics. Ultimately, the substantial performance gains and enhanced visual results over prior arts validate the effectiveness and competitiveness of our proposed approach, which achieves an improved performance-complexity trade-off.

Details DOI

AAAI Conference 2024 Conference Paper

MESED: A Multi-Modal Entity Set Expansion Dataset with Fine-Grained Semantic Classes and Hard Negative Entities

  • Yangning Li
  • Tingwei Lu
  • Hai-Tao Zheng
  • Yinghui Li
  • Shulin Huang
  • Tianyu Yu
  • Jun Yuan
  • Rui Zhang

The Entity Set Expansion (ESE) task aims to expand a handful of seed entities with new entities belonging to the same semantic class. Conventional ESE methods are based on mono-modality (i.e., literal modality), which struggle to deal with complex entities in the real world such as (1) Negative entities with fine-grained semantic differences. (2) Synonymous entities. (3) Polysemous entities. (4) Long-tailed entities. These challenges prompt us to propose novel Multi-modal Entity Set Expansion (MESE), where models integrate information from multiple modalities to represent entities. Intuitively, the benefits of multi-modal information for ESE are threefold: (1) Different modalities can provide complementary information. (2) Multi-modal information provides a unified signal via common visual properties for the same semantic class or entity. (3) Multi-modal information offers robust alignment signals for synonymous entities. To assess model performance in MESE, we constructed the MESED dataset which is the first multi-modal dataset for ESE with large-scale and elaborate manual calibration. A powerful multi-modal model MultiExpan is proposed which is pre-trained on four multimodal pre-training tasks. The extensive experiments and analyses on MESED demonstrate the high quality of the dataset and the effectiveness of our MultiExpan, as well as pointing the direction for future research. The benchmark and code are public at https://github.com/THUKElab/MESED.

PDF Details DOI

AAAI Conference 2024 Conference Paper

OCEAN-MBRL: Offline Conservative Exploration for Model-Based Offline Reinforcement Learning

  • Fan Wu
  • Rui Zhang
  • Qi Yi
  • Yunkai Gao
  • Jiaming Guo
  • Shaohui Peng
  • Siming Lan
  • Husheng Han

Model-based offline reinforcement learning (RL) algorithms have emerged as a promising paradigm for offline RL. These algorithms usually learn a dynamics model from a static dataset of transitions, use the model to generate synthetic trajectories, and perform conservative policy optimization within these trajectories. However, our observations indicate that policy optimization methods used in these model-based offline RL algorithms are not effective at exploring the learned model and induce biased exploration, which ultimately impairs the performance of the algorithm. To address this issue, we propose Offline Conservative ExplorAtioN (OCEAN), a novel rollout approach to model-based offline RL. In our method, we incorporate additional exploration techniques and introduce three conservative constraints based on uncertainty estimation to mitigate the potential impact of significant dynamic errors resulting from exploratory transitions. Our work is a plug-in method and can be combined with classical model-based RL algorithms, such as MOPO, COMBO, and RAMBO. Experiment results of our method on the D4RL MuJoCo benchmark show that OCEAN significantly improves the performance of existing algorithms.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Pre-trained Online Contrastive Learning for Insurance Fraud Detection

  • Rui Zhang
  • Dawei Cheng
  • Jie Yang
  • Yi Ouyang
  • Xian Wu
  • Yefeng Zheng
  • Changjun Jiang

Medical insurance fraud has always been a crucial challenge in the field of healthcare industry. Existing fraud detection models mostly focus on offline learning scenes. However, fraud patterns are constantly evolving, making it difficult for models trained on past data to detect newly emerging fraud patterns, posing a severe challenge in medical fraud detection. Moreover, current incremental learning models are mostly designed to address catastrophic forgetting, but often exhibit suboptimal performance in fraud detection. To address this challenge, this paper proposes an innovative online learning method for medical insurance fraud detection, named POCL. This method combines contrastive learning pre-training with online updating strategies. In the pre-training stage, we leverage contrastive learning pre-training to learn on historical data, enabling deep feature learning and obtaining rich risk representations. In the online learning stage, we adopt a Temporal Memory Aware Synapses online updating strategy, allowing the model to perform incremental learning and optimization based on continuously emerging new data. This ensures timely adaptation to fraud patterns and reduces forgetting of past knowledge. Our model undergoes extensive experiments and evaluations on real-world insurance fraud datasets. The results demonstrate our model has significant advantages in accuracy compared to the state-of-the-art baseline methods, while also exhibiting lower running time and space consumption. Our sources are released at https://github.com/finint/POCL.

PDF Details DOI

EAAI Journal 2024 Journal Article

Prompt-based learning framework for zero-shot cross-lingual text classification

  • Kai Feng
  • Lan Huang
  • Kangping Wang
  • Wei Wei
  • Rui Zhang

Cross-lingual text classification is a challenging task that aims to train classifiers with data in one language, known as the source language, and apply the acquired knowledge to data in another language, referred to as the target language. Recent advancements in multilingual pre-trained language models (PLMs) have made significant progress in addressing cross-lingual issues, and the application of prompt-based learning has further improved task performance. However, these models still face challenges such as the gap between cross-lingual classification tasks and pre-training tasks of PLMs, as well as issues related to scarce resources and data noise, which hinder the full exploitation of the implicit knowledge in PLMs. In this paper, we propose a Prompt-based Cross-lingual Learning (PCL) framework that combines language-agnostic continuous prompt learning with self-learning process. Specifically, PCL framework leverages language-agnostic prompts and PLMs to achieve semantic transfer between source and target languages. To enhance the semantic relationship between prompts and category labels, a label attention module is introduced. Additionally, a set of self-training rules is proposed, which includes a scoring function. In a few-shot setting, noisy data is dynamically filtered through scoring and ranking of the data. During each training iteration, both the model and scoring function weights are updated, further improving the discrimination capability of the model. In summary, the proposed PCL framework builds upon cross-lingual prompt learning, effectively removing noisy data and applying it to zero-shot cross-lingual text classification, which is beneficial for engineering applications. The findings of this study have implications for prompt learning method. The PCL framework achieves state-of-the-art performance in cross-lingual text classification task, with a 14% performance improvement compared to basic soft prompt learning. This demonstrates its potential in addressing classification problems in resource-limited scenarios.

Details DOI

EAAI Journal 2024 Journal Article

Research on X-ray weld seam defect detection and size measurement method based on neural network self-optimization

  • Rui Zhang
  • Donghao Liu
  • Qiaofeng Bai
  • Liuhu Fu
  • Jing Hu
  • Jinlong Song

To effectively solve the problems of low detection accuracy caused by low quality image discriminative features of X-ray weld defects, lack of intuitive representation of the defect size of existing detection algorithms, and the strong subjectivity of the detection model artificial parameter adjustment, an X-ray weld defect detection and size measurement algorithm based on neural network self-optimization is proposed. Firstly, a high-performance detection model for X-ray weld defects is constructed, and the detection accuracy is comprehensively improved through a series of featured module designs with the capabilities of feature information enhancement and multi-scale information fusion. Secondly, a model optimization strategy is proposed to obtain the optimal hyperparameter components of the model through adaptive optimization to enhance the model's self-learning capability. Finally, by constructing the mapping relationship between the actual size of defects and the screen resolution, the size measurement algorithm of weld defects is designed, and the integrated technology of defect detection and size measurement is realised. Experimental results show that the proposed algorithm achieves good results even on a small-scale X-ray weld seam defect dataset. Compared to other classical and advanced detection models used in the experiments, mAP@0. 5 is improved by an average of 16. 1% and mAP@. 5: .95 by an average of 10. 7%. The image processing speed reaches up to 68 frames per second, and the error between the size calibration and manual actual measurement is less than 0. 1 cm, which can meet the real-time detection requirements for weld seam defects in practical industrial production.

Details DOI

AAAI Conference 2023 Conference Paper

Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction

  • Shiwei Li
  • Huifeng Guo
  • Lu Hou
  • Wei Zhang
  • Xing Tang
  • Ruiming Tang
  • Rui Zhang
  • Ruixuan Li

Embedding tables are usually huge in click-through rate (CTR) prediction models. To train and deploy the CTR models efficiently and economically, it is necessary to compress their embedding tables. To this end, we formulate a novel quantization training paradigm to compress the embeddings from the training stage, termed low-precision training (LPT). Also, we provide theoretical analysis on its convergence. The results show that stochastic weight quantization has a faster convergence rate and a smaller convergence error than deterministic weight quantization in LPT. Further, to reduce accuracy degradation, we propose adaptive low-precision training (ALPT) which learns the step size (i.e., the quantization resolution). Experiments on two real-world datasets confirm our analysis and show that ALPT can significantly improve the prediction accuracy, especially at extremely low bit width. For the first time in CTR models, we successfully train 8-bit embeddings without sacrificing prediction accuracy.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

ANPL: Towards Natural Programming with Interactive Decomposition

  • Di Huang
  • Ziyuan Nan
  • Xing Hu
  • Pengwei Jin
  • Shaohui Peng
  • Yuanbo Wen
  • Rui Zhang
  • Zidong Du

Though LLMs are capable of generating plausible programs, it’s challenging to interact with the LLMs further to revise the program, especially if the user’s specific requirements are different from the initial proposal. In this paper, we introduce ANPL, an interactive programming system that ensures users can always refine the generated code towards their specific programmatic intents via structureddecompositions. Borrowing the paradigm of sketching from program synthesis, an ANPL program consists of a set of input-outputs that it must satisfy, a “sketch” — control/data flow expressed in precise code (e. g. Python), and “holes” — sub-modules to be implemented by the LLM specified with natural language. The user revises an ANPL program by either modifying the sketch, changing the language used to describe the holes, or providing additional input-outputs to a particular hole, turning it into a sub-ANPL program that can be solved recursively. This workflow allows the users to offload programming burdens to the LLM as much as possible while retaining the ability to pinpoint and resolve bugs locally, without exposing the rest of the program to the LLM. We deploy ANPL on the Abstraction and Reasoning Corpus (ARC), a set of unique tasks that are challenging for state-of-the-art AI systems, showing it outperforms baseline programming systems that (a) without the ability to decompose tasks interactively and (b) without the guarantee that the modules can be correctly composed together. Additional evaluations on APPS, HumanEval, and real-world programming tasks have validated that the ANPL framework is applicable to multiple programming domains. We release the ANPL solutions to the ARC tasks as a dataset, providing insights into how humans decompose novel tasks programmatically.

PDF Details

TMLR Journal 2023 Journal Article

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

  • Aarohi Srivastava
  • Abhinav Rastogi
  • Abhishek Rao
  • Abu Awal Md Shoeb
  • Abubakar Abid
  • Adam Fisch
  • Adam R. Brown
  • Adam Santoro

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG- bench). BIG-bench currently consists of 204 tasks, contributed by 450 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood develop- ment, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google- internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting.

PDF Details

AAAI Conference 2023 Conference Paper

Conceptual Reinforcement Learning for Language-Conditioned Tasks

  • Shaohui Peng
  • Xing Hu
  • Rui Zhang
  • Jiaming Guo
  • Qi Yi
  • Ruizhi Chen
  • Zidong Du
  • Ling Li

Despite the broad application of deep reinforcement learning (RL), transferring and adapting the policy to unseen but similar environments is still a significant challenge. Recently, the language-conditioned policy is proposed to facilitate policy transfer through learning the joint representation of observation and text that catches the compact and invariant information across various environments. Existing studies of language-conditioned RL methods often learn the joint representation as a simple latent layer for the given instances (episode-specific observation and text), which inevitably includes noisy or irrelevant information and cause spurious correlations that are dependent on instances, thus hurting generalization performance and training efficiency. To address the above issue, we propose a conceptual reinforcement learning (CRL) framework to learn the concept-like joint representation for language-conditioned policy. The key insight is that concepts are compact and invariant representations in human cognition through extracting similarities from numerous instances in real-world. In CRL, we propose a multi-level attention encoder and two mutual information constraints for learning compact and invariant concepts. Verified in two challenging environments, RTFM and Messenger, CRL significantly improves the training efficiency (up to 70%) and generalization ability (up to 30%) to the new environment dynamics.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Context Shift Reduction for Offline Meta-Reinforcement Learning

  • Yunkai Gao
  • Rui Zhang
  • Jiaming Guo
  • Fan Wu
  • Qi Yi
  • Shaohui Peng
  • Siming Lan
  • Ruizhi Chen

Offline meta-reinforcement learning (OMRL) utilizes pre-collected offline datasets to enhance the agent's generalization ability on unseen tasks. However, the context shift problem arises due to the distribution discrepancy between the contexts used for training (from the behavior policy) and testing (from the exploration policy). The context shift problem leads to incorrect task inference and further deteriorates the generalization ability of the meta-policy. Existing OMRL methods either overlook this problem or attempt to mitigate it with additional information. In this paper, we propose a novel approach called Context Shift Reduction for OMRL (CSRO) to address the context shift problem with only offline datasets. The key insight of CSRO is to minimize the influence of policy in context during both the meta-training and meta-test phases. During meta-training, we design a max-min mutual information representation learning mechanism to diminish the impact of the behavior policy on task representation. In the meta-test phase, we introduce the non-prior context collection strategy to reduce the effect of the exploration policy. Experimental results demonstrate that CSRO significantly reduces the context shift and improves the generalization ability, surpassing previous methods across various challenging domains.

PDF Details

NeurIPS Conference 2023 Conference Paper

Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning

  • Siming Lan
  • Rui Zhang
  • Qi Yi
  • Jiaming Guo
  • Shaohui Peng
  • Yunkai Gao
  • Fan Wu
  • Ruizhi Chen

In the field of multi-task reinforcement learning, the modular principle, which involves specializing functionalities into different modules and combining them appropriately, has been widely adopted as a promising approach to prevent the negative transfer problem that performance degradation due to conflicts between tasks. However, most of the existing multi-task RL methods only combine shared modules at the task level, ignoring that there may be conflicts within the task. In addition, these methods do not take into account that without constraints, some modules may learn similar functions, resulting in restricting the model's expressiveness and generalization capability of modular methods. In this paper, we propose the Contrastive Modules with Temporal Attention(CMTA) method to address these limitations. CMTA constrains the modules to be different from each other by contrastive learning and combining shared modules at a finer granularity than the task level with temporal attention, alleviating the negative transfer within the task and improving the generalization ability and the performance for multi-task RL. We conducted the experiment on Meta-World, a multi-task RL benchmark containing various robotics manipulation tasks. Experimental results show that CMTA outperforms learning each task individually for the first time and achieves substantial performance improvements over the baselines.

PDF Details

IJCAI Conference 2023 Conference Paper

CSGCL: Community-Strength-Enhanced Graph Contrastive Learning

  • Han Chen
  • Ziwen Zhao
  • Yuhua Li
  • Yixiong Zou
  • Ruixuan Li
  • Rui Zhang

Graph Contrastive Learning (GCL) is an effective way to learn generalized graph representations in a self-supervised manner, and has grown rapidly in recent years. However, the underlying community semantics has not been well explored by most previous GCL methods. Research that attempts to leverage communities in GCL regards them as having the same influence on the graph, leading to extra representation errors. To tackle this issue, we define ''community strength'' to measure the difference of influence among communities. Under this premise, we propose a Community-Strength-enhanced Graph Contrastive Learning (CSGCL) framework to preserve community strength throughout the learning process. Firstly, we present two novel graph augmentation methods, Communal Attribute Voting (CAV) and Communal Edge Dropping (CED), where the perturbations of node attributes and edges are guided by community strength. Secondly, we propose a dynamic ''Team-up'' contrastive learning scheme, where community strength is used to progressively fine-tune the contrastive objective. We report extensive experiment results on three downstream tasks: node classification, node clustering, and link prediction. CSGCL achieves state-of-the-art performance compared with other GCL methods, validating that community strength brings effectiveness and generality to graph representations. Our code is available at https: //github. com/HanChen-HUST/CSGCL.

PDF Details DOI

EAAI Journal 2023 Journal Article

CVT on-line error measurement hybrid-driven by domain knowledge and Stacking Model

  • Jingping Wang
  • Ying Shi
  • Rui Zhang
  • Zhonghua Wu
  • Hao Ye
  • Shenwei Li

The performance of Capacitive Voltage Transformer (CVT) degrades over time, making measurement error monitoring a research hotspot in the field of smart grid. At present, these are several challenges such as complex data features, a lack of criteria for selecting optimal measurement models, and low precision. CVT measurement errors can be classified into ideal error and additional one. The former is typically evaluated via mutual information and redundancy within the topology-level transformer group. Considering that a single model cannot process the time series, strong randomness and nonlinearity of the additional error, the Stacking model is selected. Based on the principle of heterogeneity and high-quality, correlation coefficient and feature contribution degree, Random Forest, eXtreme Gradient Boosting, Ridge Regression, K Nearest Neighbors, Support Vector Regression, and Long Short-Term Memory are chosen as base learners through correlation and feature contribution analysis; while extra-trees with strong generalization and robustness is chosen as the meta learner. To improve the measurement precision, the attention-like mechanism is used to scale time and accuracy weights. Finally, according to the power domain knowledge, a linear superposition model is developed to fuse the ideal and additional errors, and thus realize online error measurement for CVTs. The experimental results indicate that the improved Stacking model outperforms mainstream measurement models by an average reduction of 59. 47%, and 52. 58% in the root mean squared error and the mean absolute error with the best R 2 closest to 1. It not only effectively improves the accuracy but also meets speed requirement for online error measurement.

Details DOI

NeurIPS Conference 2023 Conference Paper

Decompose a Task into Generalizable Subtasks in Multi-Agent Reinforcement Learning

  • Zikang Tian
  • Ruizhi Chen
  • Xing Hu
  • Ling Li
  • Rui Zhang
  • Fan Wu
  • Shaohui Peng
  • Jiaming Guo

In recent years, Multi-Agent Reinforcement Learning (MARL) techniques have made significant strides in achieving high asymptotic performance in single task. However, there has been limited exploration of model transferability across tasks. Training a model from scratch for each task can be time-consuming and expensive, especially for large-scale Multi-Agent Systems. Therefore, it is crucial to develop methods for generalizing the model across tasks. Considering that there exist task-independent subtasks across MARL tasks, a model that can decompose such subtasks from the source task could generalize to target tasks. However, ensuring true task-independence of subtasks poses a challenge. In this paper, we propose to \textbf{d}ecompose a \textbf{t}ask in\textbf{to} a series of \textbf{g}eneralizable \textbf{s}ubtasks (DT2GS), a novel framework that addresses this challenge by utilizing a scalable subtask encoder and an adaptive subtask semantic module. We show that these components endow subtasks with two properties critical for task-independence: avoiding overfitting to the source task and maintaining consistent yet scalable semantics across tasks. Empirical results demonstrate that DT2GS possesses sound zero-shot generalization capability across tasks, exhibits sufficient transferability, and outperforms existing methods in both multi-task and single-task problems.

PDF Details

NeurIPS Conference 2023 Conference Paper

Efficient Symbolic Policy Learning with Differentiable Symbolic Expression

  • Jiaming Guo
  • Rui Zhang
  • Shaohui Peng
  • Qi Yi
  • Xing Hu
  • Ruizhi Chen
  • Zidong Du
  • Xishan Zhang

Deep reinforcement learning (DRL) has led to a wide range of advances in sequential decision-making tasks. However, the complexity of neural network policies makes it difficult to understand and deploy with limited computational resources. Currently, employing compact symbolic expressions as symbolic policies is a promising strategy to obtain simple and interpretable policies. Previous symbolic policy methods usually involve complex training processes and pre-trained neural network policies, which are inefficient and limit the application of symbolic policies. In this paper, we propose an efficient gradient-based learning method named Efficient Symbolic Policy Learning (ESPL) that learns the symbolic policy from scratch in an end-to-end way. We introduce a symbolic network as the search space and employ a path selector to find the compact symbolic policy. By doing so we represent the policy with a differentiable symbolic expression and train it in an off-policy manner which further improves the efficiency. In addition, in contrast with previous symbolic policies which only work in single-task RL because of complexity, we expand ESPL on meta-RL to generate symbolic policies for unseen tasks. Experimentally, we show that our approach generates symbolic policies with higher performance and greatly improves data efficiency for single-task RL. In meta-RL, we demonstrate that compared with neural network policies the proposed symbolic policy achieves higher performance and efficiency and shows the potential to be interpretable.

PDF Details

NeurIPS Conference 2023 Conference Paper

Emergent Communication for Rules Reasoning

  • Yuxuan Guo
  • Yifan Hao
  • Rui Zhang
  • Enshuai Zhou
  • Zidong Du
  • Xishan Zhang
  • Xinkai Song
  • Yuanbo Wen

Research on emergent communication between deep-learning-based agents has received extensive attention due to its inspiration for linguistics and artificial intelligence. However, previous attempts have hovered around emerging communication under perception-oriented environmental settings, that forces agents to describe low-level perceptual features intra image or symbol contexts. In this work, inspired by the classic human reasoning test (namely Raven's Progressive Matrix), we propose the Reasoning Game, a cognition-oriented environment that encourages agents to reason and communicate high-level rules, rather than perceived low-level contexts. Moreover, we propose 1) an unbiased dataset (namely rule-RAVEN) as a benchmark to avoid overfitting, 2) and a two-stage curriculum agent training method as a baseline for more stable convergence in the Reasoning Game, where contexts and semantics are bilaterally drifting. Experimental results show that, in the Reasoning Game, a semantically stable and compositional language emerges to solve reasoning problems. The emerged language helps agents apply the extracted rules to the generalization of unseen context attributes, and to the transfer between different context attributes or even tasks.

PDF Details

AAAI Conference 2023 Conference Paper

Fair-CDA: Continuous and Directional Augmentation for Group Fairness

  • Rui Sun
  • Fengwei Zhou
  • Zhenhua Dong
  • Chuanlong Xie
  • Lanqing Hong
  • Jiawei Li
  • Rui Zhang
  • Zhen Li

In this work, we propose Fair-CDA, a fine-grained data augmentation strategy for imposing fairness constraints. We use a feature disentanglement method to extract the features highly related to the sensitive attributes. Then we show that group fairness can be achieved by regularizing the models on transition paths of sensitive features between groups. By adjusting the perturbation strength in the direction of the paths, our proposed augmentation is controllable and auditable. To alleviate the accuracy degradation caused by fairness constraints, we further introduce a calibrated model to impute labels for the augmented data. Our proposed method does not assume any data generative model and ensures good generalization for both accuracy and fairness. Experimental results show that Fair-CDA consistently outperforms state-of-the-art methods on widely-used benchmarks, e.g., Adult, CelebA and MovieLens. Especially, Fair-CDA obtains an 86.3% relative improvement for fairness while maintaining the accuracy on the Adult dataset. Moreover, we evaluate Fair-CDA in an online recommendation system to demonstrate the effectiveness of our method in terms of accuracy and fairness.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

FastGR: Global Routing on CPU-GPU with Heterogeneous Task Graph Scheduler (Extended Abstract)

  • Siting Liu
  • Yuan Pu
  • Peiyu Liao
  • Hongzhong Wu
  • Rui Zhang
  • Zhitang Chen
  • Wenlong Lv
  • Yibo Lin

Running time is a key metric across the standard physical design flow stages. However, with the rapid growth in design sizes, routing runtime has become the runtime bottleneck in the physical design flow. To improve the effectiveness of the modern global router, we propose a global routing framework with GPU-accelerated routing algorithms and a heterogeneous task graph scheduler, called FastGR. Its runtime-oriented version FastGRL achieves 2. 489× speedup compared with the state-of-the-art global router. Furthermore, the GPU-accelerated L-shape pattern routing used in FastGRL can contribute to 9. 324× speedup over the sequential algorithm on CPU. Its quality-oriented version FastGRH offers further quality improvement over FastGRL with similar acceleration.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

Fighting against Organized Fraudsters Using Risk Diffusion-based Parallel Graph Neural Network

  • Jiacheng Ma
  • Fan Li
  • Rui Zhang
  • Zhikang Xu
  • Dawei Cheng
  • Yi Ouyang
  • Ruihui Zhao
  • Jianguang Zheng

Medical insurance plays a vital role in modern society, yet organized healthcare fraud causes billions of dollars in annual losses, severely harming the sustainability of the social welfare system. Existing works mostly focus on detecting individual fraud entities or claims, ignoring hidden conspiracy patterns. Hence, they face severe challenges in tackling organized fraud. In this paper, we proposed RDPGL, a novel Risk Diffusion-based Parallel Graph Learning approach, to fighting against medical insurance criminal gangs. In particular, we first leverage a heterogeneous graph attention network to encode the local context from the beneficiary-provider graph. Then, we devise a community-aware risk diffusion model to infer the global context of organized fraud behaviors with the claim-claim relation graph. The local and global representations are parallel concatenated together and trained simultaneously in an end-to-end manner. Our approach is extensively evaluated on a real-world medical insurance dataset. The experimental results demonstrate the superiority of our proposed approach, which could detect more organized fraud claims with relatively high precision compared with state-of-the-art baselines.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

JEPOO: Highly Accurate Joint Estimation of Pitch, Onset and Offset for Music Information Retrieval

  • Haojie Wei
  • Jun Yuan
  • Rui Zhang
  • Yueguo Chen
  • Gang Wang

Melody extraction is a core task in music information retrieval, and the estimation of pitch, onset and offset are key sub-tasks in melody extraction. Existing methods have limited accuracy, and work for only one type of data, either single-pitch or multi-pitch. In this paper, we propose a highly accurate method for joint estimation of pitch, onset and offset, named JEPOO. We address the challenges of joint learning optimization and handling both single-pitch and multi-pitch data through novel model design and a new optimization technique named Pareto modulated loss with loss weight regularization. This is the first method that can accurately handle both single-pitch and multi-pitch music data, and even a mix of them. A comprehensive experimental study on a wide range of real datasets shows that JEPOO outperforms state-of-the-art methods by up to 10. 6\%, 8. 3\% and 10. 3\% for the prediction of Pitch, Onset and Offset, respectively, and JEPOO is robust for various types of data and instruments. The ablation study validates the effectiveness of each component of JEPOO.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Learning Domain-Aware Detection Head with Prompt Tuning

  • Haochen Li
  • Rui Zhang
  • Hantao Yao
  • Xinkai Song
  • Yifan Hao
  • Yongwei Zhao
  • Ling Li
  • Yunji Chen

Domain adaptive object detection (DAOD) aims to generalize detectors trained on an annotated source domain to an unlabelled target domain. However, existing methods focus on reducing the domain bias of the detection backbone by inferring a discriminative visual encoder, while ignoring the domain bias in the detection head. Inspired by the high generalization of vision-language models (VLMs), applying a VLM as the robust detection backbone following a domain-aware detection head is a reasonable way to learn the discriminative detector for each domain, rather than reducing the domain bias in traditional methods. To achieve the above issue, we thus propose a novel DAOD framework named Domain-Aware detection head with Prompt tuning (DA-Pro), which applies the learnable domain-adaptive prompt to generate the dynamic detection head for each domain. Formally, the domain-adaptive prompt consists of the domain-invariant tokens, domain-specific tokens, and the domain-related textual description along with the class label. Furthermore, two constraints between the source and target domains are applied to ensure that the domain-adaptive prompt can capture the domains-shared and domain-specific knowledge. A prompt ensemble strategy is also proposed to reduce the effect of prompt disturbance. Comprehensive experiments over multiple cross-domain adaptation tasks demonstrate that using the domain-adaptive prompt can produce an effectively domain-related detection head for boosting domain-adaptive object detection. Our code is available at https: //github. com/Therock90421/DA-Pro.

PDF Details

YNIMG Journal 2023 Journal Article

Mesoscale functional connectivity in macaque visual areas

  • Xingya Cai
  • Haoran Xu
  • Chao Han
  • Peichao Li
  • Jiayu Wang
  • Rui Zhang
  • Rendong Tang
  • Chen Fang

Studies of resting-state functional connectivity (rsFC) have provided rich insights into the structures and functions of the human brain. However, most rsFC studies have focused on large-scale brain connectivity. To explore rsFC at a finer scale, we used intrinsic signal optical imaging to image the ongoing activity of the anesthetized macaque visual cortex. Differential signals from functional domains were used to quantify network-specific fluctuations. In 30-60 min resting-state imaging, a series of coherent activation patterns were observed in all three visual areas we examined (V1, V2, and V4). These patterns matched the known functional maps (ocular dominance, orientation, color) obtained in visual stimulation conditions. These functional connectivity (FC) networks fluctuated independently over time and exhibited similar temporal characteristics. Coherent fluctuations, however, were observed from orientation FC networks in different areas and even across two hemispheres. Thus, FC in the macaque visual cortex was fully mapped both on a fine scale and over a long range. Hemodynamic signals can be used to explore mesoscale rsFC in a submillimeter resolution.

Details DOI

AAAI Conference 2023 Short Paper

MGIA: Mutual Gradient Inversion Attack in Multi-Modal Federated Learning (Student Abstract)

  • Xuan Liu
  • Siqi Cai
  • Lin Li
  • Rui Zhang
  • Song Guo

Recent studies have demonstrated that local training data in Federated Learning can be recovered from gradients, which are called gradient inversion attacks. These attacks display powerful effects on either computer vision or natural language processing tasks. As it is known that there are certain correlations between multi-modality data, we argue that the threat of such attacks combined with Multi-modal Learning may cause more severe effects. Different modalities may communicate through gradients to provide richer information for the attackers, thus improving the strength and efficiency of the gradient inversion attacks. In this paper, we propose the Mutual Gradient Inversion Attack (MGIA), by utilizing the shared labels between image and text modalities combined with the idea of knowledge distillation. Our experimental results show that MGIA achieves the best quality of both modality data and label recoveries in comparison with other methods. In the meanwhile, MGIA verifies that multi-modality gradient inversion attacks are more likely to disclose private information than the existing single-modality attacks.

PDF Details DOI

TMLR Journal 2023 Journal Article

On the Robustness of Dataset Inference

  • Sebastian Szyller
  • Rui Zhang
  • Jian Liu
  • N Asokan

Machine learning (ML) models are costly to train as they can require a significant amount of data, computational resources and technical expertise. Thus, they constitute valuable intellectual property that needs protection from adversaries wanting to steal them. Ownership verification techniques allow the victims of model stealing attacks to demonstrate that a suspect model was in fact stolen from theirs. Although a number of ownership verification techniques based on watermarking or fingerprinting have been proposed, most of them fall short either in terms of security guarantees (well-equipped adversaries can evade verification) or computational cost. A fingerprinting technique, Dataset Inference (DI) has been shown to offer better robustness and efficiency than prior methods. The authors of DI provided a correctness proof for linear (suspect) models. However, in a subspace of the same setting, we prove that DI suffers from high false positives (FPs) -- it can incorrectly identify an independent model trained with non-overlapping data from the same distribution as stolen. We further prove that DI also triggers FPs in realistic, non-linear suspect models. We then confirm empirically that DI in the black-box setting leads to FPs, with high confidence. Second, we show that DI also suffers from false negatives (FNs) -- an adversary can fool DI by regularising a stolen model's decision boundaries using adversarial training, thereby leading to an FN. To this end, we demonstrate that black-box DI fails to identify a model adversarially trained from a stolen dataset -- the setting where DI is the hardest to evade. Finally, we discuss the implications of our findings, the viability of fingerprinting-based ownership verification in general, and suggest directions for future work.

PDF Details

AAAI Conference 2023 Conference Paper

Online Symbolic Regression with Informative Query

  • Pengwei Jin
  • Di Huang
  • Rui Zhang
  • Xing Hu
  • Ziyuan Nan
  • Zidong Du
  • Qi Guo
  • Yunji Chen

Symbolic regression, the task of extracting mathematical expressions from the observed data, plays a crucial role in scientific discovery. Despite the promising performance of existing methods, most of them conduct symbolic regression in an offline setting. That is, they treat the observed data points as given ones that are simply sampled from uniform distributions without exploring the expressive potential of data. However, for real-world scientific problems, the data used for symbolic regression are usually actively obtained by doing experiments, which is an online setting. Thus, how to obtain informative data that can facilitate the symbolic regression process is an important problem that remains challenging. In this paper, we propose QUOSR, a query-based framework for online symbolic regression that can automatically obtain informative data in an iterative manner. Specifically, at each step, QUOSR receives historical data points, generates new x, and then queries the symbolic expression to get the corresponding y, where the (x, y) serves as new data points. This process repeats until the maximum number of query steps is reached. To make the generated data points informative, we implement the framework with a neural network and train it by maximizing the mutual information between generated data points and the target expression. Through comprehensive experiments, we show that QUOSR can facilitate modern symbolic regression methods by generating informative data.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Optimal Sparse Regression Trees

  • Rui Zhang
  • Rui Xin
  • Margo Seltzer
  • Cynthia Rudin

Regression trees are one of the oldest forms of AI models, and their predictions can be made without a calculator, which makes them broadly useful, particularly for high-stakes applications. Within the large literature on regression trees, there has been little effort towards full provable optimization, mainly due to the computational hardness of the problem. This work proposes a dynamic programming-with-bounds approach to the construction of provably-optimal sparse regression trees. We leverage a novel lower bound based on an optimal solution to the k-Means clustering algorithm on one dimensional data. We are often able to find optimal sparse trees in seconds, even for challenging datasets that involve large numbers of samples and highly-correlated features.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

REASONER: An Explainable Recommendation Dataset with Comprehensive Labeling Ground Truths

  • Xu Chen
  • Jingsen Zhang
  • Lei Wang
  • Quanyu Dai
  • Zhenhua Dong
  • Ruiming Tang
  • Rui Zhang
  • Li Chen

Explainable recommendation has attracted much attention from the industry and academic communities. It has shown great potential to improve the recommendation persuasiveness, informativeness and user satisfaction. In the past few years, while a lot of promising explainable recommender models have been proposed, the datasets used to evaluate them still suffer from several limitations, for example, the explanation ground truths are not labeled by the real users, the explanations are mostly single-modal and around only one aspect. To bridge these gaps, in this paper, we build a new explainable recommendation dataset, which, to our knowledge, is the first contribution that provides a large amount of real user labeled multi-modal and multi-aspect explaination ground truths. In specific, we firstly develop a video recommendation platform, where a series of questions around the recommendation explainability are carefully designed. Then, we recruit about 3000 high-quality labelers with different backgrounds to use the system, and collect their behaviors and feedback to our questions. In this paper, we detail the construction process of our dataset and also provide extensive analysis on its characteristics. In addition, we develop a library, where ten well-known explainable recommender models are implemented in a unified framework. Based on this library, we build several benchmarks for different explainable recommendation tasks. At last, we present many new opportunities brought by our dataset, which are expected to promote the field of explainable recommendation. Our dataset, library and the related documents have been released at https: //reasoner2023. github. io/.

PDF Details

IJCAI Conference 2022 Conference Paper

A Survey on Gradient Inversion: Attacks, Defenses and Future Directions

  • Rui Zhang
  • Song Guo
  • Junxiao Wang
  • Xin Xie
  • Dacheng Tao

Recent studies have shown that the training samples can be recovered from gradients, which are called Gradient Inversion (GradInv) attacks. However, there remains a lack of extensive surveys covering recent advances and thorough analysis of this issue. In this paper, we present a comprehensive survey on GradInv, aiming to summarize the cutting-edge research and broaden the horizons for different domains. Firstly, we propose a taxonomy of GradInv attacks by characterizing existing attacks into two paradigms: iteration- and recursion-based attacks. In particular, we dig out some critical ingredients from the iteration-based attacks, including data initialization, model training and gradient matching. Second, we summarize emerging defense strategies against GradInv attacks. We find these approaches focus on three perspectives covering data obscuration, model improvement and gradient protection. Finally, we discuss some promising directions and open problems for further research.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

  • Shaohui Peng
  • Xing Hu
  • Rui Zhang
  • Ke Tang
  • Jiaming Guo
  • Qi Yi
  • Ruizhi Chen
  • Xishan Zhang

Hierarchical reinforcement learning (HRL) has been proven to be effective for tasks with sparse rewards, for it can improve the agent's exploration efficiency by discovering high-quality hierarchical structures (e. g. , subgoals or options). However, automatically discovering high-quality hierarchical structures is still a great challenge. Previous HRL methods can only find the hierarchical structures in simple environments, as they are mainly achieved through the randomness of agent's policies during exploration. In complicated environments, such a randomness-driven exploration paradigm can hardly discover high-quality hierarchical structures because of the low exploration efficiency. In this paper, we propose CDHRL, a causality-driven hierarchical reinforcement learning framework, to build high-quality hierarchical structures efficiently in complicated environments. The key insight is that the causalities among environment variables are naturally fit for modeling reachable subgoals and their dependencies; thus, the causality is suitable to be the guidance in building high-quality hierarchical structures. Roughly, we build the hierarchy of subgoals based on causality autonomously, and utilize the subgoal-based policies to unfold further causality efficiently. Therefore, CDHRL leverages a causality-driven discovery instead of a randomness-driven exploration for high-quality hierarchical structure construction. The results in two complex environments, 2D-Minecraft and Eden, show that CDHRL can discover high-quality hierarchical structures and significantly enhance exploration efficiency.

PDF Details

JBHI Journal 2022 Journal Article

Interpretability Analysis of One-Year Mortality Prediction for Stroke Patients Based on Deep Neural Network

  • Shuo Zhang
  • Jing Wang
  • Lulu Pei
  • Kai Liu
  • Yuan Gao
  • Hui Fang
  • Rui Zhang
  • Lu Zhao

Clinically, physicians collect the benchmark medical data to establish archives for a stroke patient and then add the follow up data regularly. It has great significance on prognosis prediction for stroke patients. In this paper, we present an interpretable deep learning model to predict the one-year mortality risk on stroke. We design sub-modules to reconstruct features from original clinical data that highlight the dissimilarity and temporality of different variables. The model consists of Bidirectional Long Short-Term Memory (Bi-LSTM), in which a novel correlation attention module is proposed that takes the correlation of variables into consideration. In experiments, datasets are collected clinically from the department of neurology in a local AAA hospital. It consists of 2, 275 stroke patients hospitalized in the department of neurology from 2014 to 2016. Our model achieves a precision of 0. 9414, a recall of 0. 9502 and an F1-score of 0. 9415. In addition, we provide the analysis of the interpretability by visualizations with reference to clinical professional guidelines.

Details DOI

IJCAI Conference 2022 Conference Paper

Multi-scale Spatial Representation Learning via Recursive Hermite Polynomial Networks

  • Lin (Yuanbo) Wu
  • Deyin Liu
  • Xiaojie Guo
  • Richang Hong
  • Liangchen Liu
  • Rui Zhang

Multi-scale representation learning aims to leverage diverse features from different layers of Convolutional Neural Networks (CNNs) for boosting the feature robustness to scale variance. For dense prediction tasks, two key properties should be satisfied: the high spatial variance across convolutional layers, and the sub-scale granularity inside a convolutional layer for fine-grained features. To pursue the two properties, this paper proposes Recursive Hermite Polynomial Networks (RHP-Nets for short). The proposed RHP-Nets consist of two major components: 1) a dilated convolution to maintain the spatial resolution across layers, and 2) a family of Hermite polynomials over a subset of dilated grids, which recursively constructs sub-scale representations to avoid the artifacts caused by naively applying the dilation convolution. The resultant sub-scale granular features are fused via trainable Hermite coefficients to form the multi-resolution representations that can be fed into the next deeper layer, and thus allowing feature interchanging at all levels. Extensive experiments are conducted to demonstrate the efficacy of our design, and reveal its superiority over state-of-the-art alternatives on a variety of image recognition tasks. Besides, introspective studies are provided to further understand the properties of our method.

PDF Details DOI

IJCAI Conference 2022 Conference Paper

Neural Re-ranking in Multi-stage Recommender Systems: A Review

  • Weiwen Liu
  • Yunjia Xi
  • Jiarui Qin
  • Fei Sun
  • Bo Chen
  • Weinan Zhang
  • Rui Zhang
  • Ruiming Tang

As the final stage of the multi-stage recommender system (MRS), re-ranking directly affects users’ experience and satisfaction by rearranging the input ranking lists, and thereby plays a critical role in MRS. With the advances in deep learning, neural re-ranking has become a trending topic and been widely adopted in industrial applications. This review aims at integrating re-ranking algorithms into a broader picture, and paving ways for more comprehensive solutions for future research. For this purpose, we first present a taxonomy of current methods on neural re-ranking. Then we give a description of these methods along with the historic development according to their objectives. The network structure, personalization, and complexity are also discussed and compared. Next, we provide a benchmark for the major neural re-ranking models and quantitatively analyze their re-ranking performance. Finally, the review concludes with a discussion on future prospects of this field. A list of papers discussed in this review, the benchmark datasets, our re-ranking library LibRerank, and detailed parameter settings are publicly available at https: //github. com/LibRerank-Community/LibRerank.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Object-Category Aware Reinforcement Learning

  • Qi Yi
  • Rui Zhang
  • Shaohui Peng
  • Jiaming Guo
  • Xing Hu
  • Zidong Du
  • Xishan Zhang
  • Qi Guo

Object-oriented reinforcement learning (OORL) is a promising way to improve the sample efficiency and generalization ability over standard RL. Recent works that try to solve OORL tasks without additional feature engineering mainly focus on learning the object representations and then solving tasks via reasoning based on these object representations. However, none of these works tries to explicitly model the inherent similarity between different object instances of the same category. Objects of the same category should share similar functionalities; therefore, the category is the most critical property of an object. Following this insight, we propose a novel framework named Object-Category Aware Reinforcement Learning (OCARL), which utilizes the category information of objects to facilitate both perception and reasoning. OCARL consists of three parts: (1) Category-Aware Unsupervised Object Discovery (UOD), which discovers the objects as well as their corresponding categories; (2) Object-Category Aware Perception, which encodes the category information and is also robust to the incompleteness of (1) at the same time; (3) Object-Centric Modular Reasoning, which adopts multiple independent and object-category-specific networks when reasoning based on objects. Our experiments show that OCARL can improve both the sample efficiency and generalization in the OORL domain.

PDF Details

IJCAI Conference 2022 Conference Paper

On the Opportunity of Causal Learning in Recommendation Systems: Foundation, Estimation, Prediction and Challenges

  • Peng Wu
  • Haoxuan Li
  • Yuhao Deng
  • Wenjie Hu
  • Quanyu Dai
  • Zhenhua Dong
  • Jie Sun
  • Rui Zhang

Recently, recommender system (RS) based on causal inference has gained much attention in the industrial community, as well as the states of the art performance in many prediction and debiasing tasks. Nevertheless, a unified causal analysis framework has not been established yet. Many causal-based prediction and debiasing studies rarely discuss the causal interpretation of various biases and the rationality of the corresponding causal assumptions. In this paper, we first provide a formal causal analysis framework to survey and unify the existing causal-inspired recommendation methods, which can accommodate different scenarios in RS. Then we propose a new taxonomy and give formal causal definitions of various biases in RS from the perspective of violating the assumptions adopted in causal analysis. Finally, we formalize many debiasing and prediction tasks in RS, and summarize the statistical and machine learning-based causal estimation methods, expecting to provide new research opportunities and perspectives to the causal RS community.

PDF Details DOI

AAAI Conference 2022 Conference Paper

Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms

  • Runpeng Yu
  • Hong Zhu
  • Kaican Li
  • Lanqing Hong
  • Rui Zhang
  • Nanyang Ye
  • Shao-Lun Huang
  • Xiuqiang He

Due to the poor generalization performance of traditional empirical risk minimization (ERM) in the case of distributional shift, Out-of-Distribution (OoD) generalization algorithms receive increasing attention. However, OoD generalization algorithms overlook the great variance in the quality of training data, which significantly compromises the accuracy of these methods. In this paper, we theoretically reveal the relationship between training data quality and algorithm performance and analyze the optimal regularization scheme for Lipschitz regularized invariant risk minimization. A novel algorithm is proposed based on the theoretical results to alleviate the influence of low-quality data at both the sample level and the domain level. The experiments on both the regression and classification benchmarks validate the effectiveness of our method with statistical significance.

PDF Details

AAAI Conference 2022 Conference Paper

Rethinking Influence Functions of Neural Networks in the Over-Parameterized Regime

  • Rui Zhang
  • Shihua Zhang

Understanding the black-box prediction for neural networks is challenging. To achieve this, early studies have designed influence function (IF) to measure the effect of removing a single training point on neural networks. However, the classic implicit Hessian-vector product (IHVP) method for calculating IF is fragile, and theoretical analysis of IF in the context of neural networks is still lacking. To this end, we utilize the neural tangent kernel (NTK) theory to calculate IF for the neural network trained with regularized mean-square loss, and prove that the approximation error can be arbitrarily small when the width is sufficiently large for two-layer ReLU networks. We analyze the error bound for the classic IHVP method in the over-parameterized regime to understand when and why it fails or not. In detail, our theoretical analysis reveals that (1) the accuracy of IHVP depends on the regularization term, and is pretty low under weak regularization; (2) the accuracy of IHVP has a significant correlation with the probability density of corresponding training points. We further borrow the theory from NTK to understand the IFs better, including quantifying the complexity for influential samples and depicting the variation of IFs during the training dynamics. Numerical experiments on real-world data confirm our theoretical results and demonstrate our findings.

PDF Details

IROS Conference 2022 Conference Paper

ULSM: Underground Localization and Semantic Mapping with Salient Region Loop Closure under Perceptually-Degraded Environment

  • Junhui Wang
  • Bin Tian 0003
  • Rui Zhang
  • Long Chen 0005

Simultaneous Localization and Mapping (SLAM) has greatly assisted in exploring perceptually-degraded underground environments, such as human-made tunnels, mine tunnels, and caves. However, the recurring sensor failures and spurious loop closures in these scenes bring significant challenges to applying SLAM. This paper proposes an architecture for underground localization and semantic mapping (ULSM) that promotes the robustness of odometry estimation and map-building. In this architecture, a two-stage robust motion compensation method is proposed to adapt to sensor-failure situations. The proposed salient region loop closure detection contributes to avoiding spurious loop closures. Meanwhile, the 2D pose as the initial value for point cloud registration is estimated without additional input. We also design a multi-robot cooperative mapping scheme based on descriptors of the salient region. Extensive experiments are conducted on datasets collected in the Tunnel Circuit of DARPA Subterranean Challenge.

Details

AAAI Conference 2021 Conference Paper

Detecting Beneficial Feature Interactions for Recommender Systems

  • Yixin Su
  • Rui Zhang
  • Sarah Erfani
  • Zhenghua Xu

Feature interactions are essential for achieving high accuracy in recommender systems. Many studies take into account the interaction between every pair of features. However, this is suboptimal because some feature interactions may not be that relevant to the recommendation result, and taking them into account may introduce noise and decrease recommendation accuracy. To make the best out of feature interactions, we propose a graph neural network approach to effectively model them, together with a novel technique to automatically detect those feature interactions that are beneficial in terms of recommendation accuracy. The automatic feature interaction detection is achieved via edge prediction with an L0 activation regularization. Our proposed model is proved to be effective through the information bottleneck principle and statistical interaction theory. Experimental results show that our model (i) outperforms existing baselines in terms of accuracy, and (ii) automatically identifies beneficial feature interactions.

PDF Details

NeurIPS Conference 2021 Conference Paper

Distilling Object Detectors with Feature Richness

  • Du Zhixing
  • Rui Zhang
  • Ming Chang
  • Xishan Zhang
  • Shaoli Liu
  • Tianshi Chen
  • Yunji Chen

In recent years, large-scale deep models have achieved great success, but the huge computational complexity and massive storage requirements make it a great challenge to deploy them in resource-limited devices. As a model compression and acceleration method, knowledge distillation effectively improves the performance of small models by transferring the dark knowledge from the teacher detector. However, most of the existing distillation-based detection methods mainly imitating features near bounding boxes, which suffer from two limitations. First, they ignore the beneficial features outside the bounding boxes. Second, these methods imitate some features which are mistakenly regarded as the background by the teacher detector. To address the above issues, we propose a novel Feature-Richness Score (FRS) method to choose important features that improve generalized detectability during distilling. The proposed method effectively retrieves the important features outside the bounding boxes and removes the detrimental features within the bounding boxes. Extensive experiments show that our methods achieve excellent performance on both anchor-based and anchor-free detectors. For example, RetinaNet with ResNet-50 achieves 39. 7% in mAP on the COCO2017 dataset, which even surpasses the ResNet-101 based teacher detector 38. 9% by 0. 8%. Our implementation is available at https: //github. com/duzhixing/FRS.

PDF Details

IJCAI Conference 2021 Conference Paper

Hindsight Value Function for Variance Reduction in Stochastic Dynamic Environment

  • Jiaming Guo
  • Rui Zhang
  • Xishan Zhang
  • Shaohui Peng
  • Qi Yi
  • Zidong Du
  • Xing Hu
  • Qi Guo

Policy gradient methods are appealing in deep reinforcement learning but suffer from high variance of gradient estimate. To reduce the variance, the state value function is applied commonly. However, the effect of the state value function becomes limited in stochastic dynamic environments, where the unexpected state dynamics and rewards will increase the variance. In this paper, we propose to replace the state value function with a novel hindsight value function, which leverages the information from the future to reduce the variance of the gradient estimate for stochastic dynamic environments. Particularly, to obtain an ideally unbiased gradient estimate, we propose an information-theoretic approach, which optimizes the embeddings of the future to be independent of previous actions. In our experiments, we apply the proposed hindsight value function in stochastic dynamic environments, including discrete-action environments and continuous-action environments. Compared with the standard state value function, the proposed hindsight value function consistently reduces the variance, stabilizes the training, and improves the eventual policy.

PDF Details DOI

AAAI Conference 2021 Short Paper

Melodic Phrase Attention Network for Symbolic Data-based Music Genre Classification (Student Abstract)

  • Li Li
  • Rui Zhang
  • Zhenyu Wang

Compared with audio data-based music genre classification, researches on symbolic data-based music are scarce. Existing methods generally utilize manually extracted features, which is very time-consuming and laborious, and use traditional classifiers for label prediction without considering specific music features. To tackle this issue, we propose the Melodic Phrase Attention Network (MPAN) for symbolic data-based music genre classification. Our model is trained in three steps: First, we adopt representation learning, instead of the traditional musical feature extraction method, to obtain a vectorized representation of the music pieces. Second, the music pieces are divided into several melodic phrases through melody segmentation. Finally, the Melodic Phrase Attention Network is designed according to music characteristics, to identify the reflection of each melodic phrase on the music genre, thereby generating more accurate predictions. Experimental results show that our proposed method is superior to baseline symbolic data-based music genre classification approaches, and has achieved significant performance improvements on two large datasets.

PDF Details

AIIM Journal 2021 Journal Article

Multiple instance convolutional neural network with modality-based attention and contextual multi-instance learning pooling layer for effective differentiation between borderline and malignant epithelial ovarian tumors

  • Junming Jian
  • Wei Xia
  • Rui Zhang
  • Xingyu Zhao
  • Jiayi Zhang
  • Xiaodong Wu
  • Yong'ai Li
  • Jinwei Qiang

Malignant epithelial ovarian tumors (MEOTs) are the most lethal gynecologic malignancies, accounting for 90% of ovarian cancer cases. By contrast, borderline epithelial ovarian tumors (BEOTs) have low malignant potential and are generally associated with a good prognosis. Accurate preoperative differentiation between BEOTs and MEOTs is crucial for determining the appropriate surgical strategies and improving the postoperative quality of life. Multimodal magnetic resonance imaging (MRI) is an essential diagnostic tool. Although state-of-the-art artificial intelligence technologies such as convolutional neural networks can be used for automated diagnoses, their application have been limited owing to their high demand for graphics processing unit memory and hardware resources when dealing with large 3D volumetric data. In this study, we used multimodal MRI with a multiple instance learning (MIL) method to differentiate between BEOT and MEOT. We proposed the use of MAC-Net, a multiple instance convolutional neural network (MICNN) with modality-based attention (MA) and contextual MIL pooling layer (C-MPL). The MA module can learn from the decision-making patterns of clinicians to automatically perceive the importance of different MRI modalities and achieve multimodal MRI feature fusion based on their importance. The C-MPL module uses strong prior knowledge of tumor distribution as an important reference and assesses contextual information between adjacent images, thus achieving a more accurate prediction. The performance of MAC-Net is superior, with an area under the receiver operating characteristic curve of 0. 878, surpassing that of several known MICNN approaches. Therefore, it can be used to assist clinical differentiation between BEOTs and MEOTs.

Details DOI

JAIR Journal 2021 Journal Article

NLP Methods for Extraction of Symptoms from Unstructured Data for Use in Prognostic COVID-19 Analytic Models

  • Greg M. Silverman
  • Himanshu S. Sahoo
  • Nicholas E. Ingraham
  • Monica Lupei
  • Michael A. Puskarich
  • Michael Usher
  • James Dries
  • Raymond L. Finzel

Statistical modeling of outcomes based on a patient's presenting symptoms (symptomatology) can help deliver high quality care and allocate essential resources, which is especially important during the COVID-19 pandemic. Patient symptoms are typically found in unstructured notes, and thus not readily available for clinical decision making. In an attempt to fill this gap, this study compared two methods for symptom extraction from Emergency Department (ED) admission notes. Both methods utilized a lexicon derived by expanding The Center for Disease Control and Prevention's (CDC) Symptoms of Coronavirus list. The first method utilized a word2vec model to expand the lexicon using a dictionary mapping to the Uni ed Medical Language System (UMLS). The second method utilized the expanded lexicon as a rule-based gazetteer and the UMLS. These methods were evaluated against a manually annotated reference (f1-score of 0.87 for UMLS-based ensemble; and 0.85 for rule-based gazetteer with UMLS). Through analyses of associations of extracted symptoms used as features against various outcomes, salient risks among the population of COVID-19 patients, including increased risk of in-hospital mortality (OR 1.85, p-value < 0.001), were identified for patients presenting with dyspnea. Disparities between English and non-English speaking patients were also identified, the most salient being a concerning finding of opposing risk signals between fatigue and in-hospital mortality (non-English: OR 1.95, p-value = 0.02; English: OR 0.63, p-value = 0.01). While use of symptomatology for modeling of outcomes is not unique, unlike previous studies this study showed that models built using symptoms with the outcome of in-hospital mortality were not significantly different from models using data collected during an in-patient encounter (AUC of 0.9 with 95% CI of [0.88, 0.91] using only vital signs; AUC of 0.87 with 95% CI of [0.85, 0.88] using only symptoms). These findings indicate that prognostic models based on symptomatology could aid in extending COVID-19 patient care through telemedicine, replacing the need for in-person options. The methods presented in this study have potential for use in development of symptomatology-based models for other diseases, including for the study of Post-Acute Sequelae of COVID-19 (PASC).

PDF Details DOI

IJCAI Conference 2021 Conference Paper

Noise Doesn't Lie: Towards Universal Detection of Deep Inpainting

  • Ang Li
  • Qiuhong Ke
  • Xingjun Ma
  • Haiqin Weng
  • Zhiyuan Zong
  • Feng Xue
  • Rui Zhang

Deep image inpainting aims to restore damaged or missing regions in an image with realistic contents. While having a wide range of applications such as object removal and image recovery, deep inpainting techniques also have the risk of being manipulated for image forgery. A promising countermeasure against such forgeries is deep inpainting detection, which aims to locate the inpainted regions in an image. In this paper, we make the first attempt towards universal detection of deep inpainting, where the detection network can generalize well when detecting different deep inpainting methods. To this end, we first propose a novel data generation approach to generate a universal training dataset, which imitates the noise discrepancies exist in real versus inpainted image contents to train universal detectors. We then design a Noise-Image Cross-fusion Network (NIX-Net) to effectively exploit the discriminative information contained in both the images and their noise patterns. We empirically show, on multiple benchmark datasets, that our approach outperforms existing detection methods by a large margin and generalize well to unseen deep inpainting techniques. Our universal training dataset can also significantly boost the generalizability of existing detection methods.

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

Structured in Space, Randomized in Time: Leveraging Dropout in RNNs for Efficient Training

  • Anup Sarma
  • Sonali Singh
  • Huaipan Jiang
  • Rui Zhang
  • Mahmut Kandemir
  • Chita Das

Recurrent Neural Networks (RNNs), more specifically their Long Short-Term Memory (LSTM) variants, have been widely used as a deep learning tool for tackling sequence-based learning tasks in text and speech. Training of such LSTM applications is computationally intensive due to the recurrent nature of hidden state computation that repeats for each time step. While sparsity in Deep Neural Nets has been widely seen as an opportunity for reducing computation time in both training and inference phases, the usage of non-ReLU activation in LSTM RNNs renders the opportunities for such dynamic sparsity associated with neuron activation and gradient values to be limited or non-existent. In this work, we identify dropout induced sparsity for LSTMs as a suitable mode of computation reduction. Dropout is a widely used regularization mechanism, which randomly drops computed neuron values during each iteration of training. We propose to structure dropout patterns, by dropping out the same set of physical neurons within a batch, resulting in column (row) level hidden state sparsity, which are well amenable to computation reduction at run-time in general-purpose SIMD hardware as well as systolic arrays. We provide a detailed analysis of how the dropout-induced sparsity propagates through the different stages of network training and how it can be leveraged in each stage. More importantly, our proposed approach works as a direct replacement for existing dropout-based application settings. We conduct our experiments for three representative NLP tasks: language modelling on the PTB dataset, OpenNMT based machine translation using the IWSLT De-En and En-Vi datasets, and named entity recognition sequence labelling using the CoNLL-2003 shared task. We demonstrate that our proposed approach can be used to translate dropout-based computation reduction into reduced training time, with improvement ranging from 1. 23$\times$ to 1. 64$\times$, without sacrificing the target metric.

PDF Details

IJCAI Conference 2021 Conference Paper

UniMF: A Unified Framework to Incorporate Multimodal Knowledge Bases intoEnd-to-End Task-Oriented Dialogue Systems

  • Shiquan Yang
  • Rui Zhang
  • Sarah M. Erfani
  • Jey Han Lau

Knowledge bases (KBs) are usually essential for building practical dialogue systems. Recently we have seen rapidly growing interest in integrating knowledge bases into dialogue systems. However, existing approaches mostly deal with knowledge bases of a single modality, typically textual information. As today's knowledge bases become abundant with multimodal information such as images, audios and videos, the limitation of existing approaches greatly hinders the development of dialogue systems. In this paper, we focus on task-oriented dialogue systems and address this limitation by proposing a novel model that integrates external multimodal KB reasoning with pre-trained language models. We further enhance the model via a novel multi-granularity fusion mechanism to capture multi-grained semantics in the dialogue history. To validate the effectiveness of the proposed model, we collect a new large-scale (14K) dialogue dataset MMDialKB, built upon multimodal KB. Both automatic and human evaluation results on MMDialKB demonstrate the superiority of our proposed framework over strong baselines.

PDF Details DOI

AAAI Conference 2020 Conference Paper

DWM: A Decomposable Winograd Method for Convolution Acceleration

  • Di Huang
  • Xishan Zhang
  • Rui Zhang
  • Tian Zhi
  • Deyuan He
  • Jiaming Guo
  • Chang Liu
  • Qi Guo

Winograd’s minimal filtering algorithm has been widely used in Convolutional Neural Networks (CNNs) to reduce the number of multiplications for faster processing. However, it is only effective on convolutions with kernel size as 3x3 and stride as 1, because it suffers from significantly increased FLOPs and numerical accuracy problem for kernel size larger than 3x3 and fails on convolution with stride larger than 1. In this paper, we propose a novel Decomposable Winograd Method (DWM), which breaks through the limitation of original Winograd’s minimal filtering algorithm to a wide and general convolutions. DWM decomposes kernels with large size or large stride to several small kernels with stride as 1 for further applying Winograd method, so that DWM can reduce the number of multiplications while keeping the numerical accuracy. It enables the fast exploring of larger kernel size and larger stride value in CNNs for high performance and accuracy and even the potential for new CNNs. Comparing against the original Winograd, the proposed DWM is able to support all kinds of convolutions with a speedup of ∼2, without affecting the numerical accuracy.

PDF Details

AAAI Conference 2020 Conference Paper

Dynamic Reward-Based Dueling Deep Dyna-Q: Robust Policy Learning in Noisy Environments

  • Yangyang Zhao
  • Zhenyu Wang
  • Kai Yin
  • Rui Zhang
  • Zhenhua Huang
  • Pei Wang

Task-oriented dialogue systems provide a convenient interface to help users complete tasks. An important consideration for task-oriented dialogue systems is the ability to against the noise commonly existed in the real-world conversation. Both rule-based strategies and statistical modeling techniques can solve noise problems, but they are costly. In this paper, we propose a new approach, called Dynamic Reward-based Dueling Deep Dyna-Q (DR-D3Q). The DR-D3Q can learn policies in noise robustly, and it is easy to implement by combining dynamic reward and the Dueling Deep Q-Network (Dueling DQN) into Deep Dyna-Q (DDQ) framework. The Dueling DQN can mitigate the negative impact of noise on learning policies, but it is inapplicable to dialogue domain due to different reward mechanisms. Unlike typical dialogue reward function, we integrate dynamic reward that provides reward in real-time for agent to make Dueling DQN adapt to dialogue domain. For the purpose of supplementing the limited amount of real user experiences, we take the DDQ framework as the basic framework. Experiments using simulation and human evaluation show that the DR-D3Q significantly improve the performance of policy learning tasks in noisy environments. 1

PDF Details

AAAI Conference 2020 Conference Paper

MALA: Cross-Domain Dialogue Generation with Action Learning

  • Xinting Huang
  • Jianzhong Qi
  • Yu Sun
  • Rui Zhang

Response generation for task-oriented dialogues involves two basic components: dialogue planning and surface realization. These two components, however, have a discrepancy in their objectives, i. e. , task completion and language quality. To deal with such discrepancy, conditioned response generation has been introduced where the generation process is factorized into action decision and language generation via explicit action representations. To obtain action representations, recent studies learn latent actions in an unsupervised manner based on the utterance lexical similarity. Such an action learning approach is prone to diversities of language surfaces, which may impinge task completion and language quality. To address this issue, we propose multi-stage adaptive latent action learning (MALA) that learns semantic latent actions by distinguishing the effects of utterances on dialogue progress. We model the utterance effect using the transition of dialogue states caused by the utterance and develop a semantic similarity measurement that estimates whether utterances have similar effects. For learning semantic actions on domains without dialogue states, MALA extends the semantic similarity measurement across domains progressively, i. e. , from aligning shared actions to learning domain-specific actions. Experiments using multi-domain datasets, SMD and MultiWOZ, show that our proposed model achieves consistent improvements over the baselines models in terms of both task completion and language quality.

PDF Details

NeurIPS Conference 2020 Conference Paper

Quantile Propagation for Wasserstein-Approximate Gaussian Processes

  • Rui Zhang
  • Christian Walder
  • Edwin V. Bonilla
  • Marian-Andrei Rizoiu
  • Lexing Xie

Approximate inference techniques are the cornerstone of probabilistic methods based on Gaussian process priors. Despite this, most work approximately optimizes standard divergence measures such as the Kullback-Leibler (KL) divergence, which lack the basic desiderata for the task at hand, while chiefly offering merely technical convenience. We develop a new approximate inference method for Gaussian process models which overcomes the technical challenges arising from abandoning these convenient divergences. Our method---dubbed Quantile Propagation (QP)---is similar to expectation propagation (EP) but minimizes the $L_2$ Wasserstein distance (WD) instead of the KL divergence. The WD exhibits all the required properties of a distance metric, while respecting the geometry of the underlying sample space. We show that QP matches quantile functions rather than moments as in EP and has the same mean update but a smaller variance update than EP, thereby alleviating EP's tendency to over-estimate posterior variances. Crucially, despite the significant complexity of dealing with the WD, QP has the same favorable locality property as EP, and thereby admits an efficient algorithm. Experiments on classification and Poisson regression show that QP outperforms both EP and variational Bayes.

PDF Details

AAAI Conference 2020 Conference Paper

Sentence Generation for Entity Description with Content-Plan Attention

  • Bayu Trisedya
  • Jianzhong Qi
  • Rui Zhang

We study neural data-to-text generation. Specifically, we consider a target entity that is associated with a set of attributes. We aim to generate a sentence to describe the target entity. Previous studies use encoder-decoder frameworks where the encoder treats the input as a linear sequence and uses LSTM to encode the sequence. However, linearizing a set of attributes may not yield the proper order of the attributes, and hence leads the encoder to produce an improper context to generate a description. To handle disordered input, recent studies propose two-stage neural models that use pointer networks to generate a content-plan (i. e. , content-planner) and use the content-plan as input for an encoder-decoder model (i. e. , text generator). However, in two-stage models, the content-planner may yield an incomplete content-plan, due to missing one or more salient attributes in the generated content-plan. This will in turn cause the text generator to generate an incomplete description. To address these problems, we propose a novel attention model that exploits contentplan to highlight salient attributes in a proper order. The challenge of integrating a content-plan in the attention model of an encoder-decoder framework is to align the content-plan and the generated description. We handle this problem by devising a coverage mechanism to track the extent to which the content-plan is exposed in the previous decoding time-step, and hence it helps our proposed attention model select the attributes to be mentioned in the description in a proper order. Experimental results show that our model outperforms stateof-the-art baselines by up to 3% and 5% in terms of BLEU score on two real-world datasets, respectively.

PDF Details

AAAI Conference 2020 Conference Paper

Variational Inference for Sparse Gaussian Process Modulated Hawkes Process

  • Rui Zhang
  • Christian Walder
  • Marian-Andrei Rizoiu

The Hawkes process (HP) has been widely applied to modeling self-exciting events including neuron spikes, earthquakes and tweets. To avoid designing parametric triggering kernel and to be able to quantify the prediction confidence, the nonparametric Bayesian HP has been proposed. However, the inference of such models suffers from unscalability or slow convergence. In this paper, we aim to solve both problems. Specifically, first, we propose a new non-parametric Bayesian HP in which the triggering kernel is modeled as a squared sparse Gaussian process. Then, we propose a novel variational inference schema for model optimization. We employ the branching structure of the HP so that maximization of evidence lower bound (ELBO) is tractable by the expectationmaximization algorithm. We propose a tighter ELBO which improves the fitting performance. Further, we accelerate the novel variational inference schema to linear time complexity by leveraging the stationarity of the triggering kernel. Different from prior acceleration methods, ours enjoys higher efficiency. Finally, we exploit synthetic data and two large social media datasets to evaluate our method. We show that our approach outperforms state-of-the-art non-parametric frequentist and Bayesian methods. We validate the efficiency of our accelerated variational inference schema and practical utility of our tighter ELBO for model selection. We observe that the tighter ELBO exceeds the common one in model selection.

PDF Details

YNIMG Journal 2019 Journal Article

Brain default-mode network dysfunction in addiction

  • Rui Zhang
  • Nora D. Volkow

Aberrant patterns of brain functional connectivity in the default mode network (DMN) have been observed across different classes of substance use disorder (SUD) and are associated with craving and relapse. In addicted individuals resting functional connectivity (RSFC) of the anterior DMN, which participates in attribution of personal value and emotional regulation, tends to be decreased, whereas RSFC of the posterior DMN, which directs attention to the internal world, tends to be increased. Aberrant RSFC within the DMN is believed to contribute to impaired self-awareness, negative emotions and to ruminations in addiction. Additionally, the disrupted connectivity between DMN and cortical regions involved with executive function, memory and emotion could be critical to drug-taking regardless of negative consequences and to stress-triggered relapse. At the system level, the dynamics of DMN interactions with the executive control and the salience networks are also disturbed in addiction. The DMN is prominently engaged during the withdrawal and preoccupation phases of the addiction cycle at the expense of the executive control network and with an enhanced participation of the salience network. In contrast, DMN prominence appears to be transitorily decreased during the intoxication phases. There is also growing evidence that disruption of the DMN in addiction reflects in part changes in dopaminergic, glutamatergic, and GABAergic signaling associated with acute and chronic drug use. Findings are starting to reveal DMN RSFC as a potential biomarker for predicting clinical outcomes in SUD and identify the DMN as a promising target for the treatment of addiction.

Details DOI

IJCAI Conference 2019 Conference Paper

Efficient Non-parametric Bayesian Hawkes Processes

  • Rui Zhang
  • Christian Walder
  • Marian-Andrei Rizoiu
  • Lexing Xie

In this paper, we develop an efficient non-parametric Bayesian estimation of the kernel function of Hawkes processes. The non-parametric Bayesian approach is important because it provides flexible Hawkes kernels and quantifies their uncertainty. Our method is based on the cluster representation of Hawkes processes. Utilizing the stationarity of the Hawkes process, we efficiently sample random branching structures and thus, we split the Hawkes process into clusters of Poisson processes. We derive two algorithms --- a block Gibbs sampler and a maximum a posteriori estimator based on expectation maximization --- and we show that our methods have a linear time complexity, both theoretically and empirically. On synthetic data, we show our methods to be able to infer flexible Hawkes triggering kernels. On two large-scale Twitter diffusion datasets, we show that our methods outperform the current state-of-the-art in goodness-of-fit and that the time complexity is linear in the size of the dataset. We also observe that on diffusions related to online videos, the learned kernels reflect the perceived longevity for different content types such as music or pets videos.

PDF Details

AAAI Conference 2019 Conference Paper

Entity Alignment between Knowledge Graphs Using Attribute Embeddings

  • Bayu Distiawan Trisedya
  • Jianzhong Qi
  • Rui Zhang

The task of entity alignment between knowledge graphs aims to find entities in two knowledge graphs that represent the same real-world entity. Recently, embedding-based models are proposed for this task. Such models are built on top of a knowledge graph embedding model that learns entity embeddings to capture the semantic similarity between entities in the same knowledge graph. We propose to learn embeddings that can capture the similarity between entities in different knowledge graphs. Our proposed model helps align entities from different knowledge graphs, and hence enables the integration of multiple knowledge graphs. Our model exploits large numbers of attribute triples existing in the knowledge graphs and generates attribute character embeddings. The attribute character embedding shifts the entity embeddings from two knowledge graphs into the same space by computing the similarity between entities based on their attributes. We use a transitivity rule to further enrich the number of attributes of an entity to enhance the attribute character embedding. Experiments using real-world knowledge bases show that our proposed model achieves consistent improvements over the baseline models by over 50% in terms of hits@1 on the entity alignment task.

PDF Details

IJCAI Conference 2019 Conference Paper

Generative Image Inpainting with Submanifold Alignment

  • Ang Li
  • Jianzhong Qi
  • Rui Zhang
  • Xingjun Ma
  • Kotagiri Ramamohanarao

Image inpainting aims at restoring missing regions of corrupted images, which has many applications such as image restoration and object removal. However, current GAN-based generative inpainting models do not explicitly exploit the structural or textural consistency between restored contents and their surrounding contexts. To address this limitation, we propose to enforce the alignment (or closeness) between the local data submanifolds (subspaces) around restored images and those around the original (uncorrupted) images during the learning process of GAN-based inpainting models. We exploit Local Intrinsic Dimensionality (LID) to measure, in deep feature space, the alignment between data submanifolds learned by a GAN model and those of the original data, from a perspective of both images (denoted as iLID) and local patches (denoted as pLID) of images. We then apply iLID and pLID as regularizations for GAN-based inpainting models to encourage two different levels of submanifold alignments: 1) an image-level alignment to improve structural consistency, and 2) a patch-level alignment to improve textural details. Experimental results on four benchmark datasets show that our proposed model can generate more accurate results than state-of-the-art models.

PDF Details

NeurIPS Conference 2019 Conference Paper

Robust Principal Component Analysis with Adaptive Neighbors

  • Rui Zhang
  • Hanghang Tong

Suppose certain data points are overly contaminated, then the existing principal component analysis (PCA) methods are frequently incapable of filtering out and eliminating the excessively polluted ones, which potentially lead to the functional degeneration of the corresponding models. To tackle the issue, we propose a general framework namely robust weight learning with adaptive neighbors (RWL-AN), via which adaptive weight vector is automatically obtained with both robustness and sparse neighbors. More significantly, the degree of the sparsity is steerable such that only exact k well-fitting samples with least reconstruction errors are activated during the optimization, while the residual samples, i. e. , the extreme noised ones are eliminated for the global robustness. Additionally, the framework is further applied to PCA problem to demonstrate the superiority and effectiveness of the proposed RWL-AN model.

PDF Details

AAAI Conference 2019 Conference Paper

ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks

  • Michihiro Yasunaga
  • Jungo Kasai
  • Rui Zhang
  • Alexander R. Fabbri
  • Irene Li
  • Dan Friedman
  • Dragomir R. Radev

Scientific documents rely on both mathematics and text to communicate ideas. Inspired by the topical correspondence between mathematical equations and word contexts observed in scientific texts, we propose a novel topic model that jointly generates mathematical equations and their surrounding text (TopicEq). Using an extension of the correlated topic model, the context is generated from a mixture of latent topics, and the equation is generated by an RNN that depends on the latent topic activations. To experiment with this model, we create a corpus of 400K equation-context pairs extracted from a range of scientific articles from arXiv, and fit the model using a variational autoencoder approach. Experimental results show that this joint model significantly outperforms existing topic models and equation models for scientific texts. Moreover, we qualitatively show that the model effectively captures the relationship between topics and mathematics, enabling novel applications such as topic-aware equation generation, equation topic inference, and topic-aware alignment of mathematical symbols and words.

PDF Details

AAAI Conference 2018 Conference Paper

Addressee and Response Selection in Multi-Party Conversations With Speaker Interaction RNNs

  • Rui Zhang
  • Honglak Lee
  • Lazaros Polymenakos
  • Dragomir Radev

In this paper, we study the problem of addressee and response selection in multi-party conversations. Understanding multi-party conversations is challenging because of complex speaker interactions: multiple speakers exchange messages with each other, playing different roles (sender, addressee, observer), and these roles vary across turns. To tackle this challenge, we propose the Speaker Interaction Recurrent Neural Network (SI-RNN). Whereas the previous stateof-the-art system updated speaker embeddings only for the sender, SI-RNN uses a novel dialog encoder to update speaker embeddings in a role-sensitive way. Additionally, unlike the previous work that selected the addressee and response separately, SI-RNN selects them jointly by viewing the task as a sequence prediction problem. Experimental results show that SI-RNN significantly improves the accuracy of addressee and response selection, particularly in complex conversations with many speakers and responses to distant messages many turns in the past.

PDF Details

IJCAI Conference 2018 Conference Paper

Collaborative Learning for Weakly Supervised Object Detection

  • Jiajie Wang
  • Jiangchao Yao
  • Ya Zhang
  • Rui Zhang

Weakly supervised object detection has recently received much attention, since it only requires image-level labels instead of the bounding-box labels consumed in strongly supervised learning. Nevertheless, the save in labeling expense is usually at the cost of model accuracy. In this paper, we propose a simple but effective weakly supervised collaborative learning framework to resolve this problem, which trains a weakly supervised learner and a strongly supervised learner jointly by enforcing partial feature sharing and prediction consistency. For object detection, taking WSDDN-like architecture as weakly supervised detector sub-network and Faster-RCNN-like architecture as strongly supervised detector sub-network, we propose an end-to-end Weakly Supervised Collaborative Detection Network. As there is no strong supervision available to train the Faster-RCNN-like sub-network, a new prediction consistency loss is defined to enforce consistency of predictions between the two sub-networks as well as within the Faster-RCNN-like sub-networks. At the same time, the two detectors are designed to partially share features to further guarantee the model consistency at perceptual level. Extensive experiments on PASCAL VOC 2007 and 2012 data sets have demonstrated the effectiveness of the proposed framework.

PDF Details

YNIMG Journal 2018 Journal Article

Effects of resveratrol on memory performance, hippocampus connectivity and microstructure in older adults – A randomized controlled trial

  • Sebastian Huhn
  • Frauke Beyer
  • Rui Zhang
  • Leonie Lampe
  • Jana Grothe
  • Jürgen Kratzsch
  • Anja Willenberg
  • Jana Breitfeld

Introduction The polyphenol resveratrol has been suggested to exert beneficial effects on memory and the aging hippocampus due to calorie-restriction mimicking effects. However, the evidence based on human interventional studies is scarce. We therefore aimed to determine the effects of resveratrol on memory performance, and to identify potential underlying mechanisms using a broad array of blood-based biomarkers as well as hippocampus connectivity and microstructure assessed with ultra-high field magnetic resonance imaging (UHF-MRI). Methods In this double-blind, randomized controlled trial, 60 elderly participants (60–79 years) with a wide body-mass index (BMI) range of 21–37 kg/m2 were randomized to receive either resveratrol (200 mg/day) or placebo for 26 weeks (registered at ClinicalTrials. gov: NCT02621554). Baseline and follow-up assessments included the California Verbal Learning Task (CVLT, main outcome), the ModBent task, anthropometry, markers of glucose and lipid metabolism, inflammation and neurotrophins derived from fasting blood, multimodal neuroimaging at 3 and 7 T, and questionnaires to assess confounding factors. Results Multivariate repeated-measures ANOVA did not detect significant time by group effects for CVLT performance. There was a trend for preserved pattern recognition memory after resveratrol, while performance decreased in the placebo group (n. s. , p = 0. 07). Further exploratory analyses showed increases in both groups over time in body fat, cholesterol, fasting glucose, interleukin 6, high sensitive C-reactive protein, tumor necrosis factor alpha and in mean diffusivity of the subiculum and presubiculum, as well as decreases in physical activity, brain-derived neurotrophic factor and insulin-like growth factor 1 at follow-up, which were partly more pronounced after resveratrol. Discussion This interventional study failed to show significant improvements in verbal memory after 6 months of resveratrol in healthy elderly with a wide BMI range. A non-significant trend emerged for positive effects on pattern recognition memory, while possible confounding effects of unfavorable changes in lifestyle behavior, neurotrophins and inflammatory markers occurred. Our findings also indicate the feasibility to detect (un)healthy aging-related changes in measures of hippocampus microstructure after 6 months using 7T diffusion MRI. More studies incorporating a longer duration and larger sample size are needed to determine if resveratrol enhances memory performance in healthy older adults.

Details DOI

IJCAI Conference 2018 Conference Paper

High Resolution Feature Recovering for Accelerating Urban Scene Parsing

  • Rui Zhang
  • Sheng Tang
  • Luoqi Liu
  • Yongdong Zhang
  • Jintao Li
  • Shuicheng Yan

Both accuracy and speed are equally important in urban scene parsing. Most of the existing methods mainly focus on improving parsing accuracy, ignoring the problem of low inference speed due to large-sized input and high resolution feature maps. To tackle this issue, we propose a High Resolution Feature Recovering (HRFR) framework to accelerate a given parsing network. A Super-Resolution Recovering module is employed to recover features of large original-sized images from features of down-sampled input. Therefore, our framework can combine the advantages of (1) fast speed of networks with down-sampled input and (2) high accuracy of networks with large original-sized input. Additionally, we employ auxiliary intermediate supervision and boundary region re-weighting to facilitate the optimization of the network. Extensive experiments on the two challenging Cityscapes and CamVid datasets well demonstrate the effectiveness of the proposed HRFR framework, which can accelerate the scene parsing inference process by about 3. 0x speedup from 1/2 down-sampled input with negligible accuracy reduction.

PDF Details

NeurIPS Conference 2018 Conference Paper

KDGAN: Knowledge Distillation with Generative Adversarial Networks

  • Xiaojie Wang
  • Rui Zhang
  • Yu Sun
  • Jianzhong Qi

Knowledge distillation (KD) aims to train a lightweight classifier suitable to provide accurate inference with constrained resources in multi-label learning. Instead of directly consuming feature-label pairs, the classifier is trained by a teacher, i. e. , a high-capacity model whose training may be resource-hungry. The accuracy of the classifier trained this way is usually suboptimal because it is difficult to learn the true data distribution from the teacher. An alternative method is to adversarially train the classifier against a discriminator in a two-player game akin to generative adversarial networks (GAN), which can ensure the classifier to learn the true data distribution at the equilibrium of this game. However, it may take excessively long time for such a two-player game to reach equilibrium due to high-variance gradient updates. To address these limitations, we propose a three-player game named KDGAN consisting of a classifier, a teacher, and a discriminator. The classifier and the teacher learn from each other via distillation losses and are adversarially trained against the discriminator via adversarial losses. By simultaneously optimizing the distillation and adversarial losses, the classifier will learn the true data distribution at the equilibrium. We approximate the discrete distribution learned by the classifier (or the teacher) with a concrete distribution. From the concrete distribution, we generate continuous samples to obtain low-variance gradient updates, which speed up the training. Extensive experiments using real datasets confirm the superiority of KDGAN in both accuracy and training speed.

PDF Details

AAAI Conference 2018 Conference Paper

Learning Datum-Wise Sampling Frequency for Energy-Efficient Human Activity Recognition

  • Weihao Cheng
  • Sarah Erfani
  • Rui Zhang
  • Ramamohanarao Kotagiri

Continuous Human Activity Recognition (HAR) is an important application of smart mobile/wearable systems for providing dynamic assistance to users. However, HAR in real-time requires continuous sampling of data using built-in sensors (e. g. , accelerometer), which significantly increases the energy cost and shortens the operating span. Reducing sampling rate can save energy but causes low recognition accuracy. Therefore, choosing adaptive sampling frequency that balances accuracy and energy efficiency becomes a critical problem in HAR. In this paper, we formalize the problem as minimizing both classification error and energy cost by choosing dynamically appropriate sampling rates. We propose Datum- Wise Frequency Selection (DWFS) to solve the problem via a continuous state Markov Decision Process (MDP). A policy function is learned from the MDP, which selects the best frequency for sampling an incoming data entity by exploiting a datum related state of the system. We propose a method for alternative learning the parameters of an activity classification model and the MDP that improves both the accuracy and the energy efficiency. We evaluate DWFS with three real-world HAR datasets, and the results show that DWFS statistically outperforms the state-of-the-arts regarding a combined measurement of accuracy and energy efficiency.

PDF Details

JBHI Journal 2018 Journal Article

Monitoring Chewing and Eating in Free-Living Using Smart Eyeglasses

  • Rui Zhang
  • Oliver Amft

We propose to 3-D-print personal fitted regular-look smart eyeglasses frames equipped with bilateral electromyography recording to monitor temporalis muscles' activity for automatic dietary monitoring. Personal fitting supported electrode-skin contacts are at temple ear bend and temple end positions. We evaluated the smart monitoring eyeglasses during in-lab and free-living studies of food chewing and eating event detection with ten participants. The in-lab study was designed to explore three natural food hardness levels and determine parameters of an energy-based chewing cycle detection. Our free-living study investigated whether chewing monitoring and eating event detection using smart eyeglasses is feasible in free-living. An eating event detection algorithm was developed to determine intake activities based on the estimated chewing rate. Results showed an average food hardness classification accuracy of 94% and chewing cycle detection precision and recall above 90% for the in-lab study and above 77% for the free-living study covering 122 hours of recordings. Eating detection revealed the 44 eating events with an average accuracy above 95%. We conclude that smart eyeglasses are suitable for monitoring chewing and eating events in free-living and even could provide further insights into the wearer's natural chewing patterns.

Details DOI

IJCAI Conference 2018 Conference Paper

Predicting Complex Activities from Ongoing Multivariate Time Series

  • Weihao Cheng
  • Sarah Erfani
  • Rui Zhang
  • Ramamohanarao Kotagiri

The rapid development of sensor networks enables recognition of complex activities (CAs) using multivariate time series. However, CAs are usually performed over long periods of time, which causes slow recognition by models based on fully observed data. Therefore, predicting CAs at early stages becomes an important problem. In this paper, we propose Simultaneous Complex Activities Recognition and Action Sequence Discovering (SimRAD), an algorithm which predicts a CA over time by mining a sequence of multivariate actions from sensor data using a Deep Neural Network. SimRAD simultaneously learns two probabilistic models for inferring CAs and action sequences, where the estimations of the two models are conditionally dependent on each other. SimRAD continuously predicts the CA and the action sequence, thus the predictions are mutually updated until the end of the CA. We conduct evaluations on a real-world CA dataset consisting of a rich amount of sensor data, and the results show that SimRAD outperforms state-of-the-art methods by average 7. 2% in prediction accuracy with high confidence.

PDF Details

YNIMG Journal 2018 Journal Article

White matter microstructural variability mediates the relation between obesity and cognition in healthy adults

  • Rui Zhang
  • Frauke Beyer
  • Leonie Lampe
  • Tobias Luck
  • Steffi G. Riedel-Heller
  • Markus Loeffler
  • Matthias L. Schroeter
  • Michael Stumvoll

Obesity has been linked with structural and functional brain changes. However, the impact of obesity on brain and cognition in aging remains debatable, especially for white matter. We therefore aimed to determine the effects of obesity on white matter microstructure and potential implications for cognition in a well-characterized large cohort of healthy adults. In total, 1255 participants (50% females, 19–80 years, BMI 16. 8–50. 2 kg/m2) with diffusion-weighted magnetic resonance imaging at 3T were analysed. Tract-based spatial statistics (TBSS) probed whether body mass index (BMI) and waist-to-hip ratio (WHR) were related to fractional anisotropy (FA). We conducted partial correlations and mediation analyses to explore whether obesity or regional FA were related to cognitive performance. Analyses were adjusted for demographic, genetic, and obesity-associated confounders. Results showed that higher BMI and higher WHR were associated with lower FA in multiple white matter tracts (p < 0. 05, FWE-corrected). Mediation analyses provided evidence for indirect negative effects of higher BMI and higher WHR on executive functions and processing speed through lower FA in fiber tracts connecting (pre)frontal, visual, and associative areas (indirect paths, |ß| ≥ 0. 01; 99% |CI| > 0). This large cross-sectional study showed that obesity is correlated with lower FA in multiple white matter tracts in otherwise healthy adults, independent of confounders. Moreover, although effect sizes were small, mediation results indicated that visceral obesity was linked to poorer executive functions and lower processing speed through lower FA in callosal and associative fiber tracts. Longitudinal studies are needed to support this hypothesis.

Details DOI

IJCAI Conference 2017 Conference Paper

Global-residual and Local-boundary Refinement Networks for Rectifying Scene Parsing Predictions

  • Rui Zhang
  • Sheng Tang
  • Min Lin
  • Jintao Li
  • Shuicheng Yan

Most of existing scene parsing methods suffer from the serious problems of both inconsistent parsing results and object boundary shift. To tackle these problems, we first propose an iterative Global-residual Refinement Network (GRN) through exploiting global contextual information to predict the parsing residuals and iteratively smoothen the inconsistent parsing labels. Furthermore, we propose a Local-boundary Refinement Network (LRN) to learn the position-adaptive propagation coefficients so that local contextual information from neighbors can be optimally captured for refining object boundaries. Finally, we cascade the proposed two refinement networks after a fully residual convolutional neural network within a uniform framework. Extensive experiments on ADE20K and Cityscapes datasets well demonstrate the effectiveness of the two refinement methods for refining scene parsing predictions.

PDF Details

AAAI Conference 2017 Conference Paper

Improving Efficiency of SVM k -Fold Cross-Validation by Alpha Seeding

  • Zeyi Wen
  • Bin Li
  • Ramamohanarao Kotagiri
  • Jian Chen
  • Yawen Chen
  • Rui Zhang

The k-fold cross-validation is commonly used to evaluate the effectiveness of SVMs with the selected hyper-parameters. It is known that the SVM k-fold cross-validation is expensive, since it requires training k SVMs. However, little work has explored reusing the hth SVM for training the (h + 1)th SVM for improving the efficiency of k-fold cross-validation. In this paper, we propose three algorithms that reuse the hth SVM for improving the efficiency of training the (h + 1)th SVM. Our key idea is to efficiently identify the support vectors and to accurately estimate their associated weights (also called alpha values) of the next SVM by using the previous SVM. Our experimental results show that our algorithms are several times faster than the k-fold cross-validation which does not make use of the previously trained SVM. Moreover, our algorithms produce the same results (hence same accuracy) as the k-fold cross-validation which does not make use of the previously trained SVM.

PDF Details

AAAI Conference 2017 Conference Paper

Incorporating Knowledge Graph Embeddings into Topic Modeling

  • Liang Yao
  • Yin Zhang
  • Baogang Wei
  • Zhe Jin
  • Rui Zhang
  • Yangyang Zhang
  • Qinfei Chen

Probabilistic topic models could be used to extract lowdimension topics from document collections. However, such models without any human knowledge often produce topics that are not interpretable. In recent years, a number of knowledge-based topic models have been proposed, but they could not process fact-oriented triple knowledge in knowledge graphs. Knowledge graph embeddings, on the other hand, automatically capture relations between entities in knowledge graphs. In this paper, we propose a novel knowledge-based topic model by incorporating knowledge graph embeddings into topic modeling. By combining latent Dirichlet allocation, a widely used topic model with knowledge encoded by entity vectors, we improve the semantic coherence significantly and capture a better representation of a document in the topic space. Our evaluation results will demonstrate the effectiveness of our method.

PDF Details

YNICL Journal 2017 Journal Article

Neurophysiological mechanisms of circadian cognitive control in RLS patients - an EEG source localization study

  • Rui Zhang
  • Moritz D. Brandt
  • Wiebke Schrempf
  • Christian Beste
  • Ann-Kathrin Stock

The circadian variation of sensory and motor symptoms with increasing severity in the evening and at night is a key diagnostic feature/symptom of the restless legs syndrome (RLS). Even though many neurological diseases have shown a strong nexus between motor and cognitive symptoms, it has remained unclear whether cognitive performance of RLS patients declines in the evening and which neurophysiological mechanisms are affected by the circadian variation. In the current study, we examined daytime effects (morning vs. evening) on cognitive performance in RLS patients (n = 33) compared to healthy controls (n = 29) by analyzing flanker interference effects in combination with EEG and source localization techniques. RLS patients showed larger flanker interference effects in the evening than in the morning (p = .023), while healthy controls did not display a comparable circadian variation. In line with this, the neurophysiological data showed smaller N1 amplitudes in RLS patients compared to controls in the interfering task condition in the evening (p = .042), but not in the morning. The results demonstrate diurnal cognitive changes in RLS patients with intensified impairments in the evening. It seems that not all dopamine-regulated cognitive processes are altered in RLS and thus show daytime-dependent impairments. Instead, the daytime-related cognitive impairment emerges from attentional selection processes within the extra-striate visual cortex, but not from later cognitive processes such as conflict monitoring and response selection.

Details DOI

AAAI Conference 2016 Conference Paper

MOOCs Meet Measurement Theory: A Topic-Modelling Approach

  • Jiazhen He
  • Benjamin Rubinstein
  • James Bailey
  • Rui Zhang
  • Sandra Milligan
  • Jeffrey Chan

This paper adapts topic models to the psychometric testing of MOOC students based on their online forum postings. Measurement theory from education and psychology provides statistical models for quantifying a person’s attainment of intangible attributes such as attitudes, abilities or intelligence. Such models infer latent skill levels by relating them to individuals’ observed responses on a series of items such as quiz questions. The set of items can be used to measure a latent skill if individuals’ responses on them conform to a Guttman scale. Such well-scaled items differentiate between individuals and inferred levels span the entire range from most basic to the advanced. In practice, education researchers manually devise items (quiz questions) while optimising well-scaled conformance. Due to the costly nature and expert requirements of this process, psychometric testing has found limited use in everyday teaching. We aim to develop usable measurement models for highly-instrumented MOOC delivery platforms, by using participation in automatically-extracted online forum topics as items. The challenge is to formalise the Guttman scale educational constraint and incorporate it into topic models. To favour topics that automatically conform to a Guttman scale, we introduce a novel regularisation into non-negative matrix factorisation-based topic modelling. We demonstrate the suitability of our approach with both quantitative experiments on three Coursera MOOCs, and with a qualitative survey of topic interpretability on two MOOCs by domain expert interviews.

PDF Details

TIST Journal 2016 Journal Article

SMARTS

  • Kotagiri Ramamohanarao
  • Hairuo Xie
  • Lars Kulik
  • Shanika Karunasekera
  • Egemen Tanin
  • Rui Zhang
  • Eman Bin Khunayn

Microscopic traffic simulators are important tools for studying transportation systems as they describe the evolution of traffic to the highest level of detail. A major challenge to microscopic simulators is the slow simulation speed due to the complexity of traffic models. We have developed the Scalable Microscopic Adaptive Road Traffic Simulator (SMARTS), a distributed microscopic traffic simulator that can utilize multiple independent processes in parallel. SMARTS can perform fast large-scale simulations. For example, when simulating 1 million vehicles in an area the size of Melbourne, the system runs 1.14 times faster than real time with 30 computing nodes and 0.2s simulation timestep. SMARTS supports various driver models and traffic rules, such as the car-following model and lane-changing model, which can be driver dependent. It can simulate multiple vehicle types, including bus and tram. The simulator is equipped with a wide range of features that help to customize, calibrate, and monitor simulations. Simulations are accurate and confirm with real traffic behaviours. For example, it achieves 79.1% accuracy in predicting traffic on a 10km freeway 90 minutes into the future. The simulator can be used for predictive traffic advisories as well as traffic management decisions as simulations complete well ahead of real time. SMARTS can be easily deployed to different operating systems as it is developed with the standard Java libraries.

Details DOI

YNIMG Journal 2016 Journal Article

Structural and functional correlates of motor imagery BCI performance: Insights from the patterns of fronto-parietal attention network

  • Tao Zhang
  • Tiejun Liu
  • Fali Li
  • Mengchen Li
  • Dongbo Liu
  • Rui Zhang
  • Hui He
  • Peiyang Li

Motor imagery (MI)-based brain-computer interfaces (BCIs) have been widely used for rehabilitation of motor abilities and prosthesis control for patients with motor impairments. However, MI-BCI performance exhibits a wide variability across subjects, and the underlying neural mechanism remains unclear. Several studies have demonstrated that both the fronto-parietal attention network (FPAN) and MI are involved in high-level cognitive processes that are crucial for the control of BCIs. Therefore, we hypothesized that the FPAN may play an important role in MI-BCI performance. In our study, we recorded multi-modal datasets consisting of MI electroencephalography (EEG) signals, T1-weighted structural and resting-state functional MRI data for each subject. MI-BCI performance was evaluated using the common spatial pattern to extract the MI features from EEG signals. One cortical structural feature (cortical thickness (CT)) and two measurements (degree centrality (DC) and eigenvector centrality (EC)) of node centrality were derived from the structural and functional MRI data, respectively. Based on the information extracted from the EEG and MRI, a correlation analysis was used to elucidate the relationships between the FPAN and MI-BCI performance. Our results show that the DC of the right ventral intraparietal sulcus, the EC and CT of the left inferior parietal lobe, and the CT of the right dorsolateral prefrontal cortex were significantly associated with MI-BCI performance. Moreover, the receiver operating characteristic analysis and machine learning classification revealed that the EC and CT of the left IPL could effectively predict the low-aptitude BCI users from the high-aptitude BCI users with 83. 3% accuracy. Those findings consistently reveal that the individuals who have efficient FPAN would perform better on MI-BCI. Our findings may deepen the understanding of individual variability in MI-BCI performance, and also may provide a new biomarker to predict individual MI-BCI performance.

Details DOI

YNIMG Journal 2016 Journal Article

The neurophysiological basis of reward effects on backward inhibition processes

  • Rui Zhang
  • Ann-Kathrin Stock
  • Christian Beste

The ability to flexibly switch between tasks is an important faculty in daily life. One process that has been suggested to be an important aspect of flexible task switching is the inhibition of a recently performed task. This is called backward inhibition. Several studies suggest that task switching performance can be enhanced by rewards. However, it is less clear in how far backward inhibition mechanisms are also affected by rewards, especially when it comes to the neuronal mechanisms underlying reward-related modulations of backward inhibition. We therefore investigated this using a system neurophysiological approach combining EEG recordings with source localization techniques. We demonstrate that rewards reduce the strength of backward inhibition processes. The neurophysiological data shows that these reward-related effects emerge from response and/or conflict monitoring processes within medial frontal cortical structures. Upstream processes of perceptual gating and attentional selection, as well as downstream processes of context updating and stimulus-response mapping are not modulated by reward, even though they also play a role in backward inhibition effects.

Details DOI

YNIMG Journal 2015 Journal Article

Characterizing nonlinear relationships in functional imaging data using eigenspace maximal information canonical correlation analysis (emiCCA)

  • Li Dong
  • Yangsong Zhang
  • Rui Zhang
  • Xingxing Zhang
  • Diankun Gong
  • Pedro A. Valdes-Sosa
  • Peng Xu
  • Cheng Luo

Many important problems in the analysis of neuroimages can be formulated as discovering the relationship between two sets of variables, a task for which linear techniques such as canonical correlation analysis (CCA) have been commonly used. However, to further explore potential nonlinear processes that might co-exist with linear ones in brain function, a more flexible method is required. Here, we propose a new unsupervised and data-driven method, termed the eigenspace maximal information canonical correlation analysis (emiCCA), which is capable of automatically capturing the linear and/or nonlinear relationships between various data sets. A simulation confirmed the superior performance of emiCCA in comparison with linear CCA and kernel CCA (a nonlinear version of CCA). An emiCCA framework for functional magnetic resonance imaging (fMRI) data processing was designed and applied to data from a real motor execution fMRI experiment. This analysis uncovered one linear (in primary motor cortex) and a few nonlinear networks (e. g. , in the supplementary motor area, bilateral insula, and cerebellum). This suggests that these various task-related brain areas are part of networks that also contribute to the execution of movements of the hand. These results suggest that emiCCA is a promising technique for exploring various data.

Details DOI

AAAI Conference 2015 Conference Paper

Identifying At-Risk Students in Massive Open Online Courses

  • Jiazhen He
  • James Bailey
  • Benjamin Rubinstein
  • Rui Zhang

Massive Open Online Courses (MOOCs) have received widespread attention for their potential to scale higher education, with multiple platforms such as Coursera, edX and Udacity recently appearing. Despite their successes, a major problem faced by MOOCs is low completion rates. In this paper, we explore the accurate early identification of students who are at risk of not completing courses. We build predictive models weekly, over multiple offerings of a course. Furthermore, we envision student interventions that present meaningful probabilities of failure, enacted only for marginal students. To be effective, predicted probabilities must be both wellcalibrated and smoothed across weeks. Based on logistic regression, we propose two transfer learning algorithms to trade-off smoothness and accuracy by adding a regularization term to minimize the difference of failure probabilities between consecutive weeks. Experimental results on two offerings of a Coursera MOOC establish the effectiveness of our algorithms.

PDF Details

TIST Journal 2014 Journal Article

Traffic Information Publication with Privacy Preservation

  • Sashi Gurung
  • Dan Lin
  • Wei Jiang
  • Ali Hurson
  • Rui Zhang

We are experiencing the expanding use of location-based services such as AT&T’s TeleNav GPS Navigator and Intel’s Thing Finder. Existing location-based services have collected a large amount of location data, which has great potential for statistical usage in applications like traffic flow analysis, infrastructure planning, and advertisement dissemination. The key challenge is how to wisely use the data without violating each user’s location privacy concerns. In this article, we first identify a new privacy problem, namely, the inference-route problem, and then present our anonymization algorithms for privacy-preserving trajectory publishing. The experimental results have demonstrated that our approach outperforms the latest related work in terms of both efficiency and effectiveness.

Details DOI

EAAI Journal 2012 Journal Article

A hybrid local search algorithm for scheduling real-world job shops with batch-wise pending due dates

  • Rui Zhang
  • Cheng Wu

This paper aims at solving a real-world job shop scheduling problem with two characteristics, i. e. , the existence of pending due dates and job batches. Due date quotation is an important decision process for contemporary companies that adopt the MTO (make to order) strategy. Although the assignment of due dates is usually performed separately with production scheduling, there exist strong interactions between the two tasks. Therefore, we integrate these two decisions into one optimization model. Meanwhile, each order placed by the customer defines a batch of jobs, for which the same due date should be set. Thus, the completion times of these jobs should be close to one another in order to reduce waiting time and cost. For this purpose, we propose a dispatching rule to synchronize their manufacturing progresses. A two-stage local search algorithm based on the PMBGA (probabilistic model-building genetic algorithm) and parameter perturbation is proposed to solve the integrated scheduling problem and its superiority is revealed by the applications to a real-world mechanical factory.

Details DOI
v2026.09.13