Yu Zhang

EAAI Journal 2026 Journal Article

Coconut germination precise prediction via multimodal fusion with Co-attention networks: A non-destructive precision agriculture and food engineering solution

• Anum Mehmood
• Zemin Wu
• Yu Zhang
• Xinpeng Bai
• Chengxu Sun
• Uzair Aslam Bhatti
• Mengxing Huang
• Shenghuang Lin

Coconut is the fruit of the coconut palm. Due to its characteristics of a long growth cycle and low germination rate, accurate prediction of its developmental status is particularly important. Traditional research primarily relies on the sectioning method to observe its internal structure. Although this approach can reveal morphological characteristics, its destructive nature prevents continuous monitoring of the internal developmental processes. Recent advancements in Computed Tomography (CT)-based nondestructive imaging and artificial intelligence have enabled novel approaches for investigating internal coconut morphology. However, current methodologies frequently overlook the impact of field environmental factors on coconut germination processes, consequently constraining prediction accuracy. To address this issue, this study proposes a Transformer-based multimodal feature fusion predictive model. Through the integration of CT images and environmental data, the model achieves precise prediction of coconut developmental status. Initially, the enhanced Deeplab V3+ model extracts deep semantic features from coconut CT images, while Fourier positional encoding is applied to amplify periodic features in environmental data (e. g. , temperature, humidity). Subsequently, a cross-modal multi-head attention mechanism is designed to achieve comprehensive fusion between CT-derived semantic features and field data characteristics, thoroughly exploring their correlations. Ultimately, to further enhance model performance, this study incorporates supervised contrastive loss functions and implements intra-class feature aggregation coupled with inter-class feature separation strategies for feature space optimization. The experimental results demonstrate that the proposed model achieves superior performance in coconut developmental stage prediction tasks: compared with conventional unimodal approaches, it improves prediction accuracy and F1-score by 9 % and 8 %, respectively, thereby validating the effectiveness of multimodal data fusion and the rationality of the model design.

AAAI Conference 2026 Conference Paper

Efficient and Effective In-context Demonstration Selection with Coreset

• Zihua Wang
• Jiarui Wang
• Haiyang Xu
• Ming Yan
• Fei Huang
• Xu Yang
• Xiu-Shen Wei
• Siya Mi

In-context learning (ICL) has emerged as a powerful paradigm for Large Visual Language Models (LVLMs), enabling them to leverage a few examples directly from input contexts. However, the effectiveness of this approach is heavily reliant on the selection of demonstrations, a process that is NP-hard. Traditional strategies, including random, similarity-based sampling and infoscore-based sampling, often lead to inefficiencies or suboptimal performance, struggling to balance both efficiency and effectiveness in demonstration selection. In this paper, we propose a novel demonstration selection framework named Coreset-based Dual Retrieval (CoDR). We show that samples within a diverse subset achieve a higher expected mutual information. To implement this, we introduce a cluster-pruning method to construct a diverse coreset that aligns more effectively with the query while maintaining diversity. Additionally, we develop a dual retrieval mechanism that enhances the selection process by achieving global demonstration selection while preserving efficiency. Experimental results demonstrate that our method significantly improves the ICL performance compared to the existing strategies, providing a robust solution for effective and efficient demonstration selection.

AAAI Conference 2026 Conference Paper

Embracing Positional Bias in Multiple-Choice Question Answering via Permutation Equivariant Neural Networks

• Chengyu Jiao
• Siyin Huang
• Yu Zhang

Several studies have demonstrated that large language models (LLMs) exhibit positional bias when answering multiple-choice questions (MCQs). Previous methods have identified such bias to be detrimental, leading to the development of techniques to mitigate it. However, we observe that certain permutations of options can actually improve the performance. Therefore, instead of eliminating such bias, we propose an EMbracing the Bias EquivaRiantly (EMBER) network. Specifically, the EMBER network, which outputs a permutation of options in MCQs, is optimized towards the beneficial permutations to which the LLM is biased. Additionally, to solve the positional bias among different permutations of options, the EMBER network is designed to grant the equivariance to the permutation to the LLMs. Theoretically and empirically, we show that the proposed EMBER network can effectively utilize the positional bias and demonstrate state-of-the-art performance over various baselines.

AAAI Conference 2026 Conference Paper

Exact Optimization for Minimum Dominating Sets

• Enqiang Zhu
• Qiqi Bao
• Yu Zhang
• Chanjuan Liu
• Pu Wu

The Minimum Dominating Set (MDS) problem is a well-established combinatorial optimization problem with numerous real-world applications. Its NP-hard nature makes it increasingly difficult to obtain exact solutions as the graph size grows. This paper introduces ParDS, an exact algorithm developed to address the MDS problem within the branch-and-bound framework. ParDS features two key innovations: an advanced linear programming technique that yields tighter lower bounds and a set of novel reduction rules that dynamically simplify instances throughout the solving process. Compared to the leading exact algorithms presented at IJCAI 2023 and 2024, ParDS demonstrates theoretically superior lower-bound quality. Experimental results on standard benchmark datasets highlight several significant advantages of ParDS: it achieves fastest solving times in 70% of graph categories, especially on large, sparse graphs, delivers a speed-up of up to 3,411 times on the fastest individual instance, and successfully solves 16 out of 43 instances that other algorithms were unable to resolve within the 5-hour time limit. These findings establish ParDS as a state-of-the-art solution for exactly solving the MDS problem

AAAI Conference 2026 Conference Paper

Graph2Video: Leveraging Video Models to Model Dynamic Graph Evolution

• Hua Liu
• Yanbin Wei
• Fei Xing
• Tyler Derr
• Haoyu Han
• Yu Zhang

Dynamic graphs are common in real‑world systems such as social media, recommender systems, and traffic networks. Existing dynamic graph models for link prediction often fall short in capturing the full complexity of temporal evolution. They tend to overlook fine‑grained variations in interaction order, struggle with dependencies that span long time horizons, and provide limited modeling of pair‑specific relational dynamics. To address those challenges, we propose Graph2Video, a video‑inspired framework that views the temporal neighborhood of a target link as a sequence of “graph frames”. By stacking temporally ordered subgraph frames into a “graph video”, Graph2Video leverages the inductive biases of video foundation models to capture both fine-grained local variations and long-range temporal dynamics. It generates a link-level embedding that serves as a lightweight, plug-and-play, link-centric memory unit. This embedding integrates seamlessly into existing dynamic graph encoders, effectively addressing the limitations of prior approaches. Extensive experiments on benchmark datasets show that Graph2Video outperforms state‑of‑the‑art baselines in the link prediction task on most cases. The results highlight that borrowing spatio‑temporal modeling techniques from computer vision provides a principled and effective avenue for advancing dynamic graph learning.

AAAI Conference 2026 Conference Paper

HalluClean: A Unified Framework to Combat Hallucinations in LLMs

• Yaxin Zhao
• Yu Zhang

Large language models (LLMs) have achieved impressive performance across a wide range of natural language processing tasks, yet they often produce hallucinated content that undermines factual reliability. To address this challenge, we introduce HalluClean, a lightweight and task-agnostic framework for detecting and correcting hallucinations in LLM-generated text. HalluClean adopts a reasoning-enhanced paradigm, explicitly decomposing the process into planning, execution, and revision stages to identify and refine unsupported claims. It employs minimal task-routing prompts to enable zero-shot generalization across diverse domains, without relying on external knowledge sources or supervised detectors. We conduct extensive evaluations on five representative tasks—question answering, dialogue, summarization, math word problems, and contradiction detection. Experimental results show that HalluClean significantly improves factual consistency and outperforms competitive baselines, demonstrating its potential to enhance the trustworthiness of LLM outputs in real-world applications.

EAAI Journal 2026 Journal Article

Hierarchical detection and evaluation method for surface defects based on dynamic feature selection and uncertainty-guided optimization

• Liangliang Li
• Peng Wang
• RuoHai Di
• Chao Xu
• Mengyu Sun
• Zhigang Lü
• Yu Zhang

Steel is an essential industrial foundational material, and surface defects can severely compromise product performance and service life. Conventional detection methods are subject to numerous limitations. While deep learning-based approaches excel at defect localization, they remain deficient in extracting precise edge contour information and quantifying model uncertainty. To address these challenges, this paper proposes a hierarchical steel surface defect detection and evaluation framework based on dynamic feature selection and uncertainty-guided optimization. The proposed ConfidenceSeg-Net model integrates feature enhancement and dynamic selection mechanisms. During encoding, the ReSidual U-block module extracts and adaptively enhances multi-scale features. In the decoding phase, multi-scale feature fusion and attention-guided feature fusion modules are employed to integrate feature maps. The segmentation head combines depth feature space and side-output modules to generate final segmentation maps, thereby improving both accuracy and fine-detail representation. Furthermore, an uncertainty-constrained, credibility-driven loss optimization function is designed, comprising basic loss, credibility-weighted loss, uncertainty regularization, and credibility consistency loss terms. The weighted combination of these components significantly enhances model performance and reliability. Finally, a multi-dimensional comprehensive evaluation system is established to assess defect detection reliability. This system encompasses seven key metrics: primary prediction reliability, average auxiliary prediction reliability, mean uncertainty, multi-scale feature expressiveness, prediction stability, inter-layer feature consistency, and prediction diversity. Dimension-specific scores are computed and fused through weighted integration to generate a comprehensive credibility score, enabling thorough and nuanced performance assessment. Experimental results on public datasets demonstrate superior performance, with Precision, Recall, and Jaccard indices reaching 0. 8808, 0. 8877, and 0. 7917, respectively. The proposed framework also provides a portable, comprehensive credibility evaluation mechanism, substantially enhancing practical applicability and reliability.

JBHI Journal 2026 Journal Article

HSD: Hough-Based Structure-Aware Detection of B-Lines in Lung Ultrasound

• Tuo Liu
• Hao Zhou
• Jia-Hao Wang
• Yu Zhang
• Chen Chen
• Yang Chen
• Guang-Quan Zhou
• Lin Li

B-lines are artifacts produced by the interaction of the ultrasound with the small air-liquid interface, which often serve as crucial biomarkers for evaluating lung pathology, such as the presence of liquid. However, due to the reverberation phenomenon, B-lines manifest as blurred, strip-like comet tails perpendicularly originating from the pleural line, making their automatic identification in speckle-noisy ultrasound images particularly challenging. This study proposes a Hough-based structure-aware detection framework, dubbed HSD, which leverages structural priors and the intrinsic relationship between the pleural line and B-lines to enhance B-line detection in ultrasound images. First, the proposed method adopts the shared encoder and two collaborative decoders to improve B-lines identification with the auxiliary pleural line detection, ensuring effective representation learning of linear structural features under inherent prior constraints. Specifically, one decoder incorporates Hough-based regression to reinforce the modeling of the global linear nature for B-line detection, alleviating the appearance influences of the fuzzy comet-tail. Simultaneously, another pathway enhances the exploration of the slender, curved morphology by integrating semantic context learning with linear heatmap regression, thereby facilitating the detection of the pleural line for calibration of B-lines. Second, we introduce a position-aware rectification module to ensure the consistency of the pleural line and its perpendicular alignment with B-lines. This post-processing module reduces the influence of ambiguous pixels, improving the robustness of B-line detection. Extensive experimental results on an in-house ultrasound dataset demonstrate the superiority of the proposed approach, which achieves a precision of 0. 743, a recall of 0. 953, and an F-measure of 0. 837, substantially ahead of other methods, suggesting its potential for detecting pathological indicators in lung ultrasound.

EAAI Journal 2026 Journal Article

Knowledge-enhanced path planning for autonomous underwater vehicle covert navigation by fusing thermocline acoustics with an improved rapidly-exploring random tree algorithm

• Yanli Chen
• Tianlong Ren
• Zhe Cao
• Yu Zhang

This paper introduces a knowledge-enhanced path planning method for Autonomous Underwater Vehicle (AUV) path planning that integrates hydroacoustic propagation experience with an enhanced Rapidly-exploring Random Tree (RRT∗) algorithm. The study primarily utilizes real hydrological data, leveraging the critical influence of oceanic thermal stratification on sound wave propagation. To augment conventional passive stealth techniques (e. g. , specialized materials and shapes), this work proposes an active, knowledge-enhanced path planning method. This approach complements passive features by enabling the AUV to dynamically exploit environmental acoustics for concealment. The proposed framework incorporates three key innovations: (1) construction of a three-dimensional acoustic exposure risk field induced by thermoclines, based on buoy data and ray-tracing simulations; (2) an adaptive path planning algorithm tailored to AUV maneuvering characteristics, featuring directional sampling with exponentially decaying step sizes and path smoothing to enhance efficiency and quality; (3) construction of a physics-based acoustic exclusion map, which the path planner utilizes as a hard constraint to ensure complete avoidance of identified high-risk zones while optimizing for geometric efficiency. Experimental results from both controlled tank and field lake trials demonstrate the algorithm's superior performance in planning time, path cost, and cumulative risk reduction. These outcomes validate the feasibility of the core methodology, establishing a proof-of-concept for knowledge-enhanced covert path planning in stratified environments.

AAAI Conference 2026 Conference Paper

Post-Hoc Refinement for Multitask Symbolic Regression via Consensus-Accelerated Shapley Analysis

• Xinyue Li
• Wang Hu
• Yu Zhang

Multitask genetic programming (MTGP) is one of the primary methods for solving multitask symbolic regression (MTSR), the problem of discovering mathematical expressions for multiple interconnected tasks simultaneously. However, conventional MTGP approaches discard a wealth of valuable knowledge from the population of expressions due to their inherent “winner-take-all” selection criteria. To address this, we introduce MTGP with bidirectional cooperation and consensus-accelerated Shapley analysis (MTGP-BS), a method whose core is a novel post-hoc refinement framework that shifts from selection to synthesis. Our method first employs a consensus-accelerated Shapley analysis to reliably identify important subexpressions by multi-model attribution. Second, to supply this analysis with high-quality candidates, we design a bidirectional subexpression cooperative extraction method to create a refined archive of effective components by improving knowledge transfer and filtering out redundancies. These allow MTGP-BS to synthesize superior expressions by integrating knowledge dispersed throughout the entire population. On diverse MTSR problems, our algorithm statistically outperformed state-of-the-art approaches in 140 out of 160 direct comparisons, with its effectiveness and practical utility further verified by real-world case studies and in-depth ablation analyses.

TMLR Journal 2026 Journal Article

Provably Efficient Reward Transfer in Reinforcement Learning with Discrete Markov Decision Processes

• Kevin Jatin Vora
• Yu Zhang

In this paper, we propose a new solution to reward adaptation (RA) in reinforcement learning, where the agent adapts to a target reward function based on one or more existing source behaviors learned a priori under the same domain dynamics but different reward functions. While learning the target behavior from scratch is possible, it is often inefficient given the available source behaviors. Our work introduces a new approach to RA through the manipulation of Q-functions. Assuming the target reward function is a known function of the source reward functions, we compute bounds on the Q-function and present an iterative process (akin to value iteration) to tighten these bounds. The iteration process is based on a lite-model, which is assumed to be given or can be learned. The computed bounds enable action pruning in the target domain before learning even starts. We refer to this method as "$Q-Manipulation$" (Q-M). We formally prove that Q-M, under discrete domains and an accurate lite-model, does not affect the optimality of the returned policy and show that it is provably efficient in terms of sample complexity. Q-M is evaluated in a variety of synthetic and simulation domains to demonstrate its effectiveness, generalizability, and practicality.

AAAI Conference 2026 Conference Paper

Revisiting Contrastive Learning in Collaborative Filtering via Parallel Graph Filters

• Fang Kai
• Yu Zhang
• Kaibin Wang
• Lei Sang
• Yiwen Zhang

Graph Contrastive Learning (GCL) has recently emerged as a powerful paradigm for modeling user–item interactions and learning high-quality representations in recommender systems. While existing GCL-based methods benefit from data augmentation and sampling strategies, they often overlook the inherent limitations of the contrastive objectives: 1) Stacking multiple Graph Convolutional Network layers to capture high-order information often causes the over-smoothing phenomenon, where node representations become overly similar. 2) Structurally similar negative sample pairs may exhibit high cosine similarity, causing gradient saturation during representation optimization. To address the above challenges, we revisit matrix factorization in recommendation models and uncover its implicit connection to a parallel graph filter bank. This perspective reveals how overly aggressive low-pass or high-pass filtering distorts feature distributions, contributing to gradient saturation. Building on this insight, we propose Light Cosine Similarity Collaborative Filtering (LightCSCF), a margin-constrained method that improves gradient optimization in contrastive learning by focusing on structurally hard examples, alleviating both gradient saturation and boundary over-smoothing. Extensive experiments on three real-world datasets demonstrate that LightCSCF consistently outperforms state-of-the-art baselines in recommendation accuracy and robustness to data sparsity.

YNIMG Journal 2026 Journal Article

Right DLPFC stimulation reveals context-dependent regulation of competing motives in third-party fairness decisions

• Rongrong Chen
• Jiwen Chen
• Yu Zhang
• Xiaoqin Mai

The right dorsolateral prefrontal cortex (rDLPFC) plays a crucial role in fairness-related decision-making, yet it remains unclear whether its role reflects a fairness-promoting mechanism or a broader control process that regulates competing motivations. Here, we combined noninvasive brain stimulation with a third-party ultimatum game to examine how modulating rDLPFC excitability influences fairness decisions in which fairness-related considerations and concern for others differ in relative salience. Ninety healthy female university students (mean age 20.92 ± 2.35) decided whether to accept or reject offers on behalf of university students or charity recipients across three inequality conditions (advantageous, disadvantageous, and equal). Behavioral analyses revealed context-dependent effects of rDLPFC stimulation. Compared with the sham condition, anodal stimulation increased acceptance of advantageous inequality offers for university students (β = 1.27, P = 0.002) but decreased acceptance for charity recipients (β = -1.77, P = 0.003), with no significant effects in disadvantageous or equality conditions. Subjective emotion and fairness ratings followed similar patterns, and mediation analyses indicated that emotion partially mediated the influence of stimulation on choice. These findings support a domain-general account, indicating that the rDLPFC flexibly regulates dominant motivational tendencies depending on social context. Overall, the study demonstrates the adaptive role of the rDLPFC in integrating fairness norms with competing social motives during decision-making.

AAAI Conference 2026 Conference Paper

Robust Integrative Analysis of Multi-omics Datasets via Nuclear-norm Maximization

• Meng-zhu Wang
• Yu Zhang
• Hongxing Zhang

Spatially multimodal omics technologies provide unprecedented opportunities to address cellular heterogeneity within tissue contexts. However, learning robust and informative latent representations from such complex data remains a significant challenge. Existing graph-based methods often rely on static connections or indirect optimization objectives, which can constrain the discriminability and diversity of the learned representations, particularly in the presence of sequencing noise and unknown biological priors. To overcome these limitations, we propose Robust Integrative Analysis of Multi-omics Datasets via Nuclear-norm Maximization (RIA) to adaptively integrate multimodal features and spatial information through a new graph-based architecture. At the core of RIA is the introduction of the batch nuclear norm maximization (bnm) loss, marking the first application of bnm within the multi-omics domain. By maximizing the nuclear norm of the batch assignment matrix derived from the latent space, RIA simultaneously enhances the discriminability and diversity of the learned embeddings. This objective is synergistically combined with a dynamic prototype contrastive learning strategy and a graph stability loss, ensuring comprehensive and robust optimization.Ultimately, RIA produces a structured, information-rich latent space that enables more reliable downstream analyses, including cell type identification, spatial domain discovery, and microenvironment characterization.

AAAI Conference 2026 Conference Paper

RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation

• Haofeng Wang
• Yu Zhang

Large Vision-Language Models (LVLMs) excel in multimodal reasoning and have shown impressive performance on various multimodal benchmarks. However, most of these benchmarks evaluate models primarily through multiple-choice or short-answer formats, which do not take the reasoning process into account. Although some benchmarks assess the reasoning process, their methods are often overly simplistic and only examine reasoning when answers are incorrect. This approach overlooks scenarios where flawed reasoning leads to correct answers. In addition, these benchmarks do not consider the impact of intermodal relationships on reasoning. To address this issue, we propose the Reasoning Process Tree Score (RPTS), a tree structure-based metric to assess reasoning processes. Specifically, we organize the reasoning steps into a reasoning tree and leverage its hierarchical information to assign weighted faithfulness scores to each reasoning step. By dynamically adjusting these weights, RPTS not only evaluates the overall correctness of the reasoning, but also pinpoints where the model fails in the reasoning. To validate RPTS in real-world multimodal scenarios, we construct a new benchmark, RPTS-Eval, comprising 374 images and 390 reasoning instances. Each instance includes reliable visual-textual clues that serve as leaf nodes of the reasoning tree. Furthermore, we define three types of intermodal relationships to investigate how intermodal interactions influence the reasoning process. We evaluated representative LVLMs (e.g., GPT4o, Llava-Next), uncovering their limitations in multimodal reasoning and highlighting the differences between open-source and closed-source commercial LVLMs. We believe that this benchmark will contribute to the advancement of research in the field of multimodal reasoning.

AAAI Conference 2026 Conference Paper

SafeNLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces

• Ruiheng Liu
• Xiaobing Chen
• Jinyu Zhang
• Qiongwen Zhang
• Yu Zhang
• Bailong Yang

The rapid advancement of Large Language Models (LLMs) has driven significant progress in Natural Language Interface to Database (NLIDB). However, the widespread adoption of LLMs has raised critical privacy and security concerns. During interactions, LLMs may unintentionally expose confidential database contents or be manipulated by attackers to exfiltrate data through seemingly benign queries. While current efforts typically rely on rule-based heuristics or LLM agents to mitigate this leakage risk, these methods still struggle with complex inference-based attacks, suffer from high false positive rates, and often compromise the reliability of SQL queries. To address these challenges, we propose SafeNLIDB, a novel privacy-security alignment framework for LLM-based NLIDB. The framework features an automated pipeline that generates hybrid chain-of-thought interaction data from scratch, seamlessly combining explicit security reasoning with SQL generation. Additionally, we introduce reasoning warm-up and alternating preference optimization to overcome the multi-preference oscillations of Direct Preference Optimization (DPO), enabling LLMs to produce security-aware SQL through fine-grained reasoning without the need for human-annotated preference data. Extensive experiments demonstrate that our method outperforms both larger-scale LLMs and ideal-setting baselines, achieving significant security improvements while preserving high utility.

EAAI Journal 2026 Journal Article

Salient feature aware network for red–green–blue–thermal crowd counting

• Shenjian Gong
• Yu Zhang
• Rui Yang
• Heng Yin
• Xing Yang

In the field of red–green–blue–thermal (RGB–T) crowd counting, mainstream methods focus on designing complex cross-modal feature fusion modules. Moreover, the issue of cross-modal spatial misalignment has received little attention in relation to this task, which significantly impacts precise feature extraction and fusion. To address these issues, we propose a novel Salient feature aware network (SFANet) for RGB–T crowd counting accompanied by an image preprocessing strategy for cross-modal spatial alignment. Specifically, on one hand, we employ a pre-trained feature matching model to calculate the cross-modal spatial transformation matrix which is applied to the RGB image to achieve the spatially aligned RGB–T image pairs. On the other hand, we classify the multi-modal information that is beneficial to crowd counting into the features of salient alignment/difference. Based on this, we design a multi-task framework with three branches. Each of the two modality-specific branches completes crowd counting separately, and the inter-modal feature subtraction provides the perception of cross-modal salient difference features. The multi-modal branch takes the RGB–T image pairs as input to extract salient alignment features. It is concatenated with the salient difference features to predict the high-quality density map. Extensive experiments, along with ablation studies on the RGBT-Crowd Counting (RGBT-CC) dataset show that our method performs better. The extended experimental results on the ShanghaiTech RGB–Depth (ShanghaiTechRGBD) and drone based RGB–T crowd counting (DroneRGBT) datasets indicate the feasibility and effectiveness of our approach. The code will be publicly available at https: //github. com/Yankeegsj/SFANet.

AAAI Conference 2026 Conference Paper

SAME: Spatial-Aware Multimodal Egocentric Human Pose Estimation

• Yurong Fu
• Peng Dai
• Yu Zhang
• Feng Yiqiang
• Yang Zhang
• Haoqian Wang

Egocentric human pose estimation (HPE) plays a crucial role in immersive applications such as virtual and augmented reality. However, existing methods relying on either visual or sparse inertial data alone often suffer from occlusion or ill-posed problems. In this work, we propose SAME, a novel spatial-aware multimodal fusion framework combining the complementary signals from the stereo images and sparse IMUs for accurate and robust egocentric HPE. It adopts a two-stage network based on a dual coordinate frame to mitigate the coordinate inconsistencies among the stereo cameras and the IMUs. In the first stage, the IMU signals are transformed into the local frame and iteratively fused with the stereo images for estimating 3D poses in the local frame. In the second stage, the local poses are transformed into the global frame with the 6DOF head poses provided by the head-mounted display's (HMD) SLAM algorithm and then temporally aggregated via a temporal Transformer network. Meanwhile, to achieve geometric and semantic alignment among multi-modal features, we present a depth-guided spatial-aware deformable stereo attention network and a modality-aware Transformer decoder for cross-view and cross-modal feature fusion. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on the public EMHI multi-modal egocentric pose estimation benchmark.

EAAI Journal 2025 Journal Article

A lightweight algorithm for wildlife detection in outdoor environments based on you only look once version 8 network

• Pengtao Jia
• Yu Zhang

To address the challenges posed by outdoor environments, such as obstructions, background noise, and the limited computational resources of detection devices that hinder accurate wildlife detection, we propose a lightweight wildlife detection algorithm named Wildlife-You Only Look Once (WL-YOLO). First, the High-level Screening-feature Pyramid Networks (HS-FPN) is introduced to enhance the processing of multi-scale wildlife features. This architecture captures comprehensive feature information while significantly reducing computational complexity. Next, the Depthwise Adaptive Spatial Attention (DASA) module is designed to improve the adaptability of the model to various environmental backgrounds, effectively addressing challenges such as object occlusion and complex backgrounds in wildlife images. Additionally, the Scylla Intersection over Union (SIoU) loss is utilized to optimize detection accuracy. Finally, a pruning method based on layer-adaptive magnitude-based pruning (LAMP) is applied to the model to trim redundant parameters. Experimental results indicate that WL-YOLO achieves comparable detection accuracy to the original YOLO model (version 8) while reducing parameters by 66. 7% and computational load by 57. 5%. It processes images at a speed of 344 frames per second. Furthermore, detection results indicate that WL-YOLO performs better in handling occlusions and complex environments. Compared to mainstream object detection algorithms, WL-YOLO achieves a favorable balance among detection accuracy, inference speed, and model complexity. The research suggests that WL-YOLO is better applicable in complex outdoor environments, offering a new pathway for research on wildlife diversity. The code will be available on GitHub (https: //github. com/xust-9527/WL-YOLO).

JBHI Journal 2025 Journal Article

ACEA-Net: Weakly Supervised Prostate 3D MRI Image Segmentation via Advanced Prompt Points

• Jie Zou
• Mengxing Huang
• Yu Zhang
• Zhiyuan Zhang
• Wenjie Zhou
• Uzair Aslam Bhatti
• Jing Chen
• Zhiming Bai

In prostate 3D MRI image segmentation methods, it is usually necessary to annotate each slice, and these annotations are generally time-consuming and specialized. In this study, we generate pseudo-labels using an annotation method with one foreground seed point and six edge relaxation points. We design a weakly supervised semantic learning segmentation framework, ACEA-Net. This segmentation framework solves the under-expansion problem due to the lack of semantic affinity of the seed point pixels in the pseudo-labeling generation process. We design a Seed Cluster Geodesic Distance Transform (SeedGeo) seed expansion strategy to provide a more complete supervised signal. In the segmentation model training phase, Adaptive Convolutional Normalization (ACN) and Enhanced Simple Parameter-Free Attention Module (SimAM) are utilized to smooth the convolutional layer's output in the U-Net baseline model to suppress noisy labels. The proposed segmentation framework achieves excellent segmentation results on the MSD prostate and PROMISE12 prostate datasets, with Dice similarity coefficients (Dice) of 87. 23% and 81. 00% for the two segmentation tasks, and Average Symmetry Surface Distances (ASSD) of 1. 73mm and 2. 02mm, respectively, which are superior to the current state-of-the-art method.

IROS Conference 2025 Conference Paper

ACoL: From Abstractions to Grounded Languages for Robust Coordination of Task Planning Robots

• Yu Zhang

In this paper, we consider a first step to bridge a gap in coordinating task planning robots. Specifically, we study the automatic construction of languages that are maximally flexible while being sufficiently explicative for coordination. To this end, we view language as a machinery for specifying temporal-state constraints of plans. Such a view enables us to reverse-engineer a language from the ground up by mapping these composable constraints to words. Our language expresses a plan for any given task as a "plan sketch" to convey just-enough details while maximizing the flexibility to realize it, leading to robust coordination with optimality guarantees among other benefits. We formulate and analyze the problem, provide approximate solutions, and validate our approach under various scenarios to shed light on its applications.

AAAI Conference 2025 Conference Paper

Adaptive Wavelet-Positional Encoding for High-Frequency Information Learning in Implicit Neural Representation

• Hongxu Zhao
• Zelin Gao
• Yue Wang
• Rong Xiong
• Yu Zhang

Implicit Neural Representation (INR) has shown great potential in constructing the complex nature signal as a continuous implicit function. However, the representation results are incomplete since different components of the signal correspond to different frequencies and neural network inherently tends to low-frequency convergence. In this paper, we propose the adaptive Wavelet-Positional Encoding (WPE) to precisely represent content under different frequency distributions for coordinate-based implicit representations. The High-Frequency Perception (HFP) method is first proposed to query locations of high-frequency components from input signals, which can be indicated as local centers of WPE. Then, motivated by wavelet series regression, we present to embed these queried low-dimensional coordinate inputs into wavelet-frequency space by WPE to represent fine details of target signals. Experiments demonstrate that the proposed method can be integrated into various INR methods without modifying training frameworks while significantly improving their performance in 1D signal fitting, 2D image regression, and even 3D scene representation.

JBHI Journal 2025 Journal Article

ADMM-ESINet: A Deep Unrolling Network for EEG Extended Source Imaging

• Ke Liu
• Hang Jiang
• Hu Yang
• Jun Zhang
• Zhenghui Gu
• Zhuliang Yu
• Yu Zhang
• Bin Xiao

Electroencephalography (EEG) source imaging (ESI) methods aim to reconstruct cortical sources from scalp EEG signals, a crucial task for understanding the normal brain as well as brain disorders. Traditional model-driven ESI methods face challenges in real-time reconstruction, while deep neural network (DNN)-based ESI methods often struggle with generalization to new data. To address these issues, we propose ADMM-ESINet, a novel deep unfolding neural network for robust and efficient reconstruction of EEG extended sources. ADMM-ESINet leverages a structured sparsity constraint within a regularization framework and employs the Alternating Direction Method of Multipliers (ADMM) to achieve iterative solutions. By unrolling the ADMM algorithm into a cascaded network architecture, ADMM-ESINet effectively integrates prior knowledge, enabling end-to-end, real-time ESI. Crucially, both the regularization parameters and the spatial transform operator are learned directly from the training data. Numerical results demonstrate that ADMM-ESINet surpasses traditional DNN-based methods in generalization ability and accurately reconstructs the location, extent, and temporal dynamics of extended sources, establishing ADMM-ESINet as a promising method for real-time ESI.

EAAI Journal 2025 Journal Article

An effective exploration method based on N-step updated Dirichlet distribution and Dempster–Shafer theory for deep reinforcement learning

• Fanghui Huang
• Yixin He
• Yu Zhang
• Bin Chen
• Lina Yang

Deep reinforcement learning (DRL) has been regarded as a promising approach for solving decision-making problems. However, how to enhance the agent exploration ability is still an extremely challenging issue for existing methods, especially under sparse rewards. Facing with this challenge, we propose a novel efficient exploration method, which can comprehensively consider the uncertainty of the environment and the uncertainty of Q function, so as to improve the agent exploration efficiency. Specifically, we first construct an exploration policy by n-step updated Dirichlet distribution to implement the adaptive exploration of the agent to the environment, which can reduce the uncertainty of the agent about the environment to achieve global efficient exploration. Next, a state–action basic probability assignment (BPA) is constructed based on the Dempster–Shafer theory. On this basis, an interval Q function is designed by combining BPA and belief interval, which can effectively characterize the uncertainty of the Q function to achieve deep exploration. Then, the proposed method is applied to classic DRL algorithms, deep Q-network (DQN) and double DQN (DDQN), two novel algorithms are proposed. Finally, under a series of sparse external reward tasks, experimental results show that our proposed algorithms outperform several state-of-the-art DRL algorithms in term of exploring efficiency.

JBHI Journal 2025 Journal Article

An Improved Microbial Object Detection Method for Low-Contrast and Occluded Scenarios Based on SMA-YOLOv8s

• Kun Sun
• Zhenqiang Song
• Jiaxing Zhang
• Shiyu Liu
• Yu Zhang
• Qinghao Song
• Qing Wu

Accurate detection and localization of microbial targets are critical for microbial trajectory tracking and analysis. However, microscopic microorganism images often exhibit low contrast and mutual occlusion between targets, which pose significant challenges for microbial object accuracy detection due to insufficient distinguishable shallow-layer information and occluded targets inadequate representation. To address these issues, a novel method of SMA-YOLOv8s is proposed for microbial object detection. Firstly, the traditional strided convolutions is replaced with SPD-Conv in downsampling of YOLOv8s to retain shallow-layer information. Secondly, a feature fusion strategy that integrates Cascaded Group Attention with Scale Sequence Feature Fusion (CSFF) is proposed, which could enrich contextual feature representation for better detecting occluded targets. Thirdly, the Wise-IoU loss function is employed to optimize bounding box regression, improving localization precision. Experimental evaluations on the BCCD, CTMCv1, and a self-constructed microscopic microorganism dataset demonstrate that SMA-YOLOv8s achieves mAP50 scores of 95. 5%, 90. 3%, and 81. 7%, respectively, surpassing baseline methods in overall performance. These results highlight the robustness and effectiveness of the proposed method in detecting microbial targets under low contrast and occlusion conditions.

TMLR Journal 2025 Journal Article

Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learning

• Huaiyuan Qin
• Muli Yang
• Siyuan Hu
• Peng Hu
• Yu Zhang
• Chen Gong
• Hongyuan Zhu

Self-supervised learning (SSL) conventionally relies on the instance consistency paradigm, assuming that different views of the same image can be treated as positive pairs. However, this assumption breaks down for non-iconic data, where different views may contain distinct objects or semantic information. In this paper, we investigate the effectiveness of SSL when instance consistency is not guaranteed. Through extensive ablation studies, we demonstrate that SSL can still learn meaningful representations even when positive pairs lack strict instance consistency. Furthermore, our analysis further reveals that increasing view diversity, by enforcing zero overlapping or using smaller crop scales, can enhance downstream performance on classification and dense prediction tasks. However, excessive diversity is found to reduce effectiveness, suggesting an optimal range for view diversity. To quantify this, we adopt the Earth Mover’s Distance (EMD) as an estimator to measure mutual information between views, finding that moderate EMD values correlate with improved SSL learning, providing insights for future SSL framework design. We validate our findings across a range of settings, highlighting their robustness and applicability on diverse data sources.

NeurIPS Conference 2025 Conference Paper

ChromFound: Towards A Universal Foundation Model for Single-Cell Chromatin Accessibiltiy Data

• Yifeng Jiao
• Yuchen Liu
• Yu Zhang
• Xin Guo
• Yushuai Wu
• Chen Jiang
• Jiyang Li
• Hongwei Zhang

The advent of single-cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq) offers an innovative perspective for deciphering regulatory mechanisms by assembling a vast repository of single-cell chromatin accessibility data. While foundation models have achieved significant success in single-cell transcriptomics, there is currently no foundation model for scATAC-seq that supports zero-shot high-quality cell identification and comprehensive multi-omics analysis simultaneously. Key challenges lie in the high dimensionality and sparsity of scATAC-seq data, as well as the lack of a standardized schema for representing open chromatin regions (OCRs). Here, we present ChromFound, a foundation model tailored for scATAC-seq. ChromFound utilizes a hybrid architecture and genome-aware tokenization to effectively capture genome-wide long contexts and regulatory signals from dynamic chromatin landscapes. Pretrained on 1. 97 million cells from 30 tissues and 6 disease conditions, ChromFound demonstrates broad applicability across 6 diverse tasks. Notably, it achieves robust zero-shot performance in generating universal cell representations and exhibits excellent transferability in cell type annotation and cross-omics prediction. By uncovering enhancer-gene links undetected by existing computational methods, ChromFound offers a promising framework for understanding disease risk variants in the noncoding genome. The implementation of ChromFound is available via https: //github. com/JohnsonKlose/ChromFound.

EAAI Journal 2025 Journal Article

DC-Net: Decomposing and coupling saliency map for lesion segmentation in ultrasound images

• Zhenyuan Ning
• Yixiao Mao
• Xiaotong Xu
• Qianjin Feng
• Shengzhou Zhong
• Yu Zhang

Accurate lesion segmentation in ultrasound images faces the unique challenge that adjacent tissues (i. e. , background) share similar intensity and often exhibit richer texture patterns than the lesion region (i. e. , foreground). This work presents a decomposition-coupling network, called DC-Net, to deal with this challenge in a (foreground-background) saliency map disentanglement-fusion manner. The DC-Net consists of decomposition and coupling subnets, and the former preliminarily disentangles original image into foreground and background saliency maps, followed by the latter for accurate segmentation under the assistance of saliency prior fusion. The coupling subnet involves three aspects of fusion strategies, including: (1) regional feature aggregation (via differentiable context pooling operator in the encoder) to adaptively preserve local contextual details with the larger receptive field during dimension reduction; (2) relation-aware representation fusion (via cross-correlation fusion module in the decoder) to efficiently fuse low-level visual characteristics and high-level semantic features during resolution restoration; (3) dependency-aware prior incorporation (via coupler) to reinforce foreground-salient representation with the complementary information derived from background representation. Furthermore, a harmonic loss function is introduced to encourage the network to focus more attention on low-confidence and hard pixels. The proposed method is evaluated on two ultrasound lesion segmentation tasks, achieving better performance than second best methods (i. e. , dice similarity coefficient: 73. 96 ± 0. 36 (+4. 18%)/76. 25 ± 1. 89 (+3. 88%), mean intersection over union/jaccard index: 64. 26 ± 0. 48 (+5. 07%)/65. 28 ± 2. 24 (+6. 46%), precision: 75. 82 ± 1. 16 (+8. 07%)/79. 35 ± 1. 92 (+3. 07%), 95% hausdorff distance: 5. 44 ± 0. 10 (+5. 39%)/6. 22 ± 0. 26 (+5. 18%) on breast/thyroid ultrasound). Our proposed DC-Net can adapt to multiple ultrasound segmentation tasks and achieve remarkable performance improvement. Our code is available at https: //github. com/smu-myx/DC-Net.

IROS Conference 2025 Conference Paper

Design and Control of SeparaTrek: A Hybrid Aerial-Ground Robot with Separable and Combinative Locomotion Parts

• Yu Zhang
• Xuechao Chen
• Yanbo Sun

The hybrid aerial-ground robots combine ground mobility and aerial flight capability, which are often designed for executing multi-terrain tasks. However, most of the existing hybrid aerial-ground robots integrate the ground and aerial functionality into one single whole system. This leads to functional coupling that prevents the full utilization of multimodal locomotion capabilities. In this paper, we design a hybrid aerial-ground robot called SeparaTrek. SeparaTrek features separable and combinative ground and aerial locomotion parts by a free separation and combination structure, reducing the coupling relationship between ground and aerial functionality. Furthermore, we design a multimodal locomotion controller based on extended Kalman filter algorithm and adaptive sliding mode control, achieving stable locomotion of SeparaTrek on complex and variable terrain. Through experiments, it is demonstrated that SeparaTrek’s design is rational and its motion is stable.

IROS Conference 2025 Conference Paper

Dual-Modal Magnetic Skin for Robust Tactile Sensing

• Pengwen Xiong
• Huan Peng
• Yu Zhang
• Aiguo Song
• Peter X. Liu

Traditional magnetic tactile sensors are highly susceptible to external magnetic field interference, limiting their reliability in practical applications. To address this challenge, we propose a dual-modal soft magnetic skin capable of simultaneously acquiring magnetic and force tactile information across spatiotemporal domains, inspired by the sensory mechanisms of human skin. The system integrates a Convolutional Neural Network-Convolutional Neural Network-Multilayer Perceptron (CNN-CNN-MLP) architecture to fuse these dual-modal signals effectively. Furthermore, we introduce a novel Dynamic Weighting Coefficient Layer (DWCL) to dynamically optimize fusion weights for each modality based on real-time input characteristics, thereby enhancing robustness against magnetic interference. The DWCL leverages temporal discrepancies between modalities during pre-contact sensing and quantifies the magnetic field strength of target objects to autonomously adjust fusion ratios, prioritizing the more reliable modality under varying interference conditions. Extensive experimental evaluations demonstrate that the proposed DWCL significantly improves interference resistance compared to conventional fusion methods, advancing the feasibility of magnetic tactile sensing in real-world environments.

ICLR Conference 2025 Conference Paper

Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models

• Xingzhuo Guo
• Yu Zhang
• Baixu Chen
• Haoran Xu
• Jianmin Wang 0001
• Mingsheng Long

Diffusion models have emerged as powerful generative frameworks by progressively adding noise to data through a forward process and then reversing this process to generate realistic samples. While these models have achieved strong performance across various tasks and modalities, their application to temporal predictive learning remains underexplored. Existing approaches treat predictive learning as a conditional generation problem, but often fail to fully exploit the temporal dynamics inherent in the data, leading to challenges in generating temporally coherent sequences. To address this, we introduce Dynamical Diffusion (DyDiff), a theoretically sound framework that incorporates temporally aware forward and reverse processes. Dynamical Diffusion explicitly models temporal transitions at each diffusion step, establishing dependencies on preceding states to better capture temporal dynamics. Through the reparameterization trick, Dynamical Diffusion achieves efficient training and inference similar to any standard diffusion model. Extensive experiments across scientific spatiotemporal forecasting, video prediction, and time series forecasting demonstrate that Dynamical Diffusion consistently improves performance in temporal predictive tasks, filling a crucial gap in existing methodologies. Code is available at this repository: https://github.com/thuml/dynamical-diffusion.

YNICL Journal 2025 Journal Article

Early cortical alterations and neuropsychological mechanisms in amyotrophic lateral sclerosis

• Qianqian Zhang
• Yu Ding
• Yu Zhang
• Qingyang Li
• Shiyu Shi
• Yaxi Liu
• Sijie Chen
• Qian Wu

OBJECTIVE: This study investigates the characteristics of cortical structural and functional alterations in amyotrophic lateral sclerosis (ALS) patients and their modulation of emotional and cognitive functions, as well as to discuss their diagnostic value in early-stage ALS. METHODS: Fifty-nine ALS patients (28 in ALS 1 and 31 in ALS 2, categorized using King's College Staging) and 31 healthy controls were evaluated using multiparametric MRI, motor and neuropsychological assessments, and serum neurofilament light chain (NfL) levels. Mediation analyses were performed to examine how cortical alterations influence the relationship between emotional and cognitive functions. Support vector machine (SVM) classification models were constructed to assess the diagnostic utility of differential cortical parameters. RESULTS: ALS 1 patients exhibited increased cortical thickness (CT) and functional activity in the cingulate and frontotemporal regions, correlating with neuropsychological performance and NfL levels. Mediation analysis revealed that perigenual and frontotemporal functional activity significantly modulated the relationship between depressive symptoms and cognitive function. SVM classification showed that the combined altered regions with Amplitude of Low Frequency Fluctuations (ALFF) model achieved slightly better performance (AUC = 0.853, 95 %CI: 0.687-1.000, p < 0.001) compared to CT (AUC = 0.779, 95 %CI: 0.587-0.972, p < 0.001), although both models showed limited efficacy in differentiating between ALS 1 and ALS 2 groups. CONCLUSIONS: Cortical structural and functional alterations in ALS mediate the impact of depression on cognitive function, offering insights into the neuropsychological mechanisms of the disease and potential biomarkers for early-stage diagnosis.

JBHI Journal 2025 Journal Article

EF-VPT-Net: Enhanced Feature-Based Vision Patch Transformer Network for Accurate Brain Tumor Segmentation in Magnetic Resonance Imaging

• Jinru Liu
• Uzair Aslam Bhatti
• Junfeng Zhang
• Yu Zhang
• Mengxing Huang

Brain tumor segmentation and the accurate detection of tumor types remain significant challenges due to image variability, tumor heterogeneity, and boundary demarcation difficulties. These challenges are exacerbated by limited annotated datasets, class imbalances, and the high computational cost of existing methods. Our study introduces the enhanced feature (EF)-based vision patch transformer network algorithm, which uses an EF module-based customized U-Net architecture. This architecture consists of an optimized vision patch transformer network as the encoder, while the decoder phase utilizes positional embedding-based convolutional U-Net blocks to identify the segmented edges of different types of tumor images accurately. We incorporated additional layers designed for efficient processing of different block sizes with positional embedding to ensure adaptability to different image resolutions and enhance the model's generality. By using the transformer as the encoder of the U-Net, we optimized the model's ability to capture global context and long-range dependencies early in the processing pipeline, thereby improving segmentation performance. In our study, we use two public datasets. Performance indicators, such as accuracy and sensitivity, show accurate segmentation up to 99. 1%. Our research contributes to the continuous progress of medical image segmentation and provides promising insights for improving diagnosis and treatment planning in healthcare.

JBHI Journal 2025 Journal Article

Efficient Click-Based Interactive Segmentation for Medical Image With Improved Plain-ViT

• Mengxing Huang
• Jie Zou
• Yu Zhang
• Uzair Aslam Bhatti
• Jing Chen

The primary objective of interactive medical image segmentation systems is to achieve more precise segmentation outcomes with reduced human intervention. This endeavor holds significant clinical importance for both pre-diagnostic pathological assessments and prognostic recovery. Among the various interaction methods available, click-based interactions stand out as an intuitive and straightforward approach compared to alternatives such as graffiti, bounding boxes, and extreme points. To improve the model's ability to interpret click-based interactions, we propose a comprehensive interactive segmentation framework that leverages an iterative weighted loss function based on user clicks. To enhance the segmentation capabilities of the Plain-ViT backbone, we introduce a Residual Multi-Headed Self-Attention encoder with hierarchical inputs and residual connections, offering multiple perspectives on the data. This innovative architecture leads to a remarkable improvement in segmentation model performance. In this research paper, we assess the robustness of our proposed framework using a self-compiled T2-MRI image dataset of the prostate and three publicly available datasets containing images of other organs. Our experimental results convincingly demonstrate that our segmentation model surpasses existing state-of-the-art methods. Furthermore, the incorporation of an iterative loss function training strategy significantly accelerates the model's convergence rate during interactions. In the prostate dataset, we achieved an impressive Intersection over Union (IoU) score of 88. 11% and Number of Clicks(NoC) at 80% are 7. 03 clicks.

TMLR Journal 2025 Journal Article

Efficient Diffusion Models: A Survey

• Hui Shen
• Jingxuan Zhang
• Boning Xiong
• Rui Hu
• Shoufa Chen
• Zhongwei Wan
• Xin Wang
• Yu Zhang

Diffusion models have emerged as powerful generative models capable of producing high-quality contents such as images, videos, and audio, demonstrating their potential to revolutionize digital content creation. However, these capabilities come at the cost of significant computational resources and lengthy generation time, underscoring the critical need to develop efficient techniques for practical deployment. In this survey, we provide a systematic and comprehensive review of research on efficient diffusion models. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient diffusion model topics from algorithm-level, system-level, and framework perspective, respectively. We have also created a GitHub repository where we organize the papers featured in this survey at github.com/AIoT-MLSys-Lab/Efficient-Diffusion-Model-Survey. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of efficient diffusion model research and inspire them to contribute to this important and exciting field.

NeurIPS Conference 2025 Conference Paper

Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning

• Yu Zhang
• Jialei Zhou
• Xinchen Li
• Qi Zhang
• Zhongwei Wan
• Duoqian Miao
• Changwei Wang
• Longbing Cao

Current text-to-image diffusion generation typically employs complete-text conditioning. Due to the intricate syntax, diffusion transformers (DiTs) inherently suffer from a comprehension defect of complete-text captions. One-fly complete-text input either overlooks critical semantic details or causes semantic confusion by simultaneously modeling diverse semantic primitive types. To mitigate this defect of DiTs, we propose a novel split-text conditioning framework named DiT-ST. This framework converts a complete-text caption into a split-text caption, a collection of simplified sentences, to explicitly express various semantic primitives and their interconnections. The split-text caption is then injected into different denoising stages of DiT-ST in a hierarchical and incremental manner. Specifically, DiT-ST leverages Large Language Models to parse captions, extracting diverse primitives and hierarchically sorting out and constructing these primitives into a split-text input. Moreover, we partition the diffusion denoising process according to its differential sensitivities to diverse semantic primitive types and determine the appropriate timesteps to incrementally inject tokens of diverse semantic primitive types into input tokens via cross-attention. In this way, DiT-ST enhances the representation learning of specific semantic primitive types across different stages. Extensive experiments validate the effectiveness of our proposed DiT-ST in mitigating the complete-text comprehension defect. Datasets and models are available.

IJCAI Conference 2025 Conference Paper

ExVideo: Extending Video Diffusion Models via Parameter-Efficient Post-Tuning

• Zhongjie Duan
• Hong Zhang
• Wenmeng Zhou
• Cen Chen
• Yaliang Li
• Yu Zhang
• Yingda Chen

Recently, advancements in video synthesis have attracted significant attention. Video synthesis models have demonstrated the practical applicability of diffusion models in creating dynamic visual content. Despite these advancements, the extension of video lengths remains constrained by computational resources. Most existing video synthesis models are limited to generating short video clips. In this paper, we propose a novel post-tuning methodology for video synthesis models, called ExVideo. This approach is designed to enhance the capability of current video synthesis models, allowing them to produce content over extended temporal durations while incurring lower training expenditures. In particular, we design extension strategies across common temporal model architectures respectively, including 3D convolution, temporal attention, and positional embedding. To evaluate the efficacy of our proposed post-tuning approach, we trained ExSVD, an extended model based on Stable Video Diffusion model. Our approach enhances the model's capacity to generate up to 5x its original number of frames, requiring only 1. 5k GPU hours of training on a dataset comprising 40k videos. Importantly, the substantial increase in video length doesn't compromise the model's innate generalization capabilities, and the model showcases its advantages in generating videos of diverse styles and resolutions. We have released the source code and the enhanced model publicly.

AAAI Conference 2025 Conference Paper

Filling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs Without Real Data Replay

• Ruiheng Liu
• Jinyu Zhang
• Yanqi Song
• Yu Zhang
• Bailong Yang

Continual Semantic Parsing (CSP) aims to train parsers to convert natural language questions into SQL across tasks with limited annotated examples, adapting to dynamically updated databases in real-world scenarios. Previous studies mitigate this challenge by replaying historical data or employing parameter-efficient tuning (PET), but they often violate data privacy or rely on ideal continual learning settings. To address these issues, we propose a new Large Language Model (LLM)-Enhanced Continuous Semantic Parsing method, named LECSP, which alleviates forgetting while encouraging generalization, without requiring real data replay or ideal settings. Specifically, it first analyzes the commonalities and differences between tasks from the SQL syntax perspective to guide LLMs in reconstructing key memories and improving memory accuracy through calibration. Then, it uses a task-aware dual-teacher distillation framework to promote the accumulation and transfer of knowledge during sequential training. Experimental results on two CSP benchmarks show that our method significantly outperforms existing methods, even those utilizing data replay or ideal settings. Additionally, we achieve generalization performance beyond upper limits, better adapting to unseen tasks.

AAAI Conference 2025 Conference Paper

Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition

• Changwei Wang
• Shunpeng Chen
• Yukun Song
• Rongtao Xu
• Zherui Zhang
• Jiguang Zhang
• Haoran Yang
• Yu Zhang

Visual Place Recognition (VPR) is aimed at predicting the location of a query image by referencing a database of geotagged images. For VPR task, often fewer discriminative local regions in an image produce important effects while mundane background regions do not contribute or even cause perceptual aliasing because of easy overlap. However, existing methods lack precisely modeling and full exploitation of these discriminative regions. In addition, the lack of pixel-level correspondence supervision in the VPR dataset hinders further improvement of the local feature matching capability in the re-ranking stage. In this paper, we propose the Focus on Local (FoL) approach to stimulate the performance of image retrieval and re-ranking in VPR simultaneously by mining and exploiting reliable discriminative local regions in images and introducing pseudo-correlation supervision. First, we design two losses, Extraction-Aggregation Spatial Alignment Loss (SAL) and Foreground-Background Contrast Enhancement Loss (CEL), to explicitly model reliable discriminative local regions and use them to guide the generation of global representations and efficient re-ranking. Second, we introduce a weakly-supervised local feature training strategy based on pseudo-correspondences obtained from aggregating global features to alleviate the lack of local correspondences ground truth for the VPR task. Third, we suggest an efficient re-ranking pipeline that is efficiently and precisely based on discriminative region guidance. Finally, experimental results show that our FoL achieves the state-of-the-art on multiple VPR benchmarks in both image retrieval and re-ranking stages and also significantly outperforms existing two-stage VPR methods in terms of computational efficiency.

NeurIPS Conference 2025 Conference Paper

From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots

• Yuxuan Wang
• Ming Yang
• Gang Ding
• Yu Zhang
• Weishuai Zeng
• Xinrun Xu
• Haobin Jiang
• Zongqing Lu

Achieving general agile whole-body control on humanoid robots remains a major challenge due to diverse motion demands and data conflicts. While existing frameworks excel in training single motion-specific policies, they struggle to generalize across highly varied behaviors due to conflicting control requirements and mismatched data distributions. In this work, we propose BumbleBee (BB), an expert-generalist learning framework that combines motion clustering and sim-to-real adaptation to overcome these challenges. BB first leverages an autoencoder-based clustering method to group behaviorally similar motions using motion features and motion descriptions. Expert policies are then trained within each cluster and refined with real-world data through iterative delta action modeling to bridge the sim-to-real gap. Finally, these experts are distilled into a unified generalist controller that preserves agility and robustness across all motion types. Experiments on two simulations and a real humanoid robot demonstrate that BB achieves state-of-the-art general whole-body control, setting a new benchmark for agile, robust, and generalizable humanoid performance in the real world.

IJCAI Conference 2025 Conference Paper

Gaussian Mixture Model for Graph Domain Adaptation

• Mengzhu Wang
• Wenhao Ren
• Yu Zhang
• Yanlong Fan
• Dianxi Shi
• Luoxi Jing
• Nan Yin

Unsupervised domain adaptation (UDA) has been widely studied with the goal of transferring knowledge from a label-rich source domain to a related but unlabeled target domain. Most UDA techniques achieve this by reducing the feature discrepancies between the two domains to learn domain-invariant feature representations. While domain-invariant feature representations can reduce the differences between the source and target domains, excessively simplifying these differences may cause the model to overlook important domain-specific features, resulting in a decline in transfer learning effectiveness. To address this issue, this paper proposes a novel Gaussian Mixture Model for graph domain adaptation (GMM). This model effectively reduces the distributional bias between the source and target domains by modeling the distribution differences on a graph structure. GMM leverages the local structural information of the graph and the clustering capability of the Gaussian mixture model to automatically learn the latent mapping relationships between the source and target domains. To the best of our knowledge, this is the first work to introduce a Gaussian mixture model into UDA. Extensive experimental results on three standard benchmarks demonstrate that the proposed GMM algorithm outperforms state-of-the-art unsupervised domain adaptation methods in terms of performance.

ICLR Conference 2025 Conference Paper

HeadMap: Locating and Enhancing Knowledge Circuits in LLMs

• Xuehao Wang
• Liyuan Wang
• Binghuai Lin
• Yu Zhang

Large language models (LLMs), through pretraining on extensive corpora, encompass rich semantic knowledge and exhibit the potential for efficient adaptation to diverse downstream tasks. However, the intrinsic mechanisms underlying LLMs remain unexplored, limiting the efficacy of applying these models to downstream tasks. In this paper, we explore the intrinsic mechanisms of LLMs from the perspective of knowledge circuits. Specifically, considering layer dependencies, we propose a layer-conditioned locating algorithm to identify a series of attention heads, which is a knowledge circuit of some tasks. Experiments demonstrate that simply masking a small portion of attention heads in the knowledge circuit can significantly reduce the model's ability to make correct predictions. This suggests that the knowledge flow within the knowledge circuit plays a critical role when the model makes a correct prediction. Inspired by this observation, we propose a novel parameter-efficient fine-tuning method called HeadMap, which maps the activations of these critical heads in the located knowledge circuit to the residual stream by two linear layers, thus enhancing knowledge flow from the knowledge circuit in the residual stream. Extensive experiments conducted on diverse datasets demonstrate the efficiency and efficacy of the proposed method. Our code is available at https://github.com/XuehaoWangFi/HeadMap.

AAAI Conference 2025 Conference Paper

HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation

• Xiaolong Wang
• Lei Yu
• Yingying Zhang
• Jiangwei Lao
• Lixiang Ru
• Liheng Zhong
• Jingdong Chen
• Yu Zhang

Feature matching between image pairs is a fundamental problem in computer vision that drives many applications, such as SLAM. Recently, semi-dense matching approaches have achieved substantial performance enhancements and established a widely-accepted coarse-to-fine paradigm. However, the majority of existing methods focus on improving coarse feature representation rather than the fine-matching module. Prior fine-matching techniques, which rely on point-to-patch matching probability expectation or direct regression, often lack precision and do not guarantee the continuity of feature points across sequential images. To address this limitation, this paper concentrates on enhancing the fine-matching module in the semi-dense matching framework. We employ a lightweight and efficient homography estimation network to generate the perspective mapping between patches obtained from coarse matching. This patch-to-patch approach achieves the overall alignment of two patches, resulting in a higher sub-pixel accuracy by incorporating additional constraints. By leveraging the homography estimation between patches, we can achieve a dense matching result with low computational cost. Extensive experiments demonstrate that our method achieves higher accuracy compared to previous semi-dense matchers. Meanwhile, our dense matching results exhibit similar end-point-error accuracy compared to previous dense matchers while maintaining semi-dense efficiency.

ICLR Conference 2025 Conference Paper

Image Watermarks are Removable using Controllable Regeneration from Clean Noise

• Yepeng Liu
• Yiren Song
• Hai Ci
• Yu Zhang
• Haofan Wang
• Mike Zheng Shou
• Yuheng Bu

Image watermark techniques provide an effective way to assert ownership, deter misuse, and trace content sources, which has become increasingly essential in the era of large generative models. A critical attribute of watermark techniques is their robustness against various manipulations. In this paper, we introduce a watermark removal approach capable of effectively nullifying state-of-the-art watermarking techniques. Our primary insight involves regenerating the watermarked image starting from a \textbf{clean Gaussian noise} via a controllable diffusion model, utilizing the extracted semantic and spatial features from the watermarked image. The semantic control adapter and the spatial control network are specifically trained to control the denoising process towards ensuring image quality and enhancing consistency between the cleaned image and the original watermarked image. To achieve a smooth trade-off between watermark removal performance and image consistency, we further propose an adjustable and controllable regeneration scheme. This scheme adds varying numbers of noise steps to the latent representation of the watermarked image, followed by a controlled denoising process starting from this noisy latent representation. As the number of noise steps increases, the latent representation progressively approaches clean Gaussian noise, facilitating the desired trade-off. We apply our watermark removal methods across various watermarking techniques, and the results demonstrate that our methods offer superior visual consistency/quality and enhanced watermark removal performance compared to existing regeneration approaches. Our code is available at \url{https://github.com/yepengliu/CtrlRegen}.

IJCAI Conference 2025 Conference Paper

Improving Efficiency of Answer Set Planning with Rough Solutions from Large Language Models for Robotic Task Planning

• Xinrui Lin
• Yangfan Wu
• Huanyu Yang
• Yuting Huang
• Yu Zhang
• Jianmin Ji
• Yanyong Zhang

Answer Set Programming (ASP) planning can be used to refine the rough solutions generated by Large Language Models (LLMs) to handle specific restrictions of actions, i. e. , reconstruct the rough solutions to be executable, for robotic task planning. However, it is still challenging to efficiently solve ASP programs that have multiple variables with large domains, which prevents the above application of ASP planning from real-world task planning problems. In this paper, we consider how to reduce the domains of variables without losing possible solutions for ASP planning, while given these rough solutions from LLMs. Based on the above reduction, we introduce CLMASP, an approach that couples LLMs with ASP for robotic task planning. We evaluate CLMASP on the VirtualHome platform for common indoor tasks, demonstrating a significant improvement in the executable rate from under 10% to nearly 90% and reducing average ASP planning time from over 2 hours to under 5 seconds. Code is available at https: //github. com/CLMASP/CLMASP.

EAAI Journal 2025 Journal Article

Incorporating prior knowledge of collision risk into deep learning networks for ship trajectory prediction in the maritime Internet of Things industry

• Yu Zhang
• Ping Tu
• Zhiyuan Zhao
• Xuan-Yan Chen

Artificial intelligence (AI) has played a key role in advancing autonomous navigation for unmanned ships, where ship trajectory prediction is crucial for ensuring maritime safety. As the shipping industry grows and the number of ships increases, especially with autonomous ships operating in complex environments, collision risks have become a major concern. Accurate trajectory prediction, supported by advanced AI techniques, is crucial for the safe operation of these ships. While current models predict ship trajectories with high precision using Automatic Identification System (AIS) data, they often fail to incorporate prior knowledge of collision risks and struggle to model ship interactions that could lead to collisions. To overcome these limitations, the DGCN-Transformer (Dynamic Graph Convolution Network-Transformer) model is proposed. This model enhances the accuracy and reliability of ship trajectory predictions by incorporating collision risk modeling into the prediction framework. It uses the Quaternion Ship Domain (QSD) to model potential collision scenarios, integrating an advanced understanding of ships' spatial and kinematic properties. The model integrates QSD-based prior knowledge into an advanced Graph Convolutional Network (GCN) for spatial modeling, while the Transformer component captures and analyzes temporal features, overcoming the limitations of traditional Long Short-Term Memory (LSTM) networks. Experiments with AIS data from Tianjin, Caofeidian, and Chengshanjiao ports demonstrate that the DGCN-Transformer model outperforms state-of-the-art models, significantly improving trajectory prediction accuracy. Specifically, at Tianjin Port, the DGCN-Transformer model reduces Final Displacement Error (FDE) by 36. 1%, Maximum Displacement Error (MDE) by 15. 4%, and Average Displacement Error (ADE) by 50% compared to the best baseline model, highlighting the model's effectiveness in enhancing the safety of autonomous ship navigation.

ICRA Conference 2025 Conference Paper

Learning Visuotactile Skills With Two Multifingered Hands

• Toru Lin
• Yu Zhang
• Qiyang Li
• Haozhi Qi
• Brent Yi
• Sergey Levine
• Jitendra Malik

Aiming to replicate human-like dexterity, perceptual experiences, and motion patterns, we explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Two significant challenges exist: the lack of an affordable and accessible teleoperation system suitable for a dual-arm setup with multifingered hands, and the scarcity of multifingered hand hardware equipped with touch sensing. To tackle the first challenge, we develop HATO, a low-cost hands-arms teleoperation system that leverages off-the-shelf electronics, complemented with a software suite that enables efficient data collection; the comprehensive software suite also supports multimodal data processing, scalable policy learning, and smooth policy deployment. To tackle the latter challenge, we introduce a novel hardware adaptation by repurposing two prosthetic hands equipped with touch sensors for research. Using visuotactile data collected from our system, we learn skills to complete long-horizon, high-precision tasks which are difficult to achieve without multifingered dexterity and touch feedback. Furthermore, we empirically investigate the effects of dataset size, sensing modality, and visual input preprocessing on policy learning. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data. Videos, code, and datasets can be found here.

IROS Conference 2025 Conference Paper

LHMM: A Tightly-Coupled LiDAR-Inertial Hybrid-Map Matching Approach for Robust and Efficient Global Localization

• Junyuan Lu
• Qishu Wu
• Yu Zhang

LiDAR map matching (LMM) faces two key challenges: the enormous number of point clouds imposes constraints on storage and computation, and traditional two-stage frameworks suffer from initial guess errors during degeneration. This paper presents LHMM, a hybrid-map framework that first compresses the prior map and then performs tightly coupled pose estimation within a Maximum A Posteriori (MAP) estimation formulation. First, a skeletonization-based prior map compression method is proposed, which retains only stable structural features, reducing the map storage while enabling fast runtime association through a dual-mode map representation. Second, constraints from IMU, skeleton-feature prior map, and local voxel map are jointly optimized within a unified MAP formulation, recovering the full system state in a single step and preventing error cascades. The local map benefits from a hole-aware keyframe mechanism, focusing on regions with environmental changes or areas with partial map coverage, thereby reducing computation compared to full mapping. Extensive evaluations across multiple datasets demonstrate that LHMM not only reduces storage and computational overhead but also outperforms state-of-the-art methods in terms of localization accuracy and robustness. We will open-source the code 1.

NeurIPS Conference 2025 Conference Paper

MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition

• Hao Zhang
• Zhan Zhuang
• Xuehao Wang
• Xiaodong Yang
• Yu Zhang

Human Activity Recognition (HAR) with wearable sensors is challenged by limited interpretability, which significantly impacts cross-dataset generalization. To address this challenge, we propose Motion-Primitive Transformer (MoPFormer), a novel self-supervised framework that enhances interpretability by tokenizing inertial measurement unit signals into semantically meaningful motion primitives and leverages a Transformer architecture to learn rich temporal representations. MoPFormer comprises two stages. The first stage is to partition multi-channel sensor streams into short segments and quantize them into discrete ``motion primitive'' codewords, while the second stage enriches those tokenized sequences through a context-aware embedding module and then processes them with a Transformer encoder. The proposed MoPFormer can be pre-trained using a masked motion-modeling objective that reconstructs missing primitives, enabling it to develop robust representations across diverse sensor configurations. Experiments on six HAR benchmarks demonstrate that MoPFormer not only outperforms state-of-the-art methods but also successfully generalizes across multiple datasets. More importantly, the learned motion primitives significantly enhance both interpretability and cross-dataset performance by capturing fundamental movement patterns that remain consistent across similar activities, regardless of dataset origin.

NeurIPS Conference 2025 Conference Paper

MRSAudio: A Large-Scale Multimodal Recorded Spatial Audio Dataset with Refined Annotations

• Wenxiang Guo
• Changhao Pan
• Zhiyuan Zhu
• Xintong Hu
• Yu Zhang
• Li Tang
• Rui Yang
• Han Wang

Humans rely on multisensory integration to perceive spatial environments, where auditory cues enable sound source localization in three-dimensional space. Despite the critical role of spatial audio in immersive technologies such as VR/AR, most existing multimodal datasets provide only monaural audio, which limits the development of spatial audio generation and understanding. To address these challenges, we introduce MRSAudio, a large-scale multimodal spatial audio dataset designed to advance research in spatial audio understanding and generation. MRSAudio spans four distinct components: MRSLife, MRSSpeech, MRSMusic, and MRSSing, covering diverse real-world scenarios. The dataset includes synchronized binaural and ambisonic audio, exocentric and egocentric video, motion trajectories, and fine-grained annotations such as transcripts, phoneme boundaries, lyrics, scores, and prompts. To demonstrate the utility and versatility of MRSAudio, we establish five foundational tasks: audio spatialization, and spatial text to speech, spatial singing voice synthesis, spatial music generation and sound event localization and detection. Results show that MRSAudio enables high-quality spatial modeling and supports a broad range of spatial audio research. Demos and dataset access are available at https: //mrsaudio. github. io.

AAAI Conference 2025 Conference Paper

Multi-Label Ranking Loss Minimization for Matrix Completion

• Jiaxuan Li
• Xiaoyan Zhu
• Hongrui Wang
• Yu Zhang
• Xin Lai
• Jiayin Wang

The common matrix completion methods minimize the rank of the matrix to be completed in addition to the Hamming loss between the incomplete and completed matrices. The rank of matrix measures the linear relation among the vectors of matrix, which may introduce ambiguity for data recovery. To cope with this issue, we extend multi-label ranking loss into matrix completion, and employ multi-label ranking loss minimization (MLRM) in this paper to exploit the relative correlation among matrix vectors. In MLRM, the original incomplete matrix is converted into a pairwise ranking matrix, and the approximation on this newly generated matrix can be viewed as a surrogate of multi-label ranking loss to replace the Hamming loss pattern in the existing methods. Extensive experiments demonstrate that MLRM outperforms the state-of-the-art matrix completion methods in varies of applications, including movie recommendation, drug-target interaction prediction and multi-label learning.

JBHI Journal 2025 Journal Article

Multi-Objective Evolutionary Optimization Boosted Deep Neural Networks for Few-Shot Medical Segmentation With Noisy Labels

• Hanbei Li
• Yu Zhang
• Qiang Zuo

Fully-supervised deep neural networks have achieved remarkable progress in medical image segmentation, yet they heavily rely on extensive manually labeled data and exhibit inflexibility for unseen tasks. Few-shot segmentation (FSS) addresses these issues by predicting unseen classes from a few labeled support examples. However, most existing FSS models struggle to generalize to diverse target tasks distinct from training domains. Furthermore, designing promising network architectures for such tasks is expertise-intensive and laborious. In this paper, we introduce MOE-FewSeg, a novel automatic design method for FSS architectures. Specifically, we construct a U-shaped encoder-decoder search space that incorporates capabilities for information interaction and feature selection, thereby enabling architectures to leverage prior knowledge from publicly available datasets across diverse domains for improved prediction of various target tasks. Given the potential conflicts among disparate target tasks, we formulate the multi-task problem as a multi-objective optimization problem. We employ a multi-objective genetic algorithm to identify the Pareto-optimal architectures for these target tasks within this search space. Furthermore, to mitigate the impact of noisy labels due to dataset quality variations, we propose a noise-robust loss function named NRL, which encourages the model to de-emphasize larger loss values. Empirical results demonstrate that MOE-FewSeg outperforms manually designed architectures and other related approaches.

JBHI Journal 2025 Journal Article

Multi-Perturbation Consistency Learning for Semi-Supervised Medical Image Segmentation

• Zhiyuan Zhang
• Yu Zhang
• Jing Chen
• Wenlong Feng
• Zihao Zhou
• Jie Zou
• Uzair Aslam Bhatti
• Gang Wang

Existing semi-supervised learning (SSL) methods primarily rely on consistency learning to enhance model performance. However, most current approaches only validate the effectiveness of consistency learning under single perturbations, while introducing multiple perturbations may lead to the failure of consistency learning and degrade model performance. To address this issue and effectively leverage multiple perturbations for consistency learning, we propose a semi-supervised medical image segmentation method based on multi-perturbation consistency learning. Specifically, we design a cross-teaching framework integrating sparsely annotated 3D and 2D networks, introducing network perturbations through multidimensional architectures while combining strong and weak data augmentation techniques to achieve input perturbations. Furthermore, to address the instability issue in multi-perturbation consistency learning, we develop two complementary uncertainty-aware correction algorithm targeting labeled and unlabeled data. These designs effectively enhance the model's robustness to both labeled and unlabeled data, overcoming the instability problem in multi-perturbation consistency learning. To validate the proposed method, we conducted experiments on four datasets(ProstateX, HPH55, ACDC, and LA). Experimental results demonstrate that our algorithm outperforms existing methods across all validation datasets and exhibits strong generalization capabilities. This indicates that our approach can maintain excellent performance with limited annotated data while achieving efficient medical image segmentation. The project code will be made publicly available upon acceptance.

AAAI Conference 2025 Conference Paper

NaFV-Net: An Adversarial Four-view Network for Mammogram Classification

• Feng Lu
• Yuxiang Hou
• Wei Li
• Xiangying Yang
• Haibo Zheng
• Wenxi Luo
• Leqing Chen
• Yuyang Cao

Breast cancer remains a leading cause of mortality among women, with millions of new cases diagnosed annually. Early detection through screening is crucial. Using neural networks to improve the accuracy of breast cancer screening has become increasingly important. In accordance with radiologists' practices, we proposed using images from the unaffected side to create adversarial samples with critical medical implications in our adversarial learning process. By introducing beneficial perturbations, this method aims to reduce overconfidence and improve the precision and robustness of breast cancer classification. Our proposed framework is an adversarial quadruple-view classification network (NaFV-Net) incorporating images from both affected and unaffected perspectives. By comprehensively capturing local and global information and implementing adversarial learning from four mammography views, this framework allows for the fusion of features and the integration of medical principles and radiologist evaluation techniques, thus facilitating the accurate identification and characterization of breast tissues. Extensive experiments have shown the high effectiveness of our model in accurately distinguishing between benign and malignant findings, demonstrating state-of-the-art classification performance on both internal and public datasets.

IROS Conference 2025 Conference Paper

Neural-Lyapunov Fusion: Stable Dynamical System Learning for Robotic Motion Generation

• Haoyu Zhang
• Yu Zhang
• Yongxiang Zou
• Houcheng Li
• Long Cheng

Point-to-point and periodic motions are ubiquitous in the world of robotics. To master these motions, Autonomous Dynamic System (ADS) based algorithms are fundamental in the domain of Learning from Demonstration (LfD). However, these algorithms face the significant challenge of balancing precision in learning with the maintenance of system stability. This paper addresses this challenge by presenting a novel ADS algorithm that leverages neural network technology. The proposed algorithm is designed to distill essential knowledge from demonstration data, ensuring stability during the learning of both point-to-point and periodic motions. For point-to-point motions, a neural Lyapunov function is proposed to align with the provided demonstrations. In the case of periodic motions, the neural Lyapunov function is used with the transversal contraction to ensure that all generated motions converge to a stable limit cycle. The model utilizes a streamlined neural network architecture, adept at achieving dual objectives: optimizing learning accuracy while maintaining global stability. To thoroughly assess the efficacy of the proposed algorithm, rigorous evaluations are conducted using the LASA dataset and a massage robot task. The assessments were complemented by empirical validation, providing evidence of the algorithm’s performance.

ICRA Conference 2025 Conference Paper

RISED: Accurate and Efficient RGB-Colorized Mapping Using Image Selection and Point Cloud Densification

• Changjian Jiang
• Lijie Wang
• Zeyu Wan
• Ruilan Gao
• Yue Wang
• Rong Xiong
• Yu Zhang

Recent advances in robotics have underscored the critical role of colorized point clouds in enhancing environmental perception accuracy. However, conventional multisensor fusion Simultaneous Localization and Mapping (SLAM) systems typically employ all available images indiscriminately for point cloud colorization, resulting in suboptimal outcomes with blurred textures. Notably, achieving precise texture-togeometry alignment remains a challenge despite the availability of accurate pose estimation. This study introduces RISED, an advanced colorized mapping system that tackles this challenge from two perspectives: projection accuracy and distribution uniformity. For projection accuracy, we analyze the influence of camera poses on colorization and carefully select the optimal viewpoint to minimize errors. Regarding distribution uniformity, point cloud densification is applied to eliminate LiDAR scanning traces. Furthermore, a novel evaluation method is introduced to provide comprehensive assessment of colorized point clouds, filling a gap in this field. Experimental results show that our method outperforms traditional approaches in RGB-colorized mapping. Specifically, our method achieves notable improvements in projection accuracy (55. 2 %), geometric accuracy (63. 1 %), and surface coverage (30. 8 %).

NeurIPS Conference 2025 Conference Paper

SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning

• Zhongwei Wan
• Zhihao Dou
• Che Liu
• Yu Zhang
• Dongfei Cui
• Qinjian Zhao
• Hui Shen
• Jing Xiong

Multimodal large language models (MLLMs) have shown promising capabilities in reasoning tasks, yet still struggle significantly with complex problems requiring explicit self-reflection and self-correction, especially compared to their unimodal text-based counterparts. Existing reflection methods are simplistic and struggle to generate meaningful, instructive feedback, as the reasoning ability and knowledge limits of pre-trained models are largely fixed during initial training. To overcome these challenges, we propose \textit{multimodal \textbf{S}elf-\textbf{R}eflection enhanced reasoning with Group Relative \textbf{P}olicy \textbf{O}ptimization} \textbf{SRPO}, a two-stage reflection-aware reinforcement learning (RL) framework explicitly designed to enhance multimodal LLM reasoning. In the first stage, we construct a high-quality, reflection-focused dataset under the guidance of an advanced MLLM, which generates reflections based on initial responses to help the policy model to learn both reasoning and self-reflection. In the second stage, we introduce a novel reward mechanism within the GRPO framework that encourages concise and cognitively meaningful reflection while avoiding redundancy. Extensive experiments across multiple multimodal reasoning benchmarks—including MathVista, MathVision, Mathverse, and MMMU-Pro—using Qwen-2. 5-VL-7B and Qwen-2. 5-VL-32B demonstrate that SRPO significantly outperforms state-of-the-art models, achieving notable improvements in both reasoning accuracy and reflection quality.

ICML Conference 2025 Conference Paper

Strategic A/B testing via Maximum Probability-driven Two-armed Bandit

• Yu Zhang
• Shanshan Zhao
• Bokui Wan
• Jinjuan Wang
• Xiaodong Yan

Detecting a minor average treatment effect is a major challenge in large-scale applications, where even minimal improvements can have a significant economic impact. Traditional methods, reliant on normal distribution-based or expanded statistics, often fail to identify such minor effects because of their inability to handle small discrepancies with sufficient sensitivity. This work leverages a counterfactual outcome framework and proposes a maximum probability-driven two-armed bandit (TAB) process by weighting the mean volatility statistic, which controls Type I error. The implementation of permutation methods further enhances the robustness and efficacy. The established strategic central limit theorem (SCLT) demonstrates that our approach yields a more concentrated distribution under the null hypothesis and a less concentrated one under the alternative hypothesis, greatly improving statistical power. The experimental results indicate a significant improvement in the A/B testing, highlighting the potential to reduce experimental costs while maintaining high statistical power.

AAAI Conference 2025 Conference Paper

TechSinger: Technique Controllable Multilingual Singing Voice Synthesis via Flow Matching

• Wenxiang Guo
• Yu Zhang
• Changhao Pan
• Rongjie Huang
• Li Tang
• Ruiqi Li
• Zhiqing Hong
• Yongqi Wang

Singing voice synthesis has made remarkable progress in generating natural and high-quality voices. However, existing methods rarely provide precise control over vocal techniques such as intensity, mixed voice, falsetto, bubble, and breathy tones, thus limiting the expressive potential of synthetic voices. We introduce TechSinger, an advanced system for controllable singing voice synthesis that supports five languages and seven vocal techniques. TechSinger leverages a flow-matching-based generative model to produce singing voices with enhanced expressive control over various techniques. To enhance the diversity of training data, we develop a technique detection model that automatically annotates datasets with phoneme-level technique labels. Additionally, our prompt-based technique prediction model enables users to specify desired vocal attributes through natural language, offering fine-grained control over the synthesized singing. Experimental results demonstrate that TechSinger significantly enhances the expressiveness and realism of synthetic singing voices, outperforming existing methods in terms of audio quality and technique-specific control.

AIIM Journal 2025 Journal Article

TIPs: Tooth instance and pulp segmentation based on hierarchical extraction and fusion of anatomical priors from cone-beam CT

• Tao Zhong
• Yang Ning
• Xueyang Wu
• Li Ye
• Chichi Li
• Yu Zhang
• Yu Du

Accurate instance segmentation of tooth and pulp from cone-beam computed tomography (CBCT) images is essential but highly challenging due to the pulp’s small structures and indistinct boundaries. To address these critical challenges, we propose TIPs designed for Tooth Instance and Pulp segmentation. TIPs initially employs a backbone model to segment a binary mask of the tooth from CBCT images, which is then utilized to derive position prior of the tooth and shape prior of the pulp. Subsequently, we propose the Hierarchical Fusion Mamba models to leverage the strengths of both anatomical priors and CBCT images by extracting and integrating shallow and deep features from Convolution Neural Networks (CNNs) and State Space Sequence Models (SSMs), respectively. This process achieves tooth instance and pulp segmentation, which are then combined to obtain the final pulp instance segmentation. Extensive experiments on CBCT scans from 147 patients demonstrate that TIPs significantly outperforms state-of-the-art methods in terms of segmentation accuracy. Furthermore, we have encapsulated this framework into an openly accessible tool for one-click using. To our knowledge, this is the first toolbox capable of segmentation of tooth and pulp instances, with its performance validated on two external datasets comprising 59 samples from the Toothfairy2 dataset and 48 samples from the STS dataset. These results demonstrate the potential of TIPs as a practical tool to boost clinical workflows in digital dentistry, enhancing the precision and efficiency of dental diagnostics and treatment planning.

NeurIPS Conference 2024 Conference Paper

ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model

• Yiming Sun
• Fan Yu
• Shaoxiang Chen
• Yu Zhang
• Junwei Huang
• Yang Li
• Chenhui Li
• Changbo Wang

Visual object tracking aims to locate a targeted object in a video sequence based on an initial bounding box. Recently, Vision-Language~(VL) trackers have proposed to utilize additional natural language descriptions to enhance versatility in various applications. However, VL trackers are still inferior to State-of-The-Art (SoTA) visual trackers in terms of tracking performance. We found that this inferiority primarily results from their heavy reliance on manual textual annotations, which include the frequent provision of ambiguous language descriptions. In this paper, we propose ChatTracker to leverage the wealth of world knowledge in the Multimodal Large Language Model (MLLM) to generate high-quality language descriptions and enhance tracking performance. To this end, we propose a novel reflection-based prompt optimization module to iteratively refine the ambiguous and inaccurate descriptions of the target with tracking feedback. To further utilize semantic information produced by MLLM, a simple yet effective VL tracking framework is proposed and can be easily integrated as a plug-and-play module to boost the performance of both VL and visual trackers. Experimental results show that our proposed ChatTracker achieves a performance comparable to existing methods.

ICML Conference 2024 Conference Paper

CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding

• Kaiyuan Chen
• Xingzhuo Guo
• Yu Zhang
• Jianmin Wang 0001
• Mingsheng Long

Predictive Coding (PC) is a theoretical framework in cognitive science suggesting that the human brain processes cognition through spatiotemporal prediction of visual world. Existing studies have developed spatiotemporal prediction neural networks based on the PC theroy, emulating its two core mechanisms: Correcting predictions from residuals and Hierarchical learning. However, these models do not show the enhancement of prediction skills on real-world forecasting tasks, and ignore the Precision Weighting mechanism of PC theory. Precision weight posits that the brain allocates more attention to signals with lower Precision, contributing to the the cognitive ability of human brains. This work introduces the Cognitive Diffusion Probabilistic Models (CogDPM) which demonstrates the connection between diffusion probabilistic models and PC theory. CogDPM features a precision estimation method based on the hierarchical sampling capabilities of diffusion models, and allocate the guidance with precision weights estimated by the inherent property of diffusion models. We experimentally show that the precision weights is an estimator of model’s predictability on the rigid body and fluid motion dataset. We also apply CogDPM to real-world prediction tasks using the U. K. precipitation and ERA surface wind datasets. Our results demonstrate that CogDPM outperforms both existing domain-specific operational models and general deep prediction models in providing more proficient forecasting.

NeurIPS Conference 2024 Conference Paper

Dissect Black Box: Interpreting for Rule-Based Explanations in Unsupervised Anomaly Detection

• Yu Zhang
• Ruoyu Li
• Nengwu Wu
• Qing Li
• Xinhan Lin
• Yang Hu
• Tao Li
• Yong Jiang

In high-stakes sectors such as network security, IoT security, accurately distinguishing between normal and anomalous data is critical due to the significant implications for operational success and safety in decision-making. The complexity is exacerbated by the presence of unlabeled data and the opaque nature of black-box anomaly detection models, which obscure the rationale behind their predictions. In this paper, we present a novel method to interpret the decision-making processes of these models, which are essential for detecting malicious activities without labeled attack data. We put forward the Segmentation Clustering Decision Tree (SCD-Tree), designed to dissect and understand the structure of normal data distributions. The SCD-Tree integrates predictions from the anomaly detection model into its splitting criteria, enhancing the clustering process with the model's insights into anomalies. To further refine these segments, the Gaussian Boundary Delineation (GBD) algorithm is employed to define boundaries within each segmented distribution, effectively delineating normal from anomalous data points. At this point, this approach addresses the curse of dimensionality by segmenting high-dimensional data and ensures resilience to data drift and perturbations through flexible boundary fitting. We transform the intricate operations of anomaly detection into an interpretable rule's format, constructing a comprehensive set of rules for understanding. Our method's evaluation on diverse datasets and models demonstrates superior explanation accuracy, fidelity, and robustness over existing method, proving its efficacy in environments where interpretability is paramount.

EAAI Journal 2024 Journal Article

Enabling temporal–spectral decoding in multi-class single-side upper limb classification

• Hao Jia
• Shuning Han
• Cesar F. Caiafa
• Feng Duan
• Yu Zhang
• Zhe Sun
• Jordi Solé-Casals

This manuscript presents a novel approach for decoding pre-movement patterns from brain signals using a two-stage-training temporal–spectral neural network (TTSNet). The TTSNet employs a combination of filter bank task-related component analysis (FBTRCA) and convolutional neural network (CNN) techniques to enhance the classification of single-upper limb movements in non-invasive brain–computer interfaces (BCIs). In our previous work, we introduced the FBTRCA method which utilized filter banks and spatial filters to handle spectral and spatial information, respectively. However, we observed limitations in the temporal decoding phase, where correlation features failed to effectively utilize temporal information because of misaligned onset and noisy spikes. To address this issue, our proposed method focuses on analyzing multi-channel signals in the temporal–spectral domain. The TTSNet first divides the signals into various filter banks, employing task-related component analysis to reduce dimensionality and eliminate noise, respectively. Subsequently, a CNN is employed to optimize the temporal characteristics of the signals and extract class-related features. Finally, the class-related features from all filter banks are concatenated and classified using the fully connected layer. To evaluate the effectiveness of our proposed method, we conducted experiments on two publicly available datasets. In binary classification tasks, the TTSNet achieved an improved accuracy of 0. 7707 ± 0. 1168, surpassing the performance of EEGNet (accuracy: 0. 7340 ± 0. 1246) and FBTRCA (accuracy: 0. 7487 ± 0. 1250). In multi-class tasks, TTSNet achieved an accuracy of 0. 4588 ± 0. 0724, exhibiting a 4. 27% and 3. 95% accuracy increase over EEGNet and FBTRCA, respectively. The findings of this study suggest that the proposed TTSNet method holds promise for detecting limb movements and assisting in the rehabilitation of stroke patients. The classification of single-side limb movements is expected to facilitate the interaction between patients and external environment by increasing the number of control commands in BCIs.

ICRA Conference 2024 Conference Paper

ERASOR++: Height Coding Plus Egocentric Ratio Based Dynamic Object Removal for Static Point Cloud Mapping

• Jiabao Zhang
• Yu Zhang

Mapping plays a crucial role in location and navigation within automatic systems. However, the presence of dynamic objects in 3D point cloud maps generated from scan sensors can introduce map distortion and long traces, thereby posing challenges for accurate mapping and navigation. To address this issue, we propose ERASOR++, an enhanced approach based on the Egocentric Ratio of Pseudo Occupancy for effective dynamic object removal. To begin, we introduce the Height Coding Descriptor, which combines height difference and height layer information to encode the point cloud. Subsequently, we propose the Height Stack Test, Ground Layer Test, and Surrounding Point Test methods to precisely and efficiently identify the dynamic bins within point cloud bins, thus overcoming the limitations of prior approaches. Through extensive evaluation on open-source datasets, our approach demonstrates superior performance in terms of precision and efficiency compared to existing methods. Furthermore, the techniques described in our work hold promise for addressing various challenging tasks or aspects through subsequent migration.

YNICL Journal 2024 Journal Article

Functional network reorganization after endovascular thrombectomy in patients with anterior circulation stroke

• Tongyue Li
• Jiaona Xu
• Luoyu Wang
• Kang Xu
• Weiwei Chen
• Liqing Zhang
• Guozhong Niu
• Yu Zhang

BACKGROUND: Endovascular thrombectomy has been confirmed to be an effective therapy for acute ischemic stroke (AIS). However, how functional brain networks reorganize after restoration of blood supply in AIS patients, and whether the degree of reperfusion associates with functional network changes remains unclear. METHODS: Resting-state fMRI data were collected from 43 AIS patients with anterior circulation occlusion after thrombectomy and 37 healthy controls (HCs). Both static and dynamic functional connectivity (FC) within four advanced functional networks including dorsal attention network (DAN), ventral attention network (VAN), executive control network (ECN) and default mode network (DMN), were calculated and compared between post-thrombectomy patients and HCs, and between two subgroups of post-thrombectomy patients with different reperfusion conditions. RESULTS: As compared to HCs, patients showed significant differences in static FC of four functional networks, and in dynamic FC of DAN, ECN and DMN. Furthermore, patients with better reperfusion conditions exhibited increased static FC with precuneus, and altered dynamic FC within precuneus. Moreover, these alterations were associated with clinical assessments of stroke severity and functional recovery in post-thrombectomy patients. CONCLUSIONS: Collectively, these findings may provide the potential imaging markers for assessment of thrombectomy efficacy and help establish the specific rehabilitation treatments for post-thrombectomy patients.

NeurIPS Conference 2024 Conference Paper

Gated Slot Attention for Efficient Linear-Time Sequence Modeling

• Yu Zhang
• Songlin Yang
• Ruijie Zhu
• Yue Zhang
• Leyang Cui
• Yiqiao Wang
• Bolun Wang
• Freda Shi

Linear attention Transformers and their gated variants, celebrated for enabling parallel training and efficient recurrent inference, still fall short in recall-intensive tasks compared to traditional Transformers and demand significant resources for training from scratch. This paper introduces Gated Slot Attention (GSA), which enhances Attention with Bounded-memory-Control (ABC) by incorporating a gating mechanism inspired by Gated Linear Attention (GLA). Essentially, GSA comprises a two-layer GLA linked via $\operatorname{softmax}$, utilizing context-aware memory reading and adaptive forgetting to improve memory capacity while maintaining compact recurrent state size. This design greatly enhances both training and inference efficiency through GLA's hardware-efficient training algorithm and reduced state size. Additionally, retaining the $\operatorname{softmax}$ operation is particularly beneficial in ``finetuning pretrained Transformers to RNNs'' (T2R) settings, reducing the need for extensive training from scratch. Extensive experiments confirm GSA's superior performance in scenarios requiring in-context recall and in T2R settings.

NeurIPS Conference 2024 Conference Paper

GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning

• Yanbin Wei
• Shuai Fu
• Weisen Jiang
• Zejian Zhang
• Zhixiong Zeng
• Qi Wu
• James Kwok
• Yu Zhang

Large Language Models (LLMs) are increasingly used for various tasks with graph structures. Though LLMs can process graph information in a textual format, they overlook the rich vision modality, which is an intuitive way for humans to comprehend structural information and conduct general graph reasoning. The potential benefits and capabilities of representing graph structures as visual images (i. e. , $\textit{visual graph}$) are still unexplored. To fill the gap, we innovatively propose an end-to-end framework, called $\textbf{G}$raph to v$\textbf{I}$sual and $\textbf{T}$extual Integr$\textbf{A}$tion (GITA), which firstly incorporates visual graphs into general graph reasoning. Besides, we establish $\textbf{G}$raph-based $\textbf{V}$ision-$\textbf{L}$anguage $\textbf{Q}$uestion $\textbf{A}$nswering (GVLQA) dataset from existing graph data, which is the first vision-language dataset for general graph reasoning purposes. Extensive experiments on the GVLQA dataset and five real-world datasets show that GITA outperforms mainstream LLMs in terms of general graph reasoning capabilities. Moreover, We highlight the effectiveness of the layout augmentation on visual graphs and pretraining on the GVLQA dataset.

ICLR Conference 2024 Conference Paper

Gradual Domain Adaptation via Gradient Flow

• Zhan Zhuang
• Yu Zhang
• Ying Wei

Domain shift degrades classification models on new data distributions. Conventional unsupervised domain adaptation (UDA) aims to learn features that bridge labeled source and unlabeled target domains. In contrast to feature learning, gradual domain adaptation (GDA) leverages extra continuous intermediate domains with pseudo-labels to boost the source classifier. However, real intermediate domains are sometimes unavailable or ineffective. In this paper, we propose $\textbf{G}$radual Domain Adaptation via $\textbf{G}$radient $\textbf{F}$low (GGF) to generate intermediate domains with preserving labels, thereby enabling us a fine-tuning method for GDA. We employ the Wasserstein gradient flow in Kullback–Leibler divergence to transport samples from the source to the target domain. To simulate the dynamics, we utilize the Langevin algorithm. Since the Langevin algorithm disregards label information and introduces diffusion noise, we introduce classifier-based and sample-based potentials to avoid label switching and dramatic deviations in the sampling process. For the proposed GGF model, we analyze its generalization bound. Experiments on several benchmark datasets demonstrate the superiority of the proposed GGF method compared to state-of-the-art baselines.

NeurIPS Conference 2024 Conference Paper

GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks

• Yu Zhang
• Changhao Pan
• Wenxiang Guo
• Ruiqi Li
• Zhiyuan Zhu
• Jialei Wang
• Wenhao Xu
• Jingyu Lu

The scarcity of high-quality and multi-task singing datasets significantly hinders the development of diverse controllable and personalized singing tasks, as existing singing datasets suffer from low quality, limited diversity of languages and singers, absence of multi-technique information and realistic music scores, and poor task suitability. To tackle these problems, we present GTSinger, a large Global, multi-Technique, free-to-use, high-quality singing corpus with realistic music scores, designed for all singing tasks, along with its benchmarks. Particularly, (1) we collect 80. 59 hours of high-quality singing voices, forming the largest recorded singing dataset; (2) 20 professional singers across nine widely spoken languages offer diverse timbres and styles; (3) we provide controlled comparison and phoneme-level annotations of six commonly used singing techniques, helping technique modeling and control; (4) GTSinger offers realistic music scores, assisting real-world musical composition; (5) singing voices are accompanied by manual phoneme-to-audio alignments, global style labels, and 16. 16 hours of paired speech for various singing tasks. Moreover, to facilitate the use of GTSinger, we conduct four benchmark experiments: technique-controllable singing voice synthesis, technique recognition, style transfer, and speech-to-singing conversion.

NeurIPS Conference 2024 Conference Paper

IDGen: Item Discrimination Induced Prompt Generation for LLM Evaluation

• Fan Lin
• Shuyi Xie
• Yong Dai
• Wenlin Yao
• Tianjiao Lang
• Yu Zhang

As Large Language Models (LLMs) become more capable of handling increasingly complex tasks, the evaluation set must keep pace with these advancements to ensure it remains sufficiently discriminative. Item Discrimination (ID) theory, which is widely used in educational assessment, measures the ability of individual test items to differentiate between high and low performers. Inspired by this theory, we propose an ID-induced prompt synthesis framework for evaluating LLMs so that the evaluation set continually updates and refines according to model abilities. Our data synthesis framework prioritizes both breadth and specificity. It can generate prompts that comprehensively evaluate the capabilities of LLMs while revealing meaningful performance differences between models, allowing for effective discrimination of their relative strengths and weaknesses across various tasks and domains. To produce high-quality data, we incorporate a self-correct mechanism into our generalization framework and develop two models to predict prompt discrimination and difficulty score to facilitate our data synthesis framework, contributing valuable tools to evaluation data synthesis research. We apply our generated data to evaluate five SOTA models. Our data achieves an average score of 51. 92, accompanied by a variance of 10. 06. By contrast, previous works (i. e. , SELF-INSTRUCT and WizardLM) obtain an average score exceeding 67, with a variance below 3. 2. The results demonstrate that the data generated by our framework is more challenging and discriminative compared to previous works. We will release a dataset of over 3, 000 carefully crafted prompts to facilitate evaluation research of LLMs.

ICRA Conference 2024 Conference Paper

Learning a Stable Dynamic System with a Lyapunov Energy Function for Demonstratives Using Neural Networks

• Yu Zhang
• Yongxiang Zou
• Houcheng Li
• Haoyu Zhang
• Long Cheng

Autonomous Dynamic System (DS)-based algorithms hold a pivotal and foundational role in the field of Learning from Demonstration (LfD). Nevertheless, they confront the formidable challenge of striking a delicate balance between achieving precision in learning and ensuring the overall stability of the system. In response to this substantial challenge, this paper introduces a novel DS algorithm rooted in neural network technology. This algorithm not only possesses the capability to extract critical insights from demonstration data but also demonstrates the capacity to learn a candidate Lyapunov energy function that is consistent with the provided demonstrations. The model presented in this paper employs a simplistic neural network architecture that excels in fulfilling a dual objective: optimizing accuracy while simultaneously preserving global stability. To comprehensively evaluate the effectiveness of the proposed algorithm, rigorous assessments are conducted using the LASA dataset, further reinforced by empirical validation through a robotic experiment.

EAAI Journal 2024 Journal Article

Mask-guided dual-perception generative adversarial network for synthesizing complex maize diseased leaves to augment datasets

• Zhiliang Zhang
• Wei Zhan
• Yong Sun
• Jinling Peng
• Yu Zhang
• Yuheng Guo
• Kanglin Sun
• Lianyou Gui

In practice, acquiring and annotating data in specialized domains can be costly, thereby constraining the performance and applicability of deep learning. Utilizing generative models to synthesize data proves to be an effective augmentation technique. Therefore, this research proposes a diseased leaf generation pipeline to diversify the maize disease datasets. We introduce the Dual-Perception Cycle-Consistent Generative Adversarial Network (DP-CycleGAN). During training, incorporating our proposed Structure Perception (SP) loss and Texture Perception (TP) loss functions. These losses guide the model's attention areas through activation reconstruction and mask mechanisms, thereby improving the overall perceptual quality of the generated images and the realism of the disease lesions. We constructed a maize leaf mixed disease dataset to simulate the complex conditions of real-world disease occurrence. Experimental results show that the DP-CycleGAN generates higher-quality and more realistic diseased leaf images. Compared to CycleGAN and state-of-the-art method, DP-CycleGAN shows a 29. 6% and 15. 7% reduction in Fréchet Inception Distance (FID) scores and a 125. 3% and 61. 5% increase in Structural Similarity (SSIM) values, respectively. Simultaneously, by incorporating synthetic data during training, our approach significantly enhances the performance of the recognition model in scenarios of both data abundance and scarcity, with improvement rates exceeding two times those of existing state-of-the-art methods. This contributes to the application of Artificial Intelligence (AI) in agricultural production practices.

AAAI Conference 2024 Conference Paper

Memory-Efficient Reversible Spiking Neural Networks

• Hong Zhang
• Yu Zhang

Spiking neural networks (SNNs) are potential competitors to artificial neural networks (ANNs) due to their high energy-efficiency on neuromorphic hardware. However, SNNs are unfolded over simulation time steps during the training process. Thus, SNNs require much more memory than ANNs, which impedes the training of deeper SNN models. In this paper, we propose the reversible spiking neural network to reduce the memory cost of intermediate activations and membrane potentials during training. Firstly, we extend the reversible architecture along temporal dimension and propose the reversible spiking block, which can reconstruct the computational graph and recompute all intermediate variables in forward pass with a reverse process. On this basis, we adopt the state-of-the-art SNN models to the reversible variants, namely reversible spiking ResNet (RevSResNet) and reversible spiking transformer (RevSFormer). Through experiments on static and neuromorphic datasets, we demonstrate that the memory cost per image of our reversible SNNs does not increase with the network depth. On CIFAR10 and CIFAR100 datasets, our RevSResNet37 and RevSFormer-4-384 achieve comparable accuracies and consume 3.79x and 3.00x lower GPU memory per image than their counterparts with roughly identical model complexity and parameters. We believe that this work can unleash the memory constraints in SNN training and pave the way for training extremely large and deep SNNs.

AIJ Journal 2024 Journal Article

Multi-objective meta-learning

• Feiyang Ye
• Baijiong Lin
• Zhixiong Yue
• Yu Zhang
• Ivor W. Tsang

Meta-learning has arisen as a powerful tool for many machine learning problems. With multiple factors to be considered when designing learning models for real-world applications, meta-learning with multiple objectives has attracted much attention recently. However, existing works either linearly combine multiple objectives into one objective or adopt evolutionary algorithms to handle it, where the former approach needs to pay high computational cost to tune the combination coefficients while the latter approach is computationally heavy and incapable to be integrated into gradient-based optimization. To alleviate those limitations, in this paper, we aim to propose a generic gradient-based Multi-Objective Meta-Learning (MOML) framework with applications in many machine learning problems. Specifically, the MOML framework formulates the objective function of meta-learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta-learner. Different from those existing works, in this paper, we propose a gradient-based algorithm to solve the MOBLP. Specifically, we devise the first gradient-based optimization algorithm by alternately solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence property and provide a non-asymptotic analysis of the proposed gradient-based optimization algorithm. Empirically, extensive experiments justify our theoretical results and demonstrate the superiority of the proposed MOML framework for different learning problems, including few-shot learning, domain adaptation, multi-task learning, neural architecture search, and reinforcement learning. The source code of MOML is available at https: //github. com/Baijiong-Lin/MOML.

YNIMG Journal 2024 Journal Article

nBEST: Deep-learning-based non-human primates Brain Extraction and Segmentation Toolbox across ages, sites and species

• Tao Zhong
• Xueyang Wu
• Shujun Liang
• Zhenyuan Ning
• Li Wang
• Yuyu Niu
• Shihua Yang
• Zhuang Kang

Accurate processing and analysis of non-human primate (NHP) brain magnetic resonance imaging (MRI) serves an indispensable role in understanding brain evolution, development, aging, and diseases. Despite the accumulation of diverse NHP brain MRI datasets at various developmental stages and from various imaging sites/scanners, existing computational tools designed for human MRI typically perform poor on NHP data, due to huge differences in brain sizes, morphologies, and imaging appearances across species, sites, and ages, highlighting the imperative for NHP-specialized MRI processing tools. To address this issue, in this paper, we present a robust, generic, and fully automated computational pipeline, called non-human primates Brain Extraction and Segmentation Toolbox (nBEST), whose main functionality includes brain extraction, non-cerebrum removal, and tissue segmentation. Building on cutting-edge deep learning techniques by employing lifelong learning to flexibly integrate data from diverse NHP populations and innovatively constructing 3D U-NeXt architecture, nBEST can well handle structural NHP brain MR images from multi-species, multi-site, and multi-developmental-stage (from neonates to the elderly). We extensively validated nBEST based on, to our knowledge, the largest assemblage dataset in NHP brain studies, encompassing 1,469 scans with 11 species (e.g., rhesus macaques, cynomolgus macaques, chimpanzees, marmosets, squirrel monkeys, etc.) from 23 independent datasets. Compared to alternative tools, nBEST outperforms in precision, applicability, robustness, comprehensiveness, and generalizability, greatly benefiting downstream longitudinal, cross-sectional, and cross-species quantitative analyses. We have made nBEST an open-source toolbox (https://github.com/TaoZhong11/nBEST) and we are committed to its continual refinement through lifelong learning with incoming data to greatly contribute to the research field.

NeurIPS Conference 2024 Conference Paper

NeuroClips: Towards High-fidelity and Smooth fMRI-to-Video Reconstruction

• Zixuan Gong
• Guangyin Bao
• Qi Zhang
• Zhongwei Wan
• Duoqian Miao
• Shoujin Wang
• Lei Zhu
• Changwei Wang

Reconstruction of static visual stimuli from non-invasion brain activity fMRI achieves great success, owning to advanced deep learning models such as CLIP and Stable Diffusion. However, the research on fMRI-to-video reconstruction remains limited since decoding the spatiotemporal perception of continuous visual experiences is formidably challenging. We contend that the key to addressing these challenges lies in accurately decoding both high-level semantics and low-level perception flows, as perceived by the brain in response to video stimuli. To the end, we propose NeuroClips, an innovative framework to decode high-fidelity and smooth video from fMRI. NeuroClips utilizes a semantics reconstructor to reconstruct video keyframes, guiding semantic accuracy and consistency, and employs a perception reconstructor to capture low-level perceptual details, ensuring video smoothness. During inference, it adopts a pre-trained T2V diffusion model injected with both keyframes and low-level perception flows for video reconstruction. Evaluated on a publicly available fMRI-video dataset, NeuroClips achieves smooth high-fidelity video reconstruction of up to 6s at 8FPS, gaining significant improvements over state-of-the-art models in various metrics, e. g. , a 128% improvement in SSIM and an 81% improvement in spatiotemporal metrics. Our project is available at https: //github. com/gongzix/NeuroClips.

NeurIPS Conference 2024 Conference Paper

Parallelizing Linear Transformers with the Delta Rule over Sequence Length

• Songlin Yang
• Bailin Wang
• Yu Zhang
• Yikang Shen
• Yoon Kim

Transformers with linear attention (i. e. , linear transformers) and state-space models have recently been suggested as a viable linear-time alternative to transformers with softmax attention. However, these models still underperform transformers especially on tasks that require in-context retrieval. While more expressive variants of linear transformers which replace the additive update in linear transformers with the delta rule (DeltaNet) have been found to be more effective at associative recall, existing algorithms for training such models do not parallelize over sequence length and are thus inefficient to train on modern hardware. This work describes a hardware-efficient algorithm for training linear transformers with the delta rule, which exploits a memory-efficient representation for computing products of Householder matrices. This algorithm allows us to scale up DeltaNet to standard language modeling settings. We train a 1. 3B model for 100B tokens and find that it outperforms recent linear-time baselines such as Mamba and GLA in terms of perplexity and zero-shot performance on downstream tasks. We also experiment with two hybrid models which combine DeltaNet layers with (1) sliding-window attention layers every other layer or (2) two global attention layers, and find that these hybrids outperform strong transformer baselines.

IJCAI Conference 2024 Conference Paper

Personalized Federated Learning for Cross-City Traffic Prediction

• Yu Zhang
• Hua Lu
• Ning Liu
• Yonghui Xu
• Qingzhong Li
• Lizhen Cui

Traffic prediction plays an important role in urban computing. However, many cities face data scarcity due to low levels of urban development. Although many approaches transfer knowledge from data-rich cities to data-scarce cities, the centralized training paradigm cannot uphold data privacy. For the sake of inter-city data privacy, Federated Learning has been used, which follows a decentralized training paradigm to enhance traffic knowledge of data-scarce cities. However, spatio-temporal data heterogeneity causes client drift, leading to unsatisfactory traffic prediction performance. In this work, we propose a novel personalized Federated learning method for Cross-city Traffic Prediction (pFedCTP). It learns traffic knowledge from multiple data-rich source cities and transfers the knowledge to a data-scarce target city while preserving inter-city data privacy. In the core of pFedCTP lies a Spatio-Temporal Neural Network (ST-Net) for clients to learn traffic representation. We decouple the ST-Net to learn space-independent traffic patterns to overcome cross-city spatial heterogeneity. Besides, pFedCTP adaptively interpolates the layer-wise global and local parameters to deal with temporal heterogeneity across cities. Extensive experiments on four real-world traffic datasets demonstrate significant advantages of pFedCTP over representative state-of-the-art methods.

NeurIPS Conference 2024 Conference Paper

RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models

• Shuhao Chen
• Weisen Jiang
• Baijiong Lin
• James Kwok
• Yu Zhang

Recent works show that assembling multiple off-the-shelf large language models (LLMs) can harness their complementary abilities. To achieve this, routing is a promising method, which learns a router to select the most suitable LLM for each query. However, existing routing models are ineffective when multiple LLMs perform well for a query. To address this problem, in this paper, we propose a method called query-based Router by Dual Contrastive learning (RouterDC). The RouterDC model, which consists of an encoder and LLM embeddings, is trained by two proposed contrastive losses (sample-LLM and sample-sample losses). Experimental results show that RouterDC is effective in assembling LLMs and largely outperforms individual top-performing LLMs as well as existing routing methods on both in-distribution (+2. 76\%) and out-of-distribution (+1. 90\%) tasks. The source code is available at https: //github. com/shuhao02/RouterDC.

AAAI Conference 2024 Conference Paper

SDAC: A Multimodal Synthetic Dataset for Anomaly and Corner Case Detection in Autonomous Driving

• Lei Gong
• Yu Zhang
• Yingqing Xia
• Yanyong Zhang
• Jianmin Ji

Nowadays, closed-set perception methods for autonomous driving perform well on datasets containing normal scenes. However, they still struggle to handle anomalies in the real world, such as unknown objects that have never been seen while training. The lack of public datasets to evaluate the model performance on anomaly and corner cases has hindered the development of reliable autonomous driving systems. Therefore, we propose a multimodal Synthetic Dataset for Anomaly and Corner case detection, called SDAC, which encompasses anomalies captured from multi-view cameras and the LiDAR sensor, providing a rich set of annotations for multiple mainstream perception tasks. SDAC is the first public dataset for autonomous driving that categorizes anomalies into object, scene, and scenario levels, allowing the evaluation under different anomalous conditions. Experiments show that closed-set models suffer significant performance drops on anomaly subsets in SDAC. Existing anomaly detection methods fail to achieve satisfactory performance, suggesting that anomaly detection remains a challenging problem. We anticipate that our SDAC dataset could foster the development of safe and reliable systems for autonomous driving.

AAAI Conference 2024 Conference Paper

Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains

• Yu Zhang
• Yunyi Zhang
• Yanzhen Shen
• Yu Deng
• Lucian Popa
• Larisa Shwartz
• ChengXiang Zhai
• Jiawei Han

Accurately typing entity mentions from text segments is a fundamental task for various natural language processing applications. Many previous approaches rely on massive human-annotated data to perform entity typing. Nevertheless, collecting such data in highly specialized science and engineering domains (e.g., software engineering and security) can be time-consuming and costly, without mentioning the domain gaps between training and inference data if the model needs to be applied to confidential datasets. In this paper, we study the task of seed-guided fine-grained entity typing in science and engineering domains, which takes the name and a few seed entities for each entity type as the only supervision and aims to classify new entity mentions into both seen and unseen types (i.e., those without seed entities). To solve this problem, we propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus using the contextualized representations of pre-trained language models. It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types. Extensive experiments on two datasets covering four domains demonstrate the effectiveness of SEType in comparison with various baselines. Code and data are available at: https://github.com/yuzhimanhua/SEType.

ICRA Conference 2024 Conference Paper

Self-Sensing Feedback Control of an Electrohydraulic Robotic Shoulder

• Clemens C. Christoph
• Amirhossein Kazemipour
• Michel R. Vogt
• Yu Zhang
• Robert K. Katzschmann

The human shoulder, with its glenohumeral joint, tendons, ligaments, and muscles, allows for the execution of complex tasks with precision and efficiency. However, current robotic shoulder designs lack the compliance and compactness inherent in their biological counterparts. A major limitation of these designs is their reliance on external sensors like rotary encoders, which restrict mechanical joint design and introduce bulk to the system. To address this constraint, we present a bio-inspired antagonistic robotic shoulder with two degrees of freedom powered by self-sensing hydraulically amplified self-healing electrostatic actuators. Our artificial muscle design decouples the high-voltage electrostatic actuation from the pair of low-voltage self-sensing electrodes. This approach allows for proprioceptive feedback control of trajectories in the task space while eliminating the necessity for any additional sensors. We assess the platform’s efficacy by comparing it to a feedback control based on position data provided by a motion capture system. The study demonstrates closed-loop controllable robotic manipulators based on an inherent self-sensing capability of electrohydraulic actuators. The proposed architecture can serve as a basis for complex musculoskeletal joint arrangements.

AAAI Conference 2024 Conference Paper

StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis

• Yu Zhang
• Rongjie Huang
• Ruiqi Li
• JinZheng He
• Yan Xia
• Feiyang Chen
• Xinyu Duan
• Baoxing Huai

Style transfer for out-of-domain (OOD) singing voice synthesis (SVS) focuses on generating high-quality singing voices with unseen styles (such as timbre, emotion, pronunciation, and articulation skills) derived from reference singing voice samples. However, the endeavor to model the intricate nuances of singing voice styles is an arduous task, as singing voices possess a remarkable degree of expressiveness. Moreover, existing SVS methods encounter a decline in the quality of synthesized singing voices in OOD scenarios, as they rest upon the assumption that the target vocal attributes are discernible during the training phase. To overcome these challenges, we propose StyleSinger, the first singing voice synthesis model for zero-shot style transfer of out-of-domain reference singing voice samples. StyleSinger incorporates two critical approaches for enhanced effectiveness: 1) the Residual Style Adaptor (RSA) which employs a residual quantization module to capture diverse style characteristics in singing voices, and 2) the Uncertainty Modeling Layer Normalization (UMLN) to perturb the style attributes within the content representation during the training phase and thus improve the model generalization. Our extensive evaluations in zero-shot style transfer undeniably establish that StyleSinger outperforms baseline models in both audio quality and similarity to the reference singing voice samples. Access to singing voice samples can be found at https://stylesinger.github.io/.

NeurIPS Conference 2024 Conference Paper

Time-Varying LoRA: Towards Effective Cross-Domain Fine-Tuning of Diffusion Models

• Zhan Zhuang
• Yulong Zhang
• Xuehao Wang
• Jiangang Lu
• Ying Wei
• Yu Zhang

Large-scale diffusion models are adept at generating high-fidelity images and facilitating image editing and interpolation. However, they have limitations when tasked with generating images in dynamic, evolving domains. In this paper, we introduce Terra, a novel Time-varying low-rank adapter that offers a fine-tuning framework specifically tailored for domain flow generation. The key innovation of Terra lies in its construction of a continuous parameter manifold through a time variable, with its expressive power analyzed theoretically. This framework not only enables interpolation of image content and style but also offers a generation-based approach to address the domain shift problems in unsupervised domain adaptation and domain generalization. Specifically, Terra transforms images from the source domain to the target domain and generates interpolated domains with various styles to bridge the gap between domains and enhance the model generalization, respectively. We conduct extensive experiments on various benchmark datasets, empirically demonstrate the effectiveness of Terra. Our source code is publicly available on https: //github. com/zwebzone/terra.

YNIMG Journal 2024 Journal Article

Voxel-based texture similarity networks reveal individual variability and correlate with biological ontologies

• Liyuan Lin
• Zhongyu Chang
• Yu Zhang
• Kaizhong Xue
• Yingying Xie
• Luli Wei
• Xin Li
• Zhen Zhao

The human brain is organized as a complex, hierarchical network. However, the structural covariance patterns among brain regions and the underlying biological substrates of such covariance networks remain to be clarified. The present study proposed a novel individualized structural covariance network termed voxel-based texture similarity networks (vTSNs) based on 76 refined voxel-based textural features derived from structural magnetic resonance images. Validated in three independent longitudinal healthy cohorts (40, 23, and 60 healthy participants, respectively) with two common brain atlases, we found that the vTSN could robustly resolve inter-subject variability with high test-retest reliability. In contrast to the regional-based texture similarity networks (rTSNs) that calculate radiomic features based on region-of-interest information, vTSNs had higher inter- and intra-subject variability ratios and test-retest reliability in connectivity strength and network topological properties. Moreover, the Spearman correlation indicated a stronger association of the gene expression similarity network (GESN) with vTSNs than with rTSNs (vTSN: r = 0.600, rTSN: r = 0.433, z = 39.784, P < 0.001). Hierarchical clustering identified 3 vTSN subnets with differential association patterns with 13 coexpression modules, 16 neurotransmitters, 7 electrophysiology, 4 metabolism, and 2 large-scale structural and 4 functional organization maps. Moreover, these subnets had unique biological hierarchical organization from the subcortex-limbic system to the ventral neocortex and then to the dorsal neocortex. Based on 424 unrelated, qualified healthy subjects from the Human Connectome Project, we found that vTSNs could sensitively represent sex differences, especially for connections in the subcortex-limbic system and between the subcortex-limbic system and the ventral neocortex. Moreover, a multivariate variance component model revealed that vTSNs could explain a significant proportion of inter-subject behavioral variance in cognition (80.0 %) and motor functions (63.4 %). Finally, using 494 healthy adults (aged 19-80 years old) from the Southwest University Adult Lifespan Dataset, the Spearman correlation identified a significant association between aging and vTSN strength, especially within the subcortex-limbic system and between the subcortex-limbic system and the dorsal neocortex. In summary, our proposed vTSN is robust in uncovering individual variability and neurobiological brain processes, which can serve as biologically plausible measures for linking biological processes and human behavior.

YNIMG Journal 2024 Journal Article

XDL-ESI: Electrophysiological Sources Imaging via explainable deep learning framework with validation on simultaneous EEG and iEEG

• Meng Jiao
• Xiaochen Xian
• Boyu Wang
• Yu Zhang
• Shihao Yang
• Spencer Chen
• Hai Sun
• Feng Liu

Electroencephalography (EEG) or Magnetoencephalography (MEG) source imaging aims to estimate the underlying activated brain sources to explain the observed EEG/MEG recordings. Solving the inverse problem of EEG/MEG Source Imaging (ESI) is challenging due to its ill-posed nature. To achieve a unique solution, it is essential to apply sophisticated regularization constraints to restrict the solution space. Traditionally, the design of regularization terms is based on assumptions about the spatiotemporal structure of the underlying source dynamics. In this paper, we propose a novel paradigm for ESI via an Explainable Deep Learning framework, termed as XDL-ESI, which connects the iterative optimization algorithm with deep learning architecture by unfolding the iterative updates with neural network modules. The proposed framework has the advantages of (1) establishing a data-driven approach to model the source solution structure instead of using hand-crafted regularization terms; (2) improving the robustness of source solutions by introducing a topological loss that leverages the geometric spatial information applying varying penalties on distinct localization errors; (3) improving the reconstruction efficiency and interpretability as it inherits the advantages from both the iterative optimization algorithms (interpretability) and deep learning approaches (function approximation). The proposed XDL-ESI framework provides an efficient, accurate, and interpretable paradigm to solve the ESI inverse problem with satisfactory performance in both simulated data and real clinical data. Specially, this approach is further validated using simultaneous EEG and intracranial EEG (iEEG).

TCS Journal 2023 Journal Article

A fault diagnosis method to defend scapegoating attack in network tomography

• Xiaojia Xu
• Yongcai Wang
• Yu Zhang
• Deying Li

The scapegoating attack can cause persistent and inconspicuous performance degradation in network tomography. Defense of scapegoating attack is therefore a critical problem. Theoretically, the ideal defending scheme is to add monitoring paths to make all the links in the network be identifiable. This requires very high monitoring cost, which is unaffordable. To overcome this problem, this paper proposes a diagnosis-based defending scheme for scapegoating attack, which diagnoses scapegoating attack when problematic links are detected by network tomography. The latent fact is that a scapegoating attack can be launched only when the link set manipulated by the attacker cuts the probing paths going through the scapegoat links and is not traversed by any monitoring path. This cut set is called unobserved cut set (UCS). To defense, we propose to find the UCS and add the minimum number of probing paths to traverse the UCS, so that the condition of scapegoating attack is broken and the attacking links can be detected if any scapegoating attack exists. A minimum set cover model is proposed to select the least number of defense links to cover the UCS, and a polynomial time algorithm is proposed to generate the least number of probing paths to go through the selected defense links. Evaluations on various network dataset show the effectiveness of the proposed attack and defense strategies.

NeurIPS Conference 2023 Conference Paper

CluB: Cluster Meets BEV for LiDAR-Based 3D Object Detection

• Yingjie Wang
• Jiajun Deng
• Yuenan Hou
• Yao Li
• Yu Zhang
• Jianmin Ji
• Wanli Ouyang
• Yanyong Zhang

Currently, LiDAR-based 3D detectors are broadly categorized into two groups, namely, BEV-based detectors and cluster-based detectors. BEV-based detectors capture the contextual information from the Bird's Eye View (BEV) and fill their center voxels via feature diffusion with a stack of convolution layers, which, however, weakens the capability of presenting an object with the center point. On the other hand, cluster-based detectors exploit the voting mechanism and aggregate the foreground points into object-centric clusters for further prediction. In this paper, we explore how to effectively combine these two complementary representations into a unified framework. Specifically, we propose a new 3D object detection framework, referred to as CluB, which incorporates an auxiliary cluster-based branch into the BEV-based detector by enriching the object representation at both feature and query levels. Technically, CluB is comprised of two steps. First, we construct a cluster feature diffusion module to establish the association between cluster features and BEV features in a subtle and adaptive fashion. Based on that, an imitation loss is introduced to distill object-centric knowledge from the cluster features to the BEV features. Second, we design a cluster query generation module to leverage the voting centers directly from the cluster branch, thus enriching the diversity of object queries. Meanwhile, a direction loss is employed to encourage a more accurate voting center for each cluster. Extensive experiments are conducted on Waymo and nuScenes datasets, and our CluB achieves state-of-the-art performance on both benchmarks.

TIST Journal 2023 Journal Article

COMET: Convolutional Dimension Interaction for Collaborative Filtering

• Zhuoyi Lin
• Lei Feng
• Xingzhi Guo
• Yu Zhang
• Rui Yin
• Chee Keong Kwoh
• Chi Xu

Representation learning-based recommendation models play a dominant role among recommendation techniques. However, most of the existing methods assume both historical interactions and embedding dimensions are independent of each other, and thus regrettably ignore the high-order interaction information among historical interactions and embedding dimensions. In this article, we propose a novel representation learning-based model called COMET ( CO nvolutional di M E nsion in T eraction), which simultaneously models the high-order interaction patterns among historical interactions and embedding dimensions. To be specific, COMET stacks the embeddings of historical interactions horizontally at first, which results in two “embedding maps”. In this way, internal interactions and dimensional interactions can be exploited by convolutional neural networks (CNN) with kernels of different sizes simultaneously. A fully connected multi-layer perceptron (MLP) is then applied to obtain two interaction vectors. Lastly, the representations of users and items are enriched by the learnt interaction vectors, which can further be used to produce the final prediction. Extensive experiments and ablation studies on various public implicit feedback datasets clearly demonstrate the effectiveness and rationality of our proposed method.

NeurIPS Conference 2023 Conference Paper

Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference

• Tao Lei
• Junwen Bai
• Siddhartha Brahma
• Joshua Ainslie
• Kenton Lee
• Yanqi Zhou
• Nan Du
• Vincent Zhao

We propose Conditional Adapter (CoDA), a parameter-efficient transfer learning method that also improves inference efficiency. CoDA generalizes beyond standard adapter approaches to enable a new way of balancing speed and accuracy using conditional computation. Starting with an existing dense pretrained model, CoDA adds sparse activation together with a small number of new parameters and a light-weight training phase. Our experiments demonstrate that the CoDA approach provides an unexpectedly efficient way to transfer knowledge. Across a variety of language, vision, and speech tasks, CoDA achieves a 2x to 8x inference speed-up compared to the state-of-the-art Adapter approaches with moderate to no accuracy loss and the same parameter efficiency.

AAAI Conference 2023 Conference Paper

Denoising Pre-training for Machine Translation Quality Estimation with Curriculum Learning

• Xiang Geng
• Yu Zhang
• Jiahuan Li
• Shujian Huang
• Hao Yang
• Shimin Tao
• Yimeng Chen
• Ning Xie

Quality estimation (QE) aims to assess the quality of machine translations when reference translations are unavailable. QE plays a crucial role in many real-world applications of machine translation. Because labeled QE data are usually limited in scale, recent research, such as DirectQE, pre-trains QE models with pseudo QE data and obtains remarkable performance. However, there tends to be inevitable noise in the pseudo data, hindering models from learning QE accurately. Our study shows that the noise mainly comes from the differences between pseudo and real translation outputs. To handle this problem, we propose CLQE, a denoising pre-training framework for QE based on curriculum learning. More specifically, we propose to measure the degree of noise in the pseudo QE data with some metrics based on statistical or distributional features. With the guidance of these metrics, CLQE gradually pre-trains the QE model using data from cleaner to noisier. Experiments on various benchmarks reveal that CLQE outperforms DirectQE and other strong baselines. We also show that with our framework, pre-training converges faster than directly using the pseudo data. We make our CLQE code available (https://github.com/NJUNLP/njuqe).

AAAI Conference 2023 Conference Paper

Electrophysiological Brain Source Imaging via Combinatorial Search with Provable Optimality

• Guihong Wan
• Meng Jiao
• Xinglong Ju
• Yu Zhang
• Haim Schweitzer
• Feng Liu

Electrophysiological Source Imaging (ESI) refers to reconstructing the underlying brain source activation from non-invasive Electroencephalography (EEG) and Magnetoencephalography (MEG) measurements on the scalp. Estimating the source locations and their extents is a fundamental tool in clinical and neuroscience applications. However, the estimation is challenging because of the ill-posedness and high coherence in the leadfield matrix as well as the noise in the EEG/MEG data. In this work, we proposed a combinatorial search framework to address the ESI problem with a provable optimality guarantee. Specifically, by exploiting the graph neighborhood information in the brain source space, we converted the ESI problem into a graph search problem and designed a combinatorial search algorithm under the framework of A* to solve it. The proposed algorithm is guaranteed to give an optimal solution to the ESI problem. Experimental results on both synthetic data and real epilepsy EEG data demonstrated that the proposed algorithm could faithfully reconstruct the source activation in the brain.

NeurIPS Conference 2023 Conference Paper

Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator

• Xiaolong Wang
• Runsen Xu
• Zhuofan Cui
• Zeyu Wan
• Yu Zhang

In this paper, we introduce a novel approach to fine-grained cross-view geo-localization. Our method aligns a warped ground image with a corresponding GPS-tagged satellite image covering the same area using homography estimation. We first employ a differentiable spherical transform, adhering to geometric principles, to accurately align the perspective of the ground image with the satellite map. This transformation effectively places ground and aerial images in the same view and on the same plane, reducing the task to an image alignment problem. To address challenges such as occlusion, small overlapping range, and seasonal variations, we propose a robust correlation-aware homography estimator to align similar parts of the transformed ground image with the satellite image. Our method achieves sub-pixel resolution and meter-level GPS accuracy by mapping the center point of the transformed ground image to the satellite image using a homography matrix and determining the orientation of the ground camera using a point above the central axis. Operating at a speed of 30 FPS, our method outperforms state-of-the-art techniques, reducing the mean metric localization error by 21. 3\% and 32. 4\% in same-area and cross-area generalization tasks on the VIGOR benchmark, respectively, and by 34. 4\% on the KITTI benchmark in same-area evaluation.

AAMAS Conference 2023 Conference Paper

From Abstractions to Grounded Languages for Robust Coordination of Task Planning Robots

• Yu Zhang

Individual robots in distributed systems must often coordinate to optimize the global performance. Where explicit coordination via communication is concerned, it is almost always achieved via a predefined “language” designed by human users. Such hand-designed languages tend to be either too rigid or too forgiving, leading to brittle solutions, excess negotiation costs, or unexpected coordination issues (e. g. , deadlocks). In this paper, we consider a first step to bridge the gap for task planning robots using symbolic planning. Specifically, we study the automatic construction of languages that are maximally flexible while being sufficiently explicative for coordination. To this end, we view language as a machinery for specifying temporal-state constraints of plans. Such a view enables us to reverse-engineer a language from the ground up by mapping these composable constraints to words. Our language expresses a plan for any given task as a “plan sketch” to convey just-enough details while maximizing the flexibility to realize it, leading to robust coordination with optimality guarantees among other benefits. We formulate the problem, analyze it, and provide an approximate solution. We validate the advantages of our approach under various scenarios to shed light on its applications.

JBHI Journal 2023 Journal Article

Interpretable Inference and Classification of Tissue Types in Histological Colorectal Cancer Slides Based on Ensembles Adaptive Boosting Prototype Tree

• Meiyan Liang
• Ru Wang
• Jianan Liang
• Lin Wang
• Bo Li
• Xiaojun Jia
• Yu Zhang
• Qinghui Chen

Digital pathology images are treated as the “gold standard” for the diagnosis of colorectal lesions, especially colon cancer. Real-time, objective and accurate inspection results will assist clinicians to choose symptomatic treatment in a timely manner, which is of great significance in clinical medicine. However, Manual methods suffers from long inspection cycle and serious reliance on subjective interpretation. It is also a challenging task for existing computer-aided diagnosis methods to obtain models that are both accurate and interpretable. Models that exhibit high accuracy are always more complex and opaque, while interpretable models may lack the necessary accuracy. Therefore, the framework of ensemble adaptive boosting prototype tree is proposed to predict the colorectal pathology images and provide interpretable inference by visualizing the decision-making process in each base learner. The results showed that the proposed method could effectively address the “accuracy-interpretability trade-off” issue by ensemble of m adaptive boosting neural prototype trees. The superior performance of the framework provides a novel paradigm for interpretable inference and high-precision prediction of pathology image patches in computational pathology.

NeurIPS Conference 2023 Conference Paper

Interpreting Unsupervised Anomaly Detection in Security via Rule Extraction

• Ruoyu Li
• Qing Li
• Yu Zhang
• Dan Zhao
• Yong Jiang
• Yong Yang

Many security applications require unsupervised anomaly detection, as malicious data are extremely rare and often only unlabeled normal data are available for training (i. e. , zero-positive). However, security operators are concerned about the high stakes of trusting black-box models due to their lack of interpretability. In this paper, we propose a post-hoc method to globally explain a black-box unsupervised anomaly detection model via rule extraction. First, we propose the concept of distribution decomposition rules that decompose the complex distribution of normal data into multiple compositional distributions. To find such rules, we design an unsupervised Interior Clustering Tree that incorporates the model prediction into the splitting criteria. Then, we propose the Compositional Boundary Exploration (CBE) algorithm to obtain the boundary inference rules that estimate the decision boundary of the original model on each compositional distribution. By merging these two types of rules into a rule set, we can present the inferential process of the unsupervised black-box model in a human-understandable way, and build a surrogate rule-based model for online deployment at the same time. We conduct comprehensive experiments on the explanation of four distinct unsupervised anomaly detection models on various real-world datasets. The evaluation shows that our method outperforms existing methods in terms of diverse metrics including fidelity, correctness and robustness.

AAAI Conference 2023 Conference Paper

Learning Conflict-Noticed Architecture for Multi-Task Learning

• Zhixiong Yue
• Yu Zhang
• Jie Liang

Multi-task learning has been widely used in many applications to enable more efficient learning by sharing part of the architecture across multiple tasks. However, a major challenge is the gradient conflict when optimizing the shared parameters, where the gradients of different tasks could have opposite directions. Directly averaging those gradients will impair the performance of some tasks and cause negative transfer. Different from most existing works that manipulate gradients to mitigate the gradient conflict, in this paper, we address this problem from the perspective of architecture learning and propose a Conflict-Noticed Architecture Learning (CoNAL) method to alleviate the gradient conflict by learning architectures. By introducing purely-specific modules specific to each task in the search space, the CoNAL method can automatically learn when to switch to purely-specific modules in the tree-structured network architectures when the gradient conflict occurs. To handle multi-task problems with a large number of tasks, we propose a progressive extension of the CoNAL method. Extensive experiments on computer vision, natural language processing, and reinforcement learning benchmarks demonstrate the effectiveness of the proposed methods.

JMLR Journal 2023 Journal Article

LibMTL: A Python Library for Deep Multi-Task Learning

• Baijiong Lin
• Yu Zhang

This paper presents LibMTL, an open-source Python library built on PyTorch, which provides a unified, comprehensive, reproducible, and extensible implementation framework for Multi-Task Learning (MTL). LibMTL considers different settings and approaches in MTL, and it supports a large number of state-of-the-art MTL methods, including 13 optimization strategies and 8 architectures. Moreover, the modular design in LibMTL makes it easy to use and well-extensible, thus users can easily and fast develop new MTL methods, compare with existing MTL methods fairly, or apply MTL algorithms to real-world applications with the support of LibMTL. The source code and detailed documentations of LibMTL are available at https://github.com/median-research-group/LibMTL and https://libmtl.readthedocs.io, respectively. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2023. ( edit, beta )

IJCAI Conference 2023 Conference Paper

Max Markov Chain

• Yu Zhang
• Mitchell Bucklew

In this paper, we introduce Max Markov Chain (MMC), a novel model for sequential data with sparse correlations among the state variables. It may also be viewed as a special class of approximate models for High-order Markov Chains (HMCs). MMC is desirable for domains where the sparse correlations are long-term and vary in their temporal stretches. Although generally intractable, parameter optimization for MMC can be solved analytically. However, based on this result, we derive an approximate solution that is highly efficient empirically. When compared with HMC and approximate HMC models, MMC combines better sample efficiency, model parsimony, and an outstanding computational advantage. Such a quality allows MMC to scale to large domains where the competing models would struggle to perform. We compare MMC with several baselines with synthetic and real-world datasets to demonstrate MMC as a valuable alternative for stochastic modeling.

NeurIPS Conference 2023 Conference Paper

MG-ViT: A Multi-Granularity Method for Compact and Efficient Vision Transformers

• Yu Zhang
• Yepeng Liu
• Duoqian Miao
• Qi Zhang
• Yiwei Shi
• Liang Hu

Vision Transformer (ViT) faces obstacles in wide application due to its huge computational cost. Almost all existing studies on compressing ViT adopt the manner of splitting an image with a single granularity, with very few exploration of splitting an image with multi-granularity. As we know, important information often randomly concentrate in few regions of an image, necessitating multi-granularity attention allocation to an image. Enlightened by this, we introduce the multi-granularity strategy to compress ViT, which is simple but effective. We propose a two-stage multi-granularity framework, MG-ViT, to balance ViT’s performance and computational cost. In single-granularity inference stage, an input image is split into a small number of patches for simple inference. If necessary, multi-granularity inference stage will be instigated, where the important patches are further subsplit into multi-finer-grained patches for subsequent inference. Moreover, prior studies on compression only for classification, while we extend the multi-granularity strategy to hierarchical ViT for downstream tasks such as detection and segmentation. Extensive experiments Prove the effectiveness of the multi-granularity strategy. For instance, on ImageNet, without any loss of performance, MG-ViT reduces 47\% FLOPs of LV-ViT-S and 56\% FLOPs of DeiT-S.

JBHI Journal 2023 Journal Article

Multi-Class Classification of Upper Limb Movements With Filter Bank Task-Related Component Analysis

• Hao Jia
• Fan Feng
• Cesar F. Caiafa
• Feng Duan
• Yu Zhang
• Zhe Sun
• Jordi Solé-Casals

The classification of limb movements can provide with control commands in non-invasive brain-computer interface. Previous studies on the classification of limb movements have focused on the classification of left/right limbs; however, the classification of different types of upper limb movements has often been ignored despite that it provides more active-evoked control commands in the brain-computer interface. Nevertheless, few machine learning method can be used as the state-of-the-art method in the multi-class classification of limb movements. This work focuses on the multi-class classification of upper limb movements and proposes the multi-class filter bank task-related component analysis (mFBTRCA) method, which consists of three steps: spatial filtering, similarity measuring and filter bank selection. The spatial filter, namely the task-related component analysis, is first used to remove noise from EEG signals. The canonical correlation measures the similarity of the spatial-filtered signals and is used for feature extraction. The correlation features are extracted from multiple low-frequency filter banks. The minimum-redundancy maximum-relevance selects the essential features from all the correlation features, and finally, the support vector machine is used to classify the selected features. The proposed method compared against previously used models is evaluated using two datasets. mFBTRCA achieved a classification accuracy of 0. 4193 $\pm$ 0. 0780 (7 classes) and 0. 4032 $\pm$ 0. 0714 (5 classes), respectively, which improves on the best accuracies achieved using the compared methods (0. 3590 $\pm$ 0. 0645 and 0. 3159 $\pm$ 0. 0736, respectively). The proposed method is expected to provide more control commands in the applications of non-invasive brain-computer interfaces.

IJCAI Conference 2023 Conference Paper

Multi-Task Learning via Time-Aware Neural ODE

• Feiyang Ye
• Xuehao Wang
• Yu Zhang
• Ivor W. Tsang

Multi-Task Learning (MTL) is a well-established paradigm for learning shared models for a diverse set of tasks. Moreover, MTL improves data efficiency by jointly training all tasks simultaneously. However, directly optimizing the losses of all the tasks may lead to imbalanced performance on all the tasks due to the competition among tasks for the shared parameters in MTL models. Many MTL methods try to mitigate this problem by dynamically weighting task losses or manipulating task gradients. Different from existing studies, in this paper, we propose a Neural Ordinal diffeRential equation based Multi-tAsk Learning (NORMAL) method to alleviate this issue by modeling task-specific feature transformations from the perspective of dynamic flows built on the Neural Ordinary Differential Equation (NODE). Specifically, the proposed NORMAL model designs a time-aware neural ODE block to learn task-specific time information, which determines task positions of feature transformations in the dynamic flow, in NODE automatically via gradient descent methods. In this way, the proposed NORMAL model handles the problem of competing shared parameters by learning task positions. Moreover, the learned task positions can be used to measure the relevance among different tasks. Extensive experiments show that the proposed NORMAL model outperforms state-of-the-art MTL models.

AAAI Conference 2023 Conference Paper

Personalized Dialogue Generation with Persona-Adaptive Attention

• Qiushi Huang
• Yu Zhang
• Tom Ko
• Xubo Liu
• Bo Wu
• Wenwu Wang
• H Tang

Persona-based dialogue systems aim to generate consistent responses based on historical context and predefined persona. Unlike conventional dialogue generation, the persona-based dialogue needs to consider both dialogue context and persona, posing a challenge for coherent training. Specifically, this requires a delicate weight balance between context and persona. To achieve that, in this paper, we propose an effective framework with Persona-Adaptive Attention (PAA), which adaptively integrates the weights from the persona and context information via our designed attention. In addition, a dynamic masking mechanism is applied to the PAA to not only drop redundant information in context and persona but also serve as a regularization mechanism to avoid overfitting. Experimental results demonstrate the superiority of the proposed PAA framework compared to the strong baselines in both automatic and human evaluation. Moreover, the proposed PAA approach can perform equivalently well in a low-resource regime compared to models trained in a full-data setting, which achieve a similar result with only 20% to 30% of data compared to the larger models trained in the full-data setting. To fully exploit the effectiveness of our design, we designed several variants for handling the weighted information in different ways, showing the necessity and sufficiency of our weighting and masking designs.

JBHI Journal 2023 Journal Article

Privacy-Preserving Multi-Source Domain Adaptation for Medical Data

• Tianyi Han
• Xiaoli Gong
• Fan Feng
• Jin Zhang
• Zhe Sun
• Yu Zhang

Great progress has been made in diagnosing medical diseases based on deep learning. Large-scale medical data are expected to improve deep learning performance further. It is almost impossible for a single institution to collect so much data due to the time-consuming and costly collection and labeling of medical data. Many studies have turned attention to data sharing among multiple medical institutions. However, due to different data acquiring and processing procedures, multiple institutions' medical data is characterized by distribution heterogeneity. Besides, the protection of patient privacy in medical data sharing has also been a common concern. To simultaneously address the problems of heterogeneous data distribution and privacy protection, we propose a novel multi-source source free domain adaptation. When aligning distributed heterogeneous data, our method only require to transfer the pre-trained source models rather than the direct source domain data, thus protecting patients' privacy. In addition, it has the advantages of being efficient and less costly in network resources. The proposed method is evaluated on the multi-site fMRI database Autism Brain Imaging Data Exchange (ABIDE) and yields an average accuracy of 69. 37%. We also analyzed its effectiveness on network resource-saving and conducted additional experiments on Camelyon17 to validate the generalization.

AAAI Conference 2023 Conference Paper

Robust Temporal Smoothness in Multi-Task Learning

• Menghui Zhou
• Yu Zhang
• Yun Yang
• Tong Liu
• Po Yang

Multi-task learning models based on temporal smoothness assumption, in which each time point of a sequence of time points concerns a task of prediction, assume the adjacent tasks are similar to each other. However, the effect of outliers is not taken into account. In this paper, we show that even only one outlier task will destroy the performance of the entire model. To solve this problem, we propose two Robust Temporal Smoothness (RoTS) frameworks. Compared with the existing models based on temporal relation, our methods not only chase the temporal smoothness information but identify outlier tasks, however, without increasing the computational complexity. Detailed theoretical analyses are presented to evaluate the performance of our methods. Experimental results on synthetic and real-life datasets demonstrate the effectiveness of our frameworks. We also discuss several potential specific applications and extensions of our RoTS frameworks.

YNIMG Journal 2022 Journal Article

A dynamic graph convolutional neural network framework reveals new insights into connectome dysfunctions in ADHD

• Kanhao Zhao
• Boris Duka
• Hua Xie
• Desmond J. Oathes
• Vince Calhoun
• Yu Zhang

The pathological mechanism of attention deficit hyperactivity disorder (ADHD) is incompletely specified, which leads to difficulty in precise diagnosis. Functional magnetic resonance imaging (fMRI) has emerged as a common neuroimaging technique for studying the brain functional connectome. Most existing methods that have either ignored or simply utilized graph structure, do not fully leverage the potentially important topological information which may be useful in characterizing brain disorders. There is a crucial need for designing novel and efficient approaches which can capture such information. To this end, we propose a new dynamic graph convolutional network (dGCN), which is trained with sparse brain regional connections from dynamically calculated graph features. We also develop a novel convolutional readout layer to improve graph representation. Our extensive experimental analysis demonstrates significantly improved performance of dGCN for ADHD diagnosis compared with existing machine learning and deep learning methods. Visualizations of the salient regions of interest (ROIs) and connectivity based on informative features learned by our model show that the identified functional abnormalities mainly involve brain regions in temporal pole, gyrus rectus, and cerebellar gyri from temporal lobe, frontal lobe, and cerebellum, respectively. A positive correlation was further observed between the identified connectomic abnormalities and ADHD symptom severity. The proposed dGCN model shows great promise in providing a functional network-based precision diagnosis of ADHD and is also broadly applicable to brain connectome-based study of mental disorders.

JBHI Journal 2022 Journal Article

Contrastive Cross-Modal Pre-Training: A General Strategy for Small Sample Medical Imaging

• Gongbo Liang
• Connor Greenwell
• Yu Zhang
• Xin Xing
• Xiaoqin Wang
• Ramakanth Kavuluru
• Nathan Jacobs

A key challenge in training neural networks for a given medical imaging task is the difficulty of obtaining a sufficient number of manually labeled examples. In contrast, textual imaging reports are often readily available in medical records and contain rich but unstructured interpretations written by experts as part of standard clinical practice. We propose using these textual reports as a form of weak supervision to improve the image interpretation performance of a neural network without requiring additional manually labeled examples. We use an image-text matching task to train a feature extractor and then fine-tune it in a transfer learning setting for a supervised task using a small labeled dataset. The end result is a neural network that automatically interprets imagery without requiring textual reports during inference. We evaluate our method on three classification tasks and find consistent performance improvements, reducing the need for labeled data by 67%–98%.

NeurIPS Conference 2022 Conference Paper

Dual-Curriculum Contrastive Multi-Instance Learning for Cancer Prognosis Analysis with Whole Slide Images

• CHAO TU
• Yu Zhang
• Zhenyuan Ning

The multi-instance learning (MIL) has advanced cancer prognosis analysis with whole slide images (WSIs). However, current MIL methods for WSI analysis still confront unique challenges. Previous methods typically generate instance representations via a pre-trained model or a model trained by the instances with bag-level annotations, which, however, may not generalize well to the downstream task due to the introduction of excessive label noises and the lack of fine-grained information across multi-magnification WSIs. Additionally, existing methods generally aggregate instance representations as bag ones for prognosis prediction and have no consideration of intra-bag redundancy and inter-bag discrimination. To address these issues, we propose a dual-curriculum contrastive MIL method for cancer prognosis analysis with WSIs. The proposed method consists of two curriculums, i. e. , saliency-guided weakly-supervised instance encoding with cross-scale tiles and contrastive-enhanced soft-bag prognosis inference. Extensive experiments on three public datasets demonstrate that our method outperforms state-of-the-art methods in this field. The code is available at https: //github. com/YuZhang-SMU/Cancer-Prognosis-Analysis/tree/main/DC_MIL%20Code.

NeurIPS Conference 2022 Conference Paper

Dynamic Sparse Network for Time Series Classification: Learning What to “See”

• Qiao Xiao
• Boqian Wu
• Yu Zhang
• Shiwei Liu
• Mykola Pechenizkiy
• Elena Mocanu
• Decebal Constantin Mocanu

The receptive field (RF), which determines the region of time series to be “seen” and used, is critical to improve the performance for time series classification (TSC). However, the variation of signal scales across and within time series data, makes it challenging to decide on proper RF sizes for TSC. In this paper, we propose a dynamic sparse network (DSN) with sparse connections for TSC, which can learn to cover various RF without cumbersome hyper-parameters tuning. The kernels in each sparse layer are sparse and can be explored under the constraint regions by dynamic sparse training, which makes it possible to reduce the resource cost. The experimental results show that the proposed DSN model can achieve state-of-art performance on both univariate and multivariate TSC datasets with less than 50% computational cost compared with recent baseline methods, opening the path towards more accurate resource-aware methods for time series analyses. Our code is publicly available at: https: //github. com/QiaoXiao7282/DSN.

JBHI Journal 2022 Journal Article

Efficient Subject-Independent Detection of Anterior Cruciate Ligament Deficiency Based on Marine Predator Algorithm and Support Vector Machine

• Gengyuan Wang
• Xiaolong Zeng
• Guanquan Lai
• Guoqing Zhong
• Ke Ma
• Yu Zhang

Anterior cruciate ligament (ACL) deficiency not only reduces knee stability, but also increases the risk of more disease and impairs daily life, thus requiring efficient detection of ACL deficiency. To build an efficient subject-independent ACL deficiency detection model, this study proposes a new method called SVM-MPA that fuses marine predator algorithm (MPA) and support vector machine (SVM) for simultaneous feature selection, hyperparameter optimization and classification. 35ACL-deficient (ACLD) and 35 ACL-intact (ACLI) participants were recruited to collect 6-degree-of-freedom knee kinematic data. Then, 216-dimensional multi-domain features covering time domain, frequency domain, time-frequency domain and nonlinearity were extracted. The error rate of SVM classification based on 5-fold cross-validation was used to construct the fitness of MPA, and MPA served to select features and optimize two hyperparameters for SVM. The majority voting strategy-based post-processing was introduced to convert the gait cycle-level to knee-level ACL deficiency detection. Comparing with 7 well-known meta-heuristic algorithms and running all 20 times, the best average gait cycle-level ACL deficiency detection performance (sensitivity: 96. 78±0. 4. 84%, specificity: 99. 43±5. 70%, and accuracy: 98. 48±1. 70%) was obtained using the proposed method. With post-processing, this study improved the best (final) detection performance (sensitivity: 97. 78±4. 97%, specificity: 100±0. 00%, and accuracy: 99. 13±1. 94%). These results demonstrate the feasibility and effectiveness of the proposed method and shows that an efficient subject-independent ACL deficiency detection model can be constructed using the proposed method, which makes it possible to provide a non-invasive, objective and accurate preoperative auxiliary detection method for diagnosing ACL deficiency clinically.

NeurIPS Conference 2022 Conference Paper

Generating Training Data with Language Models: Towards Zero-Shot Language Understanding

• Yu Meng
• Jiaxin Huang
• Yu Zhang
• Jiawei Han

Pretrained language models (PLMs) have demonstrated remarkable performance in various natural language processing tasks: Unidirectional PLMs (e. g. , GPT) are well known for their superior text generation capabilities; bidirectional PLMs (e. g. , BERT) have been the prominent choice for natural language understanding (NLU) tasks. While both types of models have achieved promising few-shot learning performance, their potential for zero-shot learning has been underexplored. In this paper, we present a simple approach that uses both types of PLMs for fully zero-shot learning of NLU tasks without requiring any task-specific data: A unidirectional PLM generates class-conditioned texts guided by prompts, which are used as the training data for fine-tuning a bidirectional PLM. With quality training data selected based on the generation probability and regularization techniques (label smoothing and temporal ensembling) applied to the fine-tuning stage for better generalization and stability, our approach demonstrates strong performance across seven classification tasks of the GLUE benchmark (e. g. , 72. 3/73. 8 on MNLI-m/mm and 92. 8 on SST-2), significantly outperforming zero-shot prompting methods and achieving even comparable results to strong few-shot approaches using 32 training samples per class.

YNIMG Journal 2022 Journal Article

Longitudinal brain atlases of early developing cynomolgus macaques from birth to 48 months of age

• Tao Zhong
• Jingkuan Wei
• Kunhua Wu
• Liangjun Chen
• Fenqiang Zhao
• Yuchen Pei
• Ya Wang
• Hongjiang Zhang

Longitudinal brain imaging atlases with densely sampled time-points and ancillary anatomical information are of fundamental importance in studying early developmental characteristics of human and non-human primate brains during infancy, which feature extremely dynamic imaging appearance, brain shape and size. However, for non-human primates, which are highly valuable animal models for understanding human brains, the existing brain atlases are mainly developed based on adults or adolescents, denoting a notable lack of temporally densely-sampled atlases covering the dynamic early brain development. To fill this critical gap, in this paper, we construct a comprehensive set of longitudinal brain atlases and associated tissue probability maps (gray matter, white matter, and cerebrospinal fluid) with totally 12 time-points from birth to 4 years of age (i.e., 1, 2, 3, 4, 5, 6, 9, 12, 18, 24, 36, and 48 months of age) based on 175 longitudinal structural MRI scans from 39 typically-developing cynomolgus macaques, by leveraging state-of-the-art computational techniques tailored for early developing brains. Furthermore, to facilitate region-based analysis using our atlases, we also provide two popular hierarchy parcellations, i.e., cortical hierarchy maps (6 levels) and subcortical hierarchy maps (6 levels), on our longitudinal macaque brain atlases. These early developing atlases, which have the densest time-points during infancy (to the best of our knowledge), will greatly facilitate the studies of macaque brain development.

IROS Conference 2022 Conference Paper

Model-Based Disturbance Estimation for a Fiber-Reinforced Soft Manipulator using Orientation Sensing

• Barnabas Gavin Cangan
• Stefan Escaida Navarro
• Bai Yang
• Yu Zhang
• Christian Duriez
• Robert K. Katzschmann

To aid in real-world situations, soft robots need to be able to estimate their state and external interactions based on proprioceptive sensors. Estimating disturbances allows a soft robot to perform desirable force control. However, even in the case of rigid manipulators, force estimation at the end-effector is seen as a non-trivial problem. And indeed, current approaches to address this challenge have shortcomings that prevent their general application. They are often based on simplified soft dynamic models, such as the ones relying on a piece-wise constant curvature approximation or matched rigid-body models that do not represent enough details of the problem. This severely limits applications in complex human-robot interaction. Finite element method (FEM) based modeling allows for predictions of soft robot dynamics in a more generic fashion. Here, using the soft robot modeling capabilities of the frame-work SOFA, we built a detailed FEM model of a multi-segment soft continuum robotic arm composed of compliant deformable materials and fiber-reinforced pressurized actuation chambers. In addition, a model for sensors that provide orientation output is presented. This model is used to establish a state observer for the manipulator. The sensor model is adequate for representing the output of flexible bend sensors as well as orientations provided by IMUs or coming from tracking systems, all of which are popular choices in soft robotics. Model parameters were calibrated to match imperfections of the manual fabrication process using physical experiments. We then solve a quadratic programming inverse statics problem to compute the components of external force that explain the pose mismatch. Our experiments show an average force estimation error of around 1. 2%. As the methods proposed are generic, these results are encouraging for the task of building soft robots exhibiting complex, reactive, sensor-based behavior that can be deployed in human-centered environments.

ICRA Conference 2022 Conference Paper

Path-Aware Graph Attention for HD Maps in Motion Prediction

• Fang Da
• Yu Zhang

The success of motion prediction for autonomous driving relies on integration of information from the HD maps. As maps are naturally graph-structured, investigation on graph neural networks (GNNs) for encoding HD maps is burgeoning in recent years. However, unlike many other applications where GNNs have been straightforwardly deployed, HD maps are heterogeneous graphs where vertices (lanes) are connected by edges (lane-lane interaction relationships) of various nature, and most graph-based models are not designed to understand the variety of edge types which provide crucial cues for predicting how the agents would travel the lanes. To overcome this challenge, we propose Path-Aware Graph Attention, a novel attention architecture that infers the attention between two vertices by parsing the sequence of edges forming the paths that connect them. Our analysis illustrates how the proposed attention mechanism can facilitate learning in a didactic problem where existing graph networks like GCN struggle. By improving map encoding, the proposed model surpasses previous state of the art on the Argoverse Motion Forecasting dataset, and won the first place in the 2021 Argoverse Motion Forecasting Competition.

AAAI Conference 2022 Conference Paper

Policy Optimization with Stochastic Mirror Descent

• Long Yang
• Yu Zhang
• Gang Zheng
• Qian Zheng
• Pengfei Li
• Jianhang Huang
• Gang Pan

Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In VRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency. We prove that the proposed VRMPO needs only O( −3 ) sample trajectories to achieve an -approximate first-order stationary point, which matches the best sample complexity for policy optimization. Extensive empirical results demonstrate that VRMPO outperforms the state-of-the-art policy gradient methods in various settings.

TMLR Journal 2022 Journal Article

Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning

• Baijiong Lin
• Feiyang Ye
• Yu Zhang
• Ivor Tsang

Multi-Task Learning (MTL) has achieved success in various fields. However, training with equal weights for all tasks may cause unsatisfactory performance for part of tasks. To address this problem, there are many works to carefully design dynamical loss/gradient weighting strategies but the basic random experiments are ignored to examine their effectiveness. In this paper, we propose the Random Weighting (RW) methods, including Random Loss Weighting (RLW) and Random Gradient Weighting (RGW), where an MTL model is trained with random loss/gradient weights sampled from a distribution. To show the effectiveness and necessity of RW methods, theoretically, we analyze the convergence of RW and reveal that RW has a higher probability to escape local minima, resulting in better generalization ability. Empirically, we extensively evaluate the proposed RW methods to compare with twelve state-of-the-art methods on five image datasets and two multilingual problems from the XTREME benchmark to show that RW methods can achieve comparable performance with state-of-the-art baselines. Therefore, we think the RW methods are important baselines for MTL and should attract more attention.

JBHI Journal 2022 Journal Article

Reconstruction of Missing Samples in Antepartum and Intrapartum FHR Measurements Via Mini-Batch-Based Minimized Sparse Dictionary Learning

• Yefei Zhang
• Zhidong Zhao
• Yanjun Deng
• Xiaohong Zhang
• Yu Zhang

Fetal Heart Rate (FHR), an important recording in Cardiotocography (CTG)-based fetal health status monitoring, is the only information that clinical obstetricians can directly obtain and use. A challenge, however, is that missing samples are very common in FHR due to various causes such as fetal movements and sensor malfunctions. The aim is the development of an inpainting tool which is suitable for different missing lengths $q$ and various total missing percentages $Q$, as well as for use in online mode. This study focused on two major impediments to existing inpainting methods: the longer the missing length, the more difficult it is to recover with mathematical methods; the reliance on tens of thousands of training samples, and the computational burden caused by full batch-based dictionary learning algorithms. We present a regularized minimization approach to signal recovery, which combines a ${{\rm{L}}_{{0}{\rm{. 6}}}}{\rm{ - norm}}$ minimized sparse dictionary learning algorithm (MSDL) and a model optimization strategy for using a mini-batch version for signal recovery. Using 100 FHR recordings with 2 protocols designed to simulate missing clinical data scenarios, the combined method performed favorably in terms of 5 data analysis metrics and 3 clinical indicators. Comparing 4 inpainting methods, we were able to prove the superiority of the proposed algorithm for both large $q$ and large $Q$. The experimental results showed the lowest values (2. 64 (MAE), 4. 68 (RMSE)) when ${\rm{Q}} = {\rm{5\% }}$ with short interval lengths. The developed architecture provides a reference value for the practical application of recovering missing samples online.

ICML Conference 2022 Conference Paper

Self-supervised learning with random-projection quantizer for speech recognition

• Chung-Cheng Chiu
• James Qin
• Yu Zhang
• Jiahui Yu
• Yonghui Wu

We present a simple and effective self-supervised learning approach for speech recognition. The approach learns a model to predict the masked speech signals, in the form of discrete labels generated with a random-projection quantizer. In particular the quantizer projects speech inputs with a randomly initialized matrix, and does a nearest-neighbor lookup in a randomly-initialized codebook. Neither the matrix nor the codebook are updated during self-supervised learning. Since the random-projection quantizer is not trained and is separated from the speech recognition model, the design makes the approach flexible and is compatible with universal speech recognition architecture. On LibriSpeech our approach achieves similar word-error-rates as previous work using self-supervised learning with non-streaming models, and provides lower word-error-rates than previous work with streaming models. On multilingual tasks the approach also provides significant improvement over wav2vec 2. 0 and w2v-BERT.

IROS Conference 2022 Conference Paper

Sim2Real for Soft Robotic Fish via Differentiable Simulation

• John Z. Zhang
• Yu Zhang
• Pingchuan Ma 0002
• Elvis Nava
• Tao Du 0001
• Philip Arm
• Wojciech Matusik
• Robert K. Katzschmann

Accurate simulation of soft mechanisms under dynamic actuation is critical for the design of soft robots. We address this gap with our differentiable simulation tool by learning the material parameters of our soft robotic fish. On the example of a soft robotic fish, we demonstrate an experimentally-verified, fast optimization pipeline for learning the material parameters from quasi-static data via differentiable simulation and apply it to the prediction of dynamic performance. Our method identifies physically plausible Young's moduli for various soft silicone elastomers and stiff acetal copolymers used in creation of our three different robotic fish tail designs. We show that our method is compatible with varying internal geometry of the actuators, such as the number of hollow cavities. Our framework allows high fidelity prediction of dynamic behavior for composite bi-morph bending structures in real hardware to millimeter-accuracy and within 3% error normalized to actuator length. We provide a differentiable and robust estimate of the thrust force using a neural network thrust predictor; this estimate allows for accurate modeling of our experimental setup measuring bollard pull. This work presents a prototypical hardware and simulation problem solved using our differentiable framework; the framework can be applied to higher dimensional parameter inference, learning control policies, and computational design due to its differentiable character.

EAAI Journal 2021 Journal Article

An effective hybrid meta-heuristic for flexible flow shop scheduling with limited buffers and step-deteriorating jobs

• Qian-Qian Zheng
• Yu Zhang
• Hong-Wei Tian
• Li-Jun He

This paper addresses a flexible flow shop scheduling problem considering limited buffers and step-deteriorating jobs, where there are multiple non-identical parallel machines. A mixed integer programming model is proposed, with the criterion of minimizing the makespan and total tardiness simultaneously. To handle this problem, an effective hybrid meta-heuristic algorithm, named GVNSA, is developed based on genetic algorithm (GA), variable neighborhood search (VNS) and simulated annealing (SA). In the algorithm, with a two-dimensional matrix encoding scheme, the NEH (Nawaz–Enscore–Ham) heuristic and bottleneck elimination method are implemented to determine the initial population. A three-level rolling translation approach is designed for decoding. To balance the exploration and exploitation abilities, three effective steps are executed: 1) partial matching crossover and mutation strategy based on multiple neighborhood search structures are imposed on the GA operators; 2) a VNS with SA is introduced to re-optimize some individuals from GA, where four neighborhood structures are constructed; 3) a modified CDS (Campbell–Dudek–Smith) heuristic is embedded to disturb population in the mid-iteration. Numerical experiments are carried out on test problems with different scales. Computational results demonstrate that the proposed GVNSA can obtain higher quality solutions in comparison with other heuristics and meta-heuristics existing in literature.

YNIMG Journal 2021 Journal Article

DIKA-Nets: Domain-invariant knowledge-guided attention networks for brain skull stripping of early developing macaques

• Tao Zhong
• Fenqiang Zhao
• Yuchen Pei
• Zhenyuan Ning
• Lufan Liao
• Zhengwang Wu
• Yuyu Niu
• Li Wang

As non-human primates, macaques have a close phylogenetic relationship to human beings and have been proven to be a valuable and widely used animal model in human neuroscience research. Accurate skull stripping (aka. brain extraction) of brain magnetic resonance imaging (MRI) is a crucial prerequisite in neuroimaging analysis of macaques. Most of the current skull stripping methods can achieve satisfactory results for human brains, but when applied to macaque brains, especially during early brain development, the results are often unsatisfactory. In fact, the early dynamic, regionally-heterogeneous development of macaque brains, accompanied by poor and age-related contrast between different anatomical structures, poses significant challenges for accurate skull stripping. To overcome these challenges, we propose a fully-automated framework to effectively fuse the age-specific intensity information and domain-invariant prior knowledge as important guiding information for robust skull stripping of developing macaques from 0 to 36 months of age. Specifically, we generate Signed Distance Map (SDM) and Center of Gravity Distance Map (CGDM) based on the intermediate segmentation results as guidance. Instead of using local convolution, we fuse all information using the Dual Self-Attention Module (DSAM), which can capture global spatial and channel-dependent information of feature maps. To extensively evaluate the performance, we adopt two relatively-large challenging MRI datasets from rhesus macaques and cynomolgus macaques, respectively, with a total of 361 scans from two different scanners with different imaging protocols. We perform cross-validation by using one dataset for training and the other one for testing. Our method outperforms five popular brain extraction tools and three deep-learning-based methods on cross-source MRI datasets without any transfer learning.

AAAI Conference 2021 Conference Paper

Distant Transfer Learning via Deep Random Walk

• Qiao Xiao
• Yu Zhang

Transfer learning, which is to improve the learning performance in the target domain by leveraging useful knowledge from the source domain, often requires that those two domains are very close, which limits its application scope. Recently, distant transfer learning has been studied to transfer knowledge between two distant or even totally unrelated domains via unlabeled auxiliary domains that act as a bridge in the spirit of human transitive inference that two completely unrelated concepts can be connected through gradual knowledge transfer. In this paper, we study distant transfer learning by proposing a DeEp Random Walk basEd distaNt Transfer (DERWENT) method. Different from existing distant transfer learning models that implicitly identify the path of knowledge transfer between the source and target instances through auxiliary instances, the proposed DERWENT model can explicitly learn such paths via the deep random walk technique. Specifically, based on sequences identified by the random walk technique on a data graph where source and target data have no direct connection, the proposed DERWENT model enforces adjacent data points in a sequence to be similar, makes the ending data point be represented by other data points in the same sequence, and considers weighted classification losses of source data. Empirical studies on several benchmark datasets demonstrate that the proposed DERWENT algorithm yields the state-ofthe-art performance.

NeurIPS Conference 2021 Conference Paper

Effective Meta-Regularization by Kernelized Proximal Regularization

• Weisen Jiang
• James Kwok
• Yu Zhang

We study the problem of meta-learning, which has proved to be advantageous to accelerate learning new tasks with a few samples. The recent approaches based on deep kernels achieve the state-of-the-art performance. However, the regularizers in their base learners are not learnable. In this paper, we propose an algorithm called MetaProx to learn a proximal regularizer for the base learner. We theoretically establish the convergence of MetaProx. Experimental results confirm the advantage of the proposed algorithm.

YNIMG Journal 2021 Journal Article

Functional annotation of human cognitive states using deep graph convolution

• Yu Zhang
• Loïc Tetrel
• Bertrand Thirion
• Pierre Bellec

A key goal in neuroscience is to understand brain mechanisms of cognitive functions. An emerging approach is "brain decoding", which consists of inferring a set of experimental conditions performed by a participant, using pattern classification of brain activity. Few works so far have attempted to train a brain decoding model that would generalize across many different cognitive tasks drawn from multiple cognitive domains. To tackle this problem, we proposed a multidomain brain decoder that automatically learns the spatiotemporal dynamics of brain response within a short time window using a deep learning approach. We evaluated the decoding model on a large population of 1200 participants, under 21 different experimental conditions spanning six different cognitive domains, acquired from the Human Connectome Project task-fMRI database. Using a 10s window of fMRI response, the 21 cognitive states were identified with a test accuracy of 90% (chance level 4.8%). Performance remained good when using a 6s window (82%). It was even feasible to decode cognitive states from a single fMRI volume (720ms), with the performance following the shape of the hemodynamic response. Moreover, a saliency map analysis demonstrated that the high decoding performance was driven by the response of biologically meaningful brain regions. Together, we provide an automated tool to annotate human brain activity with fine temporal resolution and fine cognitive granularity. Our model shows potential applications as a reference model for domain adaptation, possibly making contributions in a variety of domains, including neurological and psychiatric disorders.

NeurIPS Conference 2021 Conference Paper

Multi-Objective Meta Learning

• Feiyang Ye
• Baijiong Lin
• Zhixiong Yue
• Pengxin Guo
• Qiao Xiao
• Yu Zhang

Meta learning with multiple objectives has been attracted much attention recently since many applications need to consider multiple factors when designing learning models. Existing gradient-based works on meta learning with multiple objectives mainly combine multiple objectives into a single objective in a weighted sum manner. This simple strategy usually works but it requires to tune the weights associated with all the objectives, which could be time consuming. Different from those works, in this paper, we propose a gradient-based Multi-Objective Meta Learning (MOML) framework without manually tuning weights. Specifically, MOML formulates the objective function of meta learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta learner. To solve the MOBLP, we devise the first gradient-based optimization algorithm by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search. The source code of MOML is available at https: //github. com/Baijiong-Lin/MOML.

AAAI Conference 2021 Conference Paper

On Convergence of Gradient Expected Sarsa(λ)

• Long Yang
• Gang Zheng
• Yu Zhang
• Qian Zheng
• Pengfei Li
• Gang Pan

We study the convergence of Expected Sarsa(λ) with function approximation. We show that with off-line estimate (multi-step bootstrapping) to Expected Sarsa(λ) is unstable for off-policy learning. Furthermore, based on convex-concave saddle-point framework, we propose a convergent Gradient Expected Sarsa(λ) (GES(λ)) algorithm. The theoretical analysis shows that the proposed GES(λ) converges to the optimal solution at a linear convergence rate under true gradient setting. Furthermore, we develop a Lyapunov function technique to investigate how the stepsize influences finite-time performance of GES(λ). Additionally, such a technique of Lyapunov function can be potentially generalized to other gradient temporal difference algorithms. Finally, our experiments verify the effectiveness of our GES(λ). For the details of proof, please refer to https: //arxiv. org/pdf/2012. 07199. pdf.

JBHI Journal 2020 Journal Article

Adversarial Representation Learning for Robust Patient-Independent Epileptic Seizure Detection

• Xiang Zhang
• Lina Yao
• Manqing Dong
• Zhe Liu
• Yu Zhang
• Yong Li

Epilepsy is a chronic neurological disorder characterized by the occurrence of spontaneous seizures, which affects about one percent of the worlds population. Most of the current seizure detection approaches strongly rely on patient history records and thus fail in the patient-independent situation of detecting the new patients. To overcome such limitation, we propose a robust and explainable epileptic seizure detection model that effectively learns from seizure states while eliminates the inter-patient noises. A complex deep neural network model is proposed to learn the pure seizure-specific representation from the raw non-invasive electroencephalography (EEG) signals through adversarial training. Furthermore, to enhance the explainability, we develop an attention mechanism to automatically learn the importance of each EEG channels in the seizure diagnosis procedure. The proposed approach is evaluated over the Temple University Hospital EEG (TUH EEG) database. The experimental results illustrate that our model outperforms the competitive state-of-the-art baselines with low latency. Moreover, the designed attention mechanism is demonstrated ables to provide fine-grained information for pathological analysis. We propose an effective and efficient patient-independent diagnosis approach of epileptic seizure based on raw EEG signals without manually feature engineering, which is a step toward the development of large-scale deployment for real-life use.

IJCAI Conference 2020 Conference Paper

Fast and Accurate Neural CRF Constituency Parsing

• Yu Zhang
• Houquan Zhou
• Zhenghua Li

Estimating probability distribution is one of the core issues in the NLP field. However, in both deep learning (DL) and pre-DL eras, unlike the vast applications of linear-chain CRF in sequence labeling tasks, very few works have applied tree-structure CRF to constituency parsing, mainly due to the complexity and inefficiency of the inside-outside algorithm. This work presents a fast and accurate neural CRF constituency parser. The key idea is to batchify the inside algorithm for loss computation by direct large tensor operations on GPU, and meanwhile avoid the outside algorithm for gradient computation via efficient back-propagation. We also propose a simple two-stage bracketing-then-labeling parsing approach to improve efficiency further. To improve the parsing performance, inspired by recent progress in dependency parsing, we introduce a new scoring architecture based on boundary representation and biaffine attention, and a beneficial dropout strategy. Experiments on PTB, CTB5. 1, and CTB7 show that our two-stage CRF parser achieves new state-of-the-art performance on both settings of w/o and w/ BERT, and can parse over 1, 000 sentences per second. We release our code at https: //github. com/yzhangcs/crfpar.

AAAI Conference 2020 Conference Paper

Knowledge Distillation from Internal Representations

• Gustavo Aguilar
• Yuan Ling
• Yu Zhang
• Benjamin Yao
• Xing Fan
• Chenlei Guo

Knowledge distillation is typically conducted by training a small model (the student) to mimic a large and cumbersome model (the teacher). The idea is to compress the knowledge from the teacher by using its output probabilities as softlabels to optimize the student. However, when the teacher is considerably large, there is no guarantee that the internal knowledge of the teacher will be transferred into the student; even if the student closely matches the soft-labels, its internal representations may be considerably different. This internal mismatch can undermine the generalization capabilities originally intended to be transferred from the teacher to the student. In this paper, we propose to distill the internal representations of a large model such as BERT into a simplified version of it. We formulate two ways to distill such representations and various algorithms to conduct the distillation. We experiment with datasets from the GLUE benchmark and consistently show that adding knowledge distillation from internal representations is a more powerful method than only using soft-label distillation.

IJCAI Conference 2020 Conference Paper

Label Distribution for Learning with Noisy Labels

• Yun-Peng Liu
• Ning Xu
• Yu Zhang
• Xin Geng

The performances of deep neural networks (DNNs) crucially rely on the quality of labeling. In some situations, labels are easily corrupted, and therefore some labels become noisy labels. Thus, designing algorithms that deal with noisy labels is of great importance for learning robust DNNs. However, it is difficult to distinguish between clean labels and noisy labels, which becomes the bottleneck of many methods. To address the problem, this paper proposes a novel method named Label Distribution based Confidence Estimation (LDCE). LDCE estimates the confidence of the observed labels based on label distribution. Then, the boundary between clean labels and noisy labels becomes clear according to confidence scores. To verify the effectiveness of the method, LDCE is combined with the existing learning algorithm to train robust DNNs. Experiments on both synthetic and real-world datasets substantiate the superiority of the proposed algorithm against state-of-the-art methods.

IJCAI Conference 2020 Conference Paper

Label Enhancement for Label Distribution Learning via Prior Knowledge

• Yongbiao Gao
• Yu Zhang
• Xin Geng

Label distribution learning (LDL) is a novel machine learning paradigm that gives a description degree of each label to an instance. However, most of training datasets only contain simple logical labels rather than label distributions due to the difficulty of obtaining the label distributions directly. We propose to use the prior knowledge to recover the label distributions. The process of recovering the label distributions from the logical labels is called label enhancement. In this paper, we formulate the label enhancement as a dynamic decision process. Thus, the label distribution is adjusted by a series of actions conducted by a reinforcement learning agent according to sequential state representations. The target state is defined by the prior knowledge. Experimental results show that the proposed approach outperforms the state-of-the-art methods in both age estimation and image emotion recognition.

AAAI Conference 2020 Conference Paper

What Is It You Really Want of Me? Generalized Reward Learning with Biased Beliefs about Domain Dynamics

• Ze Gong
• Yu Zhang

Reward learning as a method for inferring human intent and preferences has been studied extensively. Prior approaches make an implicit assumption that the human maintains a correct belief about the robot’s domain dynamics. However, this may not always hold since the human’s belief may be biased, which can ultimately lead to a misguided estimation of the human’s intent and preferences, which is often derived from human feedback on the robot’s behaviors. In this paper, we remove this restrictive assumption by considering that the human may have an inaccurate understanding of the robot. We propose a method called Generalized Reward Learning with biased beliefs about domain dynamics (GeReL) to infer both the reward function and human’s belief about the robot in a Bayesian setting based on human ratings. Due to the complex forms of the posteriors, we formulate it as a variational inference problem to infer the posteriors of the parameters that govern the reward function and human’s belief about the robot simultaneously. We evaluate our method in a simulated domain and with a user study where the user has a bias based on the robot’s appearances. The results show that our method can recover the true human preferences while subject to such biased beliefs, in contrast to prior approaches that could have misinterpreted them completely.

AAMAS Conference 2019 Conference Paper

Explicable Planning as Minimizing Distance from Expected Behavior

• Anagha Kulkarni
• Yantian Zha
• Tathagata Chakraborti
• Satya Gautam Vadlamudi
• Yu Zhang
• Subbarao Kambhampati

In order to achieve effective human-AI collaboration, it is necessary for an AI agent to align its behavior with the human’s expectations. When the agent generates a task plan without such considerations, it may often result in inexplicable behavior from the human’s point of view. This may have serious implications for the human, from increased cognitive load to more serious concerns of safety around the physical agent. In this work, we present an approach to generate explicable behavior by minimizing the distance between the agent’s plan and the plan expected by the human. To this end, we learn a mapping between plan distances (distances between expected and agent plans) and human’s plan scoring scheme. The plan generation process uses this learned model as a heuristic. We demonstrate the effectiveness of our approach in a delivery robot domain.

AAAI Conference 2019 Conference Paper

Exploiting Coarse-to-Fine Task Transfer for Aspect-Level Sentiment Classification

• Zheng Li
• Ying Wei
• Yu Zhang
• Xiang Zhang
• Xin Li

Aspect-level sentiment classification (ASC) aims at identifying sentiment polarities towards aspects in a sentence, where the aspect can behave as a general Aspect Category (AC) or a specific Aspect Term (AT). However, due to the especially expensive and labor-intensive labeling, existing public corpora in AT-level are all relatively small. Meanwhile, most of the previous methods rely on complicated structures with given scarce data, which largely limits the efficacy of the neural models. In this paper, we exploit a new direction named coarse-to-fine task transfer, which aims to leverage knowledge learned from a rich-resource source domain of the coarse-grained AC task, which is more easily accessible, to improve the learning in a low-resource target domain of the fine-grained AT task. To resolve both the aspect granularity inconsistency and feature mismatch between domains, we propose a Multi-Granularity Alignment Network (MGAN). In MGAN, a novel Coarse2Fine attention guided by an auxiliary task can help the AC task modeling at the same finegrained level with the AT task. To alleviate the feature false alignment, a contrastive feature alignment method is adopted to align aspect-specific feature representations semantically. In addition, a large-scale multi-domain dataset for the AC task is provided. Empirically, extensive experiments demonstrate the effectiveness of the MGAN.

AAAI Conference 2019 Conference Paper

Gaussian Transformer: A Lightweight Approach for Natural Language Inference

• Maosheng Guo
• Yu Zhang
• Ting Liu

Natural Language Inference (NLI) is an active research area, where numerous approaches based on recurrent neural networks (RNNs), convolutional neural networks (CNNs), and self-attention networks (SANs) has been proposed. Although obtaining impressive performance, previous recurrent approaches are hard to train in parallel; convolutional models tend to cost more parameters, while self-attention networks are not good at capturing local dependency of texts. To address this problem, we introduce a Gaussian prior to selfattention mechanism, for better modeling the local structure of sentences. Then we propose an efficient RNN/CNN-free architecture named Gaussian Transformer for NLI, which consists of encoding blocks modeling both local and global dependency, high-order interaction blocks collecting the evidence of multi-step inference, and a lightweight comparison block saving lots of parameters. Experiments show that our model achieves new state-of-the-art performance on both SNLI and MultiNLI benchmarks with significantly fewer parameters and considerably less training time. Besides, evaluation using the Hard NLI datasets demonstrates that our approach is less affected by the undesirable annotation artifacts.

AAAI Conference 2019 System Paper

K3S: Knowledge-Driven Solution Support System

• Yu Zhang
• Morteza Saberi
• Min Wang
• Elizabeth Chang

As the volume of scientific papers grows rapidly in size, knowledge management for scientific publications is greatly needed. Information extraction and knowledge fusion techniques have been proposed to obtain information from scholarly publications and build knowledge repositories. However, retrieving the knowledge of problem/solution from academic papers to support users on solving specific research problems is rarely seen in the state of the art. Therefore, to remedy this gap, a knowledge-driven solution support system (K3S) is proposed in this paper to extract the information of research problems and proposed solutions from academic papers, and integrate them into knowledge maps. With the bibliometric information of the papers, K3S is capable of providing recommended solutions for any extracted problems. The subject of intrusion detection is chosen for demonstration in which required information is extracted with high accuracy, a knowledge map is constructed properly, and solutions to address intrusion problems are recommended.

AAAI Conference 2019 Conference Paper

Learning (from) Deep Hierarchical Structure among Features

• Yu Zhang
• Lei Han

Data features usually can be organized in a hierarchical structure to reflect the relations among them. Most of previous studies that utilize the hierarchical structure to help improve the performance of supervised learning tasks can only handle the structure of a limited height such as 2. In this paper, we propose a Deep Hierarchical Structure (DHS) method to handle the hierarchical structure of an arbitrary height with a convex objective function. The DHS method relies on the exponents of the edge weights in the hierarchical structure but the exponents need to be given by users or set to be identical by default, which may be suboptimal. Based on the DHS method, we propose a variant to learn the exponents from data. Moreover, we consider a case where even the hierarchical structure is not available. Based on the DHS method, we propose a Learning Deep Hierarchical Structure (LDHS) method which can learn the hierarchical structure via a generalized fused-Lasso regularizer and a proposed sequential constraint. All the optimization problems are solved by proximal methods where each subproblem has an efficient solution. Experiments on synthetic and real-world datasets show the effectiveness of the proposed methods.

JBHI Journal 2019 Journal Article

Pattern Classification for Gastrointestinal Stromal Tumors by Integration of Radiomics and Deep Convolutional Features

• Zhenyuan Ning
• Jiaxiu Luo
• Yong Li
• Shuai Han
• Qianjin Feng
• Yikai Xu
• Wufan Chen
• Tao Chen

Predicting malignant potential is one of the most critical components of a computer-aided diagnosis system for gastrointestinal stromal tumors (GISTs). These tumors have been studied only on the basis of subjective computed tomography findings. Among various methodologies, radiomics, and deep learning algorithms, specifically convolutional neural networks (CNNs), have recently been confirmed to achieve significant success by outperforming the state-of-the-art performance in medical image pattern classification and have rapidly become leading methodologies in this field. However, the existing methods generally use radiomics or deep convolutional features independently for pattern classification, which tend to take into account only global or local features, respectively. In this paper, we introduce and evaluate a hybrid structure that includes different features selected with radiomics model and CNNs and integrates these features to deal with GISTs classification. The Radiomics model and CNNs are constructed for global radiomics and local convolutional feature selection, respectively. Subsequently, we utilize distinct radiomics and deep convolutional features to perform pattern classification for GISTs. Specifically, we propose a new pooling strategy to assemble the deep convolutional features of 54 three-dimensional patches from the same case and integrate these features with the radiomics features for independent case, followed by random forest classifier. Our method can be extensively evaluated using multiple clinical datasets. The classification performance (area under the curve (AUC): 0. 882; 95% confidence interval (CI): 0. 816-0. 947) consistently outperforms those of independent radiomics (AUC: 0. 807; 95% CI: 0. 724-0. 892) and CNNs (AUC: 0. 826; 95% CI: 0. 795-0. 856) approaches.

JBHI Journal 2019 Journal Article

Sparse Group Representation Model for Motor Imagery EEG Classification

• Yong Jiao
• Yu Zhang
• Xun Chen
• Erwei Yin
• Jing Jin
• Xingyu Wang
• Andrzej Cichocki

A potential limitation of a motor imagery (MI) based brain-computer interface (BCI) is that it usually requires a relatively long time to record sufficient electroencephalogram (EEG) data for robust classifier training. The calibration burden during data acquisition phase will most probably cause a subject to be reluctant to use a BCI system. To alleviate this issue, we propose a novel sparse group representation model (SGRM) for improving the efficiency of MI-based BCI by exploiting the intersubject information. Specifically, preceded by feature extraction using common spatial pattern, a composite dictionary matrix is constructed with training samples from both the target subject and other subjects. By explicitly exploiting within-group sparse and group-wise sparse constraints, the most compact representation of a test sample of the target subject is then estimated as a linear combination of columns in the dictionary matrix. Classification is implemented by calculating the class-specific representation residual based on the significant training samples corresponding to the nonzero representation coefficients. Accordingly, the proposed SGRM method effectively reduces the required training samples from the target subject due to auxiliary data available from other subjects. With two public EEG data sets, extensive experimental comparisons are carried out between SGRM and other state-of-the-art approaches. Superior classification performance of our method using 40 trials of the target subject for model calibration (Averaged accuracy = 78. 2%, Kappa = 0. 57 and Averaged accuracy = 77. 7%, Kappa = 0. 55 for the two data sets, respectively) indicates its promising potential for improving the practicality of MI-based BCI.

JBHI Journal 2018 Journal Article

A Clinical Decision Support Framework for Heterogeneous Data Sources

• Mengxing Huang
• Huirui Han
• Hao Wang
• Lefei Li
• Yu Zhang
• Uzair Aslam Bhatti

To keep pace with the developments in medical informatics, health medical data is being collected continually. But, owing to the diversity of its categories and sources, medical data has become so complicated in many hospitals that it now needs a clinical decision support (CDS) system for its management. To effectively utilize the accumulating health data, we propose a CDS framework that can integrate heterogeneous health data from different sources such as laboratory test results, basic information of patients, and health records into a consolidated representation of features of all patients. Using the electronic health medical data so created, multilabel classification was employed to recommend a list of diseases and thus assist physicians in diagnosing or treating their patients' health issues more efficiently. Once the physician diagnoses the disease of a patient, the next step is to consider the likely complications of that disease, which can lead to more diseases. Previous studies reveal that correlations do exist among some diseases. Considering these correlations, a k-nearest neighbors algorithm is improved for multilabel learning by using correlations among labels (CML-kNN). The CML- kNN algorithm first exploits the dependence between every two labels to update the origin label matrix and then performs multilabel learning to estimate the probabilities of labels by using the integrated features. Finally, it recommends the top N diseases to the physicians. Experimental results on real health medical data establish the effectiveness and practicability of the proposed CDS framework.

AAAI Conference 2018 Conference Paper

Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification

• Zheng Li
• Ying Wei
• Yu Zhang
• Qiang Yang

Cross-domain sentiment classification aims to leverage useful information in a source domain to help do sentiment classification in a target domain that has no or little supervised information. Existing cross-domain sentiment classification methods cannot automatically capture non-pivots, i. e. , the domainspecific sentiment words, and pivots, i. e. , the domain-shared sentiment words, simultaneously. In order to solve this problem, we propose a Hierarchical Attention Transfer Network (HATN) for cross-domain sentiment classification. The proposed HATN provides a hierarchical attention transfer mechanism which can transfer attentions for emotions across domains by automatically capturing pivots and non-pivots. Besides, the hierarchy of the attention mechanism mirrors the hierarchical structure of documents, which can help locate the pivots and non-pivots better. The proposed HATN consists of two hierarchical attention networks, with one named P-net aiming to find the pivots and the other named NP-net aligning the non-pivots by using the pivots as a bridge. Specifically, P-net firstly conducts individual attention learning to provide positive and negative pivots for NP-net. Then, Pnet and NP-net conduct joint attention learning such that the HATN can simultaneously capture pivots and non-pivots and realize transferring attentions for emotions across domains. Experiments on the Amazon review dataset demonstrate the effectiveness of HATN.

YNIMG Journal 2018 Journal Article

How acute stress may enhance subsequent memory for threat stimuli outside the focus of attention: DLPFC-amygdala decoupling

• Yu Luo
• Guillén Fernández
• Erno Hermans
• Susanne Vogel
• Yu Zhang
• Hong Li
• Floris Klumpers

Stress-related disorders, e. g. , anxiety and depression, are characterized by decreased top-down control for distracting information, as well as a memory bias for threatening information. However, it is unclear how acute stress biases mnemonic encoding and leads to prioritized storage of threat-related information even if outside the focus of attention. In the current study, healthy adults (N = 53, all male) were randomly assigned to stress induction using the socially evaluated cold-pressor test (SECPT) or a control condition. Participants performed a task in which they were required to identify a target letter within a string of letters that were either identical to the target and thereby facilitating detection (low distractor load) or mixed with other letters to complicate the search (high load). Either a fearful or neutral face was presented on the background, outside the focus of attention. Twenty-four hours later, participants were asked to perform a surprise recognition memory test for those background faces. Stress induction resulted in increased cortisol and negative subjective mood ratings. Stress did not affect visual search performance, however, participants in the stress group showed stronger memory compared to the control group for fearful faces in the low attentional load condition. Critically, the stress induced memory bias was accompanied by decoupling between amygdala and DLFPC during encoding, which may represent a mechanism for decreased ability to filter task-irrelevant threatening background information. The current study provides a potential neural account for how stress can produce a negative memory bias for threatening information even if presented outside the focus of attention. Despite of an adaptive advantage for survival, such tendencies may ultimately also lead to generalized fear, a possibility requiring additional investigation.

AAAI Conference 2018 Conference Paper

Label Distribution Learning by Exploiting Label Correlations

• Xiuyi Jia
• Weiwei Li
• Junyu Liu
• Yu Zhang

Label distribution learning (LDL) is a newly arisen machine learning method that has been increasingly studied in recent years. In theory, LDL can be seen as a generalization of multilabel learning. Previous studies have shown that LDL is an effective approach to solve the label ambiguity problem. However, the dramatic increase in the number of possible label sets brings a challenge in performance to LDL. In this paper, we propose a novel label distribution learning algorithm to address the above issue. The key idea is to exploit correlations between different labels. We encode the label correlation into a distance to measure the similarity of any two labels. Moreover, we construct a distance-mapping function from the label set to the parameter matrix. Experimental results on eight real label distributed data sets demonstrate that the proposed algorithm performs remarkably better than both the state-ofthe-art LDL methods and multi-label learning methods.

NeurIPS Conference 2018 Conference Paper

Learning to Multitask

• Yu Zhang
• Ying Wei
• Qiang Yang

Multitask learning has shown promising performance in many applications and many multitask models have been proposed. In order to identify an effective multitask model for a given multitask problem, we propose a learning framework called Learning to MultiTask (L2MT). To achieve the goal, L2MT exploits historical multitask experience which is organized as a training set consisting of several tuples, each of which contains a multitask problem with multiple tasks, a multitask model, and the relative test error. Based on such training set, L2MT first uses a proposed layerwise graph neural network to learn task embeddings for all the tasks in a multitask problem and then learns an estimation function to estimate the relative test error based on task embeddings and the representation of the multitask model based on a unified formulation. Given a new multitask problem, the estimation function is used to identify a suitable multitask model. Experiments on benchmark datasets show the effectiveness of the proposed L2MT framework.

AAAI Conference 2018 Conference Paper

Personalizing a Dialogue System With Transfer Reinforcement Learning

• Kaixiang Mo
• Yu Zhang
• Shuangyin Li
• Jiajun Li
• Qiang Yang

It is difficult to train a personalized task-oriented dialogue system because the data collected from each individual is often insufficient. Personalized dialogue systems trained on a small dataset is likely to overfit and make it difficult to adapt to different user needs. One way to solve this problem is to consider a collection of multiple users as a source domain and an individual user as a target domain, and to perform transfer learning from the source domain to the target domain. By following this idea, we propose a PErsonalized Task-oriented diALogue (PETAL) system, a transfer reinforcement learning framework based on POMDP, to construct a personalized dialogue system. The PETAL system first learns common dialogue knowledge from the source domain and then adapts this knowledge to the target domain. The proposed PETAL system can avoid the negative transfer problem by considering differences between the source and target users in a personalized Q-function. Experimental results on a real-world coffee-shopping data and simulation data show that the proposed PETAL system can learn optimal policies for different users, and thus effectively improve the dialogue quality under the personalized setting.

NeurIPS Conference 2018 Conference Paper

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

• Ye Jia
• Yu Zhang
• Ron Weiss
• Quan Wang
• Jonathan Shen
• Fei Ren
• Zhifeng Chen
• Patrick Nguyen

We describe a neural network-based system for text-to-speech (TTS) synthesis that is able to generate speech audio in the voice of many different speakers, including those unseen during training. Our system consists of three independently trained components: (1) a speaker encoder network, trained on a speaker verification task using an independent dataset of noisy speech from thousands of speakers without transcripts, to generate a fixed-dimensional embedding vector from seconds of reference speech from a target speaker; (2) a sequence-to-sequence synthesis network based on Tacotron 2, which generates a mel spectrogram from text, conditioned on the speaker embedding; (3) an auto-regressive WaveNet-based vocoder that converts the mel spectrogram into a sequence of time domain waveform samples. We demonstrate that the proposed model is able to transfer the knowledge of speaker variability learned by the discriminatively-trained speaker encoder to the new task, and is able to synthesize natural speech from speakers that were not seen during training. We quantify the importance of training the speaker encoder on a large and diverse speaker set in order to obtain the best generalization performance. Finally, we show that randomly sampled speaker embeddings can be used to synthesize speech in the voice of novel speakers dissimilar from those used in training, indicating that the model has learned a high quality speaker representation.

AAAI Conference 2018 Conference Paper

Transferable Contextual Bandit for Cross-Domain Recommendation

• Bo Liu
• Ying Wei
• Yu Zhang
• Zhixian Yan
• Qiang Yang

Traditional recommendation systems (RecSys) suffer from two problems: the exploitation-exploration dilemma and the cold-start problem. One solution to solving the exploitationexploration dilemma is the contextual bandit policy, which adaptively exploits and explores user interests. As a result, the contextual bandit policy achieves increased rewards in the long run. The contextual bandit policy, however, may cause the system to explore more than needed in the cold-start situations, which can lead to worse short-term rewards. Crossdomain RecSys methods adopt transfer learning to leverage prior knowledge in a source RecSys domain to jump start the cold-start target RecSys. To solve the two problems together, in this paper, we propose the first applicable transferable contextual bandit (TCB) policy for the cross-domain recommendation. TCB not only benefits the exploitation but also accelerates the exploration in the target RecSys. TCB’s exploration, in turn, helps to learn how to transfer between different domains. TCB is a general algorithm for both homogeneous and heterogeneous domains. We perform both theoretical regret analysis and empirical experiments. The empirical results show that TCB outperforms the state-of-the-art algorithms over time.

IJCAI Conference 2017 Conference Paper

A Deep Neural Network for Chinese Zero Pronoun Resolution

• Qingyu Yin
• Weinan Zhang
• Yu Zhang
• Ting Liu

Existing approaches for Chinese zero pronoun resolution overlook semantic information. This is because zero pronouns have no descriptive information, which results in difficulty in explicitly capturing their semantic similarities with antecedents. Moreover, when dealing with candidate antecedents, traditional systems simply take advantage of the local information of a single candidate antecedent while failing to consider the underlying information provided by the other candidates from a global perspective. To address these weaknesses, we propose a novel zero pronoun-specific neural network, which is capable of representing zero pronouns by utilizing the contextual information at the semantic level. In addition, when dealing with candidate antecedents, a two-level candidate encoder is employed to explicitly capture both the local and global information of candidate antecedents. We conduct experiments on the Chinese portion of the OntoNotes 5. 0 corpus. Experimental results show that our approach substantially outperforms the state-of-the-art method in various experimental settings.

YNIMG Journal 2017 Journal Article

Cross-cultural consistency and diversity in intrinsic functional organization of Broca's Region

• Yu Zhang
• Lingzhong Fan
• Svenja Caspers
• Stefan Heim
• Ming Song
• Cirong Liu
• Yin Mo
• Simon B. Eickhoff

As a core language area, Broca's region was consistently activated in a variety of language studies even across different language systems. Moreover, a high degree of structural and functional heterogeneity in Broca's region has been reported in many studies. This raised the issue of how the intrinsic organization of Broca's region effects by different language experiences in light of its subdivisions. To address this question, we used multi-center resting-state fMRI data to explore the cross-cultural consistency and diversity of Broca's region in terms of its subdivisions, connectivity patterns and modularity organization in Chinese and German speakers. A consistent topological organization of the 13 subdivisions within the extended Broca's region was revealed on the basis of a new in-vivo parcellation map, which corresponded well to the previously reported receptorarchitectonic map. Based on this parcellation map, consistent functional connectivity patterns and modularity organization of these subdivisions were found. Some cultural difference in the functional connectivity patterns was also found, for instance stronger connectivity in Chinese subjects between area 6v2 and the motor hand area, as well as higher correlations between area 45p and middle frontal gyrus. Our study suggests that a generally invariant organization of Broca's region, together with certain regulations of different language experiences on functional connectivity, might exists to support language processing in human brain.

IJCAI Conference 2017 Conference Paper

Deep Neural Networks for High Dimension, Low Sample Size Data

• Bo Liu
• Ying Wei
• Yu Zhang
• Qiang Yang

Deep neural networks (DNN) have achieved breakthroughs in applications with large sample size. However, when facing high dimension, low sample size (HDLSS) data, such as the phenotype prediction problem using genetic data in bioinformatics, DNN suffers from overfitting and high-variance gradients. In this paper, we propose a DNN model tailored for the HDLSS data, named Deep Neural Pursuit (DNP). DNP selects a subset of high dimensional features for the alleviation of overfitting and takes the average over multiple dropouts to calculate gradients with low variance. As the first DNN method applied on the HDLSS data, DNP enjoys the advantages of the high nonlinearity, the robustness to high dimensionality, the capability of learning from a small number of samples, the stability in feature selection, and the end-to-end training. We demonstrate these advantages of DNP via empirical results on both synthetic and real-world biological datasets.

AAAI Conference 2017 Conference Paper

Distant Domain Transfer Learning

• Ben Tan
• Yu Zhang
• Sinno Pan
• Qiang Yang

In this paper, we study a novel transfer learning problem termed Distant Domain Transfer Learning (DDTL). Different from existing transfer learning problems which assume that there is a close relation between the source domain and the target domain, in the DDTL problem, the target domain can be totally different from the source domain. For example, the source domain classifies face images but the target domain distinguishes plane images. Inspired by the cognitive process of human where two seemingly unrelated concepts can be connected by learning intermediate concepts gradually, we propose a Selective Learning Algorithm (SLA) to solve the DDTL problem with supervised autoencoder or supervised convolutional autoencoder as a base model for handling different types of inputs. Intuitively, the SLA algorithm selects usefully unlabeled data gradually from intermediate domains as a bridge to break the large distribution gap for transferring knowledge between two distant domains. Empirical studies on image classification problems demonstrate the effectiveness of the proposed algorithm, and on some tasks the improvement in terms of the classification accuracy is up to 17% over “non-transfer” methods.

IJCAI Conference 2017 Conference Paper

End-to-End Adversarial Memory Network for Cross-domain Sentiment Classification

• Zheng Li
• Yu Zhang
• Ying Wei
• Yuxiang Wu
• Qiang Yang

Domain adaptation tasks such as cross-domain sentiment classification have raised much attention in recent years. Due to the domain discrepancy, a sentiment classifier trained in a source domain may not work well when directly applied to a target domain. Traditional methods need to manually select pivots, which behave in the same way for discriminative learning in both domains. Recently, deep learning methods have been proposed to learn a representation shared by domains. However, they lack the interpretability to directly identify the pivots. To address the problem, we introduce an end-to-end Adversarial Memory Network (AMN) for cross-domain sentiment classification. Unlike existing methods, our approach can automatically capture the pivots using an attention mechanism. Our framework consists of two parameter-shared memory networks: one is for sentiment classification and the other is for domain classification. The two networks are jointly trained so that the selected features minimize the sentiment classification error and at the same time make the domain classifier indiscriminative between the representations from the source or target domains. Moreover, unlike deep learning methods that cannot tell us which words are the pivots, our approach can offer a direct visualization of them. Experiments on the Amazon review dataset demonstrate that our approach can significantly outperform state-of-the-art methods.

AAAI Conference 2017 Conference Paper

Learning Sparse Task Relations in Multi-Task Learning

• Yu Zhang
• Qiang Yang

In multi-task learning, when the number of tasks is large, pairwise task relations exhibit sparse patterns since usually a task cannot be helpful to all of the other tasks and moreover, sparse task relations can reduce the risk of overfitting compared with the dense ones. In this paper, we focus on learning sparse task relations. Based on a regularization framework which can learn task relations among multiple tasks, we propose a SParse covAriance based mulTi-taSk (SPATS) model to learn a sparse covariance by using the 1 regularization. The resulting objective function of the SPATS method is convex, which allows us to devise an alternating method to solve it. Moreover, some theoretical properties of the proposed model are studied. Experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

IJCAI Conference 2017 Conference Paper

Multimodal Linear Discriminant Analysis via Structural Sparsity

• Yu Zhang
• Yuan Jiang

Linear discriminant analysis (LDA) is a widely used supervised dimensionality reduction technique. Even though the LDA method has many real-world applications, it has some limitations such as the single-modal problem that each class follows a normal distribution. To solve this problem, we propose a method called multimodal linear discriminant analysis (MLDA). By generalizing the between-class and within-class scatter matrices, the MLDA model can allow each data point to have its own class mean which is called the instance-specific class mean. Then in each class, data points which share the same or similar instance-specific class means are considered to form one cluster or modal. In order to learn the instance-specific class means, we use the ratio of the proposed generalized between-class scatter measure over the proposed generalized within-class scatter measure, which encourages the class separability, as a criterion. The observation that each class will have a limited number of clusters inspires us to use a structural sparse regularizor to control the number of unique instance-specific class means in each class. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed MLDA method.

IJCAI Conference 2017 Conference Paper

Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy

• Tathagata Chakraborti
• Sarath Sreedharan
• Yu Zhang
• Subbarao Kambhampati

When AI systems interact with humans in the loop, they are often called on to provide explanations for their plans and behavior. Past work on plan explanations primarily involved the AI system explaining the correctness of its plan and the rationale for its decision in terms of its own model. Such soliloquy is wholly inadequate in most realistic scenarios where the humans have domain and task models that differ significantly from that used by the AI system. We posit that the explanations are best studied in light of these differing models. In particular, we show how explanation can be seen as a "model reconciliation problem" (MRP), where the AI system in effect suggests changes to the human's model, so as to make its plan be optimal with respect to that changed human model. We will study the properties of such explanations, present algorithms for automatically computing them, and evaluate the performance of the algorithms.

YNIMG Journal 2017 Journal Article

Predicting individualized clinical measures by a generalized prediction framework and multimodal fusion of MRI data

• Xing Meng
• Rongtao Jiang
• Dongdong Lin
• Juan Bustillo
• Thomas Jones
• Jiayu Chen
• Qingbao Yu
• Yuhui Du

Neuroimaging techniques have greatly enhanced the understanding of neurodiversity (human brain variation across individuals) in both health and disease. The ultimate goal of using brain imaging biomarkers is to perform individualized predictions. Here we proposed a generalized framework that can predict explicit values of the targeted measures by taking advantage of joint information from multiple modalities. This framework also enables whole brain voxel-wise searching by combining multivariate techniques such as ReliefF, clustering, correlation-based feature selection and multiple regression models, which is more flexible and can achieve better prediction performance than alternative atlas-based methods. For 50 healthy controls and 47 schizophrenia patients, three kinds of features derived from resting-state fMRI (fALFF), sMRI (gray matter) and DTI (fractional anisotropy) were extracted and fed into a regression model, achieving high prediction for both cognitive scores (MCCB composite r =0. 7033, MCCB social cognition r =0. 7084) and symptomatic scores (positive and negative syndrome scale [PANSS] positive r =0. 7785, PANSS negative r =0. 7804). Moreover, the brain areas likely responsible for cognitive deficits of schizophrenia, including middle temporal gyrus, dorsolateral prefrontal cortex, striatum, cuneus and cerebellum, were located with different weights, as well as regions predicting PANSS symptoms, including thalamus, striatum and inferior parietal lobule, pinpointing the potential neuromarkers. Finally, compared to a single modality, multimodal combination achieves higher prediction accuracy and enables individualized prediction on multiple clinical measures. There is more work to be done, but the current results highlight the potential utility of multimodal brain imaging biomarkers to eventually inform clinical decision-making.

AAAI Conference 2017 Conference Paper

Recurrent Attentional Topic Model

• Shuangyin Li
• Yu Zhang
• Rong Pan
• Mingzhi Mao
• Yang Yang

In a document, the topic distribution of a sentence depends on both the topics of preceding sentences and its own content, and it is usually affected by the topics of the preceding sentences with different weights. It is natural that a document can be treated as a sequence of sentences. Most existing works for Bayesian document modeling do not take these points into consideration. To fill this gap, we propose a Recurrent Attentional Topic Model (RATM) for document embedding. The RATM not only takes advantage of the sequential orders among sentence but also use the attention mechanism to model the relations among successive sentences. In RATM, we propose a Recurrent Attentional Bayesian Process (RABP) to handle the sequences. Based on the RABP, RATM fully utilizes the sequential information of the sentences in a document. Experiments on two copora show that our model outperforms state-of-the-art methods on document modeling and classification.

ICRA Conference 2017 Conference Paper

Sequence-based multimodal apprenticeship learning for robot perception and decision making

• Fei Han 0002
• Xue Yang
• Yu Zhang
• Hao Zhang 0011

Apprenticeship learning has recently attracted a wide attention due to its capability of allowing robots to learn physical tasks directly from demonstrations provided by human experts. Most previous techniques assumed that the state space is known a priori or employed simple state representations that usually suffer from perceptual aliasing. Different from previous research, we propose a novel approach named Sequence-based Multimodal Apprenticeship Learning (SMAL), which is capable to simultaneously fusing temporal information and multimodal data, and to integrate robot perception with decision making. To evaluate the SMAL approach, experiments are performed using both simulations and real-world robots in the challenging search and rescue scenarios. The empirical study has validated that our SMAL approach can effectively learn plans for robots to make decisions using sequence of multimodal observations. Experimental results have also showed that SMAL outperforms the baseline methods using individual images.

ICRA Conference 2017 Conference Paper

Simultaneous Feature and Body-Part Learning for real-time robot awareness of human behaviors

• Fei Han 0002
• Xue Yang
• Christopher M. Reardon
• Yu Zhang
• Hao Zhang 0011

Robot awareness of human actions is an essential research problem in robotics with many important real-world applications, including human-robot collaboration and teaming. Over the past few years, depth sensors have become a standard device widely used by intelligent robots for 3D perception, which can also offer human skeletal data in 3D space. Several methods based on skeletal data were designed to enable robot awareness of human actions with satisfactory accuracy. However, previous methods treated all body parts and features equally important, without the capability to identify discriminative body parts and features. In this paper, we propose a novel simultaneous Feature And Body-part Learning (FABL) approach that simultaneously identifies discriminative body parts and features, and efficiently integrates all available information together to enable real-time robot awareness of human behaviors. We formulate FABL as a regression-like optimization problem with structured sparsity-inducing norms to model interrelationships of body parts and features. We also develop an optimization algorithm to solve the formulated problem, which possesses a theoretical guarantee to find the optimal solution. To evaluate FABL, three experiments were performed using public benchmark datasets, including the MSR Action3D and CAD-60 datasets, as well as a Baxter robot in practical assistive living applications. Experimental results show that our FABL approach obtains a high recognition accuracy with a processing speed of the order-of-magnitude of 10 1 Hz, which makes FABL a promising method to enable real-time robot awareness of human behaviors in practical robotics applications.

YNIMG Journal 2017 Journal Article

The quantification of blood-brain barrier disruption using dynamic contrast-enhanced magnetic resonance imaging in aging rhesus monkeys with spontaneous type 2 diabetes mellitus

• Ziqian Xu
• Wen Zeng
• Jiayu Sun
• Wei Chen
• Ruzhi Zhang
• Zunyuan Yang
• Zunwei Yao
• Lei Wang

Microvascular lesions of the body are one of the most serious complications that can affect patients with type 2 diabetes mellitus. The blood-brain barrier (BBB) is a highly selective permeable barrier around the microvessels of the brain. This study investigated BBB disruption in diabetic rhesus monkeys using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). Multi-slice DCE-MRI was used to quantify BBB permeability. Five diabetic monkeys and six control monkeys underwent magnetic resonance brain imaging in 3 Tesla MRI system. Regions of the frontal cortex, the temporal cortex, the basal ganglia, the thalamus, and the hippocampus in the two groups were selected as regions of interest to calculate the value of the transport coefficient Ktrans using the extended Tofts model. Permeability in the diabetic monkeys was significantly increased as compared with permeability in the normal control monkeys. Histopathologically, zonula occludens protein-1 decreased, immunoglobulin G leaked out of the blood, and nuclear factor E2–related factor translocated from the cytoplasm to the nuclei. It is likely that diabetes contributed to the increased BBB permeability.

NeurIPS Conference 2017 Conference Paper

Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data

• Wei-Ning Hsu
• Yu Zhang
• James Glass

We present a factorized hierarchical variational autoencoder, which learns disentangled and interpretable representations from sequential data without supervision. Specifically, we exploit the multi-scale nature of information in sequential data by formulating it explicitly within a factorized hierarchical graphical model that imposes sequence-dependent priors and sequence-independent priors to different sets of latent variables. The model is evaluated on two speech corpora to demonstrate, qualitatively, its ability to transform speakers or linguistic content by manipulating different sets of latent variables; and quantitatively, its ability to outperform an i-vector baseline for speaker verification and reduce the word error rate by as much as 35% in mismatched train/test scenarios for automatic speech recognition tasks.

AAAI Conference 2016 Conference Paper

Multi-Stage Multi-Task Learning with Reduced Rank

• Lei Han
• Yu Zhang

Multi-task learning (MTL) seeks to improve the generalization performance by sharing information among multiple tasks. Many existing MTL approaches aim to learn the lowrank structure on the weight matrix, which stores the model parameters of all tasks, to achieve task sharing, and as a consequence the trace norm regularization is widely used in the MTL literature. A major limitation of these approaches based on trace norm regularization is that all the singular values of the weight matrix are penalized simultaneously, leading to impaired estimation on recovering the larger singular values in the weight matrix. To address the issue, we propose a Reduced rAnk MUlti-Stage multi-tAsk learning (RAMUSA) method based on the recently proposed capped norms. Different from existing trace-norm-based MTL approaches which minimize the sum of all the singular values, the RAMUSA method uses a capped trace norm regularizer to minimize only the singular values smaller than some threshold. Due to the non-convexity of the capped trace norm, we develop a simple but well guaranteed multi-stage algorithm to learn the weight matrix iteratively. We theoretically prove that the estimation error at each stage in the proposed algorithm shrinks and finally achieves a lower upper-bound as the number of stages becomes large enough. Empirical studies on synthetic and real-world datasets demonstrate the effectiveness of the RAMUSA method in comparison with the state-of-the-art methods.

AAMAS Conference 2016 Conference Paper

Planning with Resource Conflicts in Human-Robot Cohabitation

• Tathagata Chakraborti
• Yu Zhang
• David E. Smith
• Subbarao Kambhampati

AAAI Conference 2016 Conference Paper

Reduction Techniques for Graph-Based Convex Clustering

• Lei Han
• Yu Zhang

The Graph-based Convex Clustering (GCC) method has gained increasing attention recently. The GCC method adopts a fused regularizer to learn the cluster centers and obtains a geometric clusterpath by varying the regularization parameter. One major limitation is that solving the GCC model is computationally expensive. In this paper, we develop efficient graph reduction techniques for the GCC model to eliminate edges, each of which corresponds to two data points from the same cluster, without solving the optimization problem in the GCC method, leading to improved computational efficiency. Specifically, two reduction techniques are proposed according to tree-based and cyclic-graph-based convex clustering methods separately. The proposed reduction techniques are appealing since they only need to scan the data once with negligibly additional cost and they are independent of solvers for the GCC method, making them capable of improving the efficiency of any existing solver. Experiments on both synthetic and real-world datasets show that our methods can largely improve the efficiency of the GCC model.

IROS Conference 2015 Conference Paper

A practical and effective method for identifying the complete inertia parameters of space robots

• Wenfu Xu
• Zhonghua Hu
• Yu Zhang
• Zhiying Wang 0006
• Xinyu Wu 0001

This paper propose a practical and effective method for identifying the complete inertia parameters of a space robotic system. The key is the following steps: equivalent single-body identification, and equivalent two-body identification. For the former, all joints are locked into a normal configuration, and the thrusters are used for orbital maneuvering. The object function is defined from the view of acceleration and velocity. For the latter, only one joint is unlocked and driven to move along the exciting trajectory under the free-floating mode. The linear and angular momentum equations are used to define the object function. Hence, the parameter identification problem is transformed into a non-linear optimization problem, and the PSO algorithm is used to determine the optimal parameters. By sequentially unlocking the 1st to nth joint, the mass properties of body 0 to n are completely identified. The proposed method is general and can be easily extended for many cases.

AAAI Conference 2015 Conference Paper

Discriminative Feature Grouping

• Lei Han
• Yu Zhang

Feature grouping has been demonstrated to be promising in learning with high-dimensional data. It helps reduce the variances in the estimation and improves the stability of feature selection. One major limitation of existing feature grouping approaches is that some similar but different feature groups are often mis-fused, leading to impaired performance. In this paper, we propose a Discriminative Feature Grouping (DFG) method to discover the feature groups with enhanced discrimination. Different from existing methods, DFG adopts a novel regularizer for the feature coefficients to tradeoff between fusing and discriminating feature groups. The proposed regularizer consists of a 1 norm to enforce feature sparsity and a pairwise ∞ norm to encourage the absolute differences among any three feature coefficients to be similar. To achieve better asymptotic property, we generalize the proposed regularizer to an adaptive one where the feature coefficients are weighted based on the solution of some estimator with root-n consistency. For optimization, we employ the alternating direction method of multipliers to solve the proposed methods efficiently. Experimental results on synthetic and real-world datasets demonstrate that the proposed methods have good performance compared with the state-of-the-art feature grouping methods.

AAAI Conference 2015 Conference Paper

Exploring Key Concept Paraphrasing Based on Pivot Language Translation for Question Retrieval

• Wei-Nan Zhang
• Zhao-Yan Ming
• Yu Zhang
• Ting Liu
• Tat-Seng Chua

Question retrieval in current community-based question answering (CQA) services does not, in general, work well for long and complex queries. One of the main difficulties lies in the word mismatch between queries and candidate questions. Existing solutions try to expand the queries at word level, but they usually fail to consider concept level enrichment. In this paper, we explore a pivot language translation based approach to derive the paraphrases of key concepts. We further propose a unified question retrieval model which integrates the key concepts and their paraphrases for the query question. Experimental results demonstrate that the paraphrase enhanced retrieval model significantly outperforms the state-of-the-art models in question retrieval.

I&C Journal 2015 Journal Article

Formal security proofs with minimal fuss: Implicit computational complexity at work

• David Nowak
• Yu Zhang

We show how implicit computational complexity can be used in order to increase confidence in game-based security proofs in cryptography. For this purpose we extend CSLR, a probabilistic lambda-calculus with a type system that guarantees the existence of a probabilistic polynomial-time bound on computations. This allows us to define cryptographic constructions, feasible adversaries, security notions, computational assumptions, game transformations, and game-based security proofs in a unified framework. We also show that the standard practice of cryptographers, ignoring that polynomial-time Turing machines cannot generate all uniform distributions, is actually sound. We illustrate our calculus on cryptographic constructions for public-key encryption and pseudorandom bit generation.

AAAI Conference 2015 Conference Paper

Learning Multi-Level Task Groups in Multi-Task Learning

• Lei Han
• Yu Zhang

In multi-task learning (MTL), multiple related tasks are learned jointly by sharing information across them. Many MTL algorithms have been proposed to learn the underlying task groups. However, those methods are limited to learn the task groups at only a single level, which may be not sufficient to model the complex structure among tasks in many real-world applications. In this paper, we propose a Multi-Level Task Grouping (MeTaG) method to learn the multi-level grouping structure instead of only one level among tasks. Specifically, by assuming the number of levels to be H, we decompose the parameter matrix into a sum of H component matrices, each of which is regularized with a 2 norm on the pairwise difference among parameters of all the tasks to construct level-specific task groups. For optimization, we employ the smoothing proximal gradient method to efficiently solve the objective function of the MeTaG model. Moreover, we provide theoretical analysis to show that under certain conditions the MeTaG model can recover the true parameter matrix and the true task groups in each level with high probability. We experiment our approach on both synthetic and real-world datasets, showing competitive performance over state-of-the-art MTL methods.

AAAI Conference 2015 Conference Paper

Multi-Task Learning and Algorithmic Stability

• Yu Zhang

In this paper, we study multi-task algorithms from the perspective of the algorithmic stability. We give a definition of the multi-task uniform stability, a generalization of the conventional uniform stability, which measures the maximum difference between the loss of a multi-task algorithm trained on a data set and that of the multitask algorithm trained on the same data set but with a data point removed in each task. In order to analyze multi-task algorithms based on multi-task uniform stability, we prove a generalized McDiarmid’s inequality which assumes the difference bound condition holds by changing multiple input arguments instead of only one in the conventional McDiarmid’s inequality. By using the generalized McDiarmid’s inequality as a tool, we can analyze the generalization performance of general multitask algorithms in terms of the multi-task uniform stability. Moreover, as applications, we prove generalization bounds of several representative regularized multi-task algorithms.

TCS Journal 2015 Journal Article

Program equivalence in linear contexts

• Yuxin Deng
• Yu Zhang

Program equivalence in linear contexts, where programs are used or executed exactly once, is an important issue in programming languages. However, existing techniques like those based on bisimulations and logical relations only target at contextual equivalence in the usual (non-linear) functional languages, and fail in capturing non-trivial equivalent programs in linear contexts, particularly when non-determinism is present. We propose the notion of linear contextual equivalence to formally characterize such program equivalence, as well as a novel and general approach to studying it in higher-order languages, based on labeled transition systems specifically designed for functional languages. We show that linear contextual equivalence indeed coincides with trace equivalence. We illustrate our technique in both deterministic (a linear version of PCF) and non-deterministic (linear PCF in Moggi's framework) functional languages.

YNIMG Journal 2014 Journal Article

Brain extraction based on locally linear representation-based classification

• Meiyan Huang
• Wei Yang
• Jun Jiang
• Yao Wu
• Yu Zhang
• Wufan Chen
• Qianjin Feng

Brain extraction is an important procedure in brain image analysis. Although numerous brain extraction methods have been presented, enhancing brain extraction methods remains challenging because brain MRI images exhibit complex characteristics, such as anatomical variability and intensity differences across different sequences and scanners. To address this problem, we present a Locally Linear Representation-based Classification (LLRC) method for brain extraction. A novel classification framework is derived by introducing the locally linear representation to the classical classification model. Under this classification framework, a common label fusion approach can be considered as a special case and thoroughly interpreted. Locality is important to calculate fusion weights for LLRC; this factor is also considered to determine that Local Anchor Embedding is more applicable in solving locally linear coefficients compared with other linear representation approaches. Moreover, LLRC supplies a way to learn the optimal classification scores of the training samples in the dictionary to obtain accurate classification. The International Consortium for Brain Mapping and the Alzheimer's Disease Neuroimaging Initiative databases were used to build a training dataset containing 70 scans. To evaluate the proposed method, we used four publicly available datasets (IBSR1, IBSR2, LPBA40, and ADNI3T, with a total of 241 scans). Experimental results demonstrate that the proposed method outperforms the four common brain extraction methods (BET, BSE, GCUT, and ROBEX), and is comparable to the performance of BEaST, while being more accurate on some datasets compared with BEaST.

AAAI Conference 2014 Conference Paper

Encoding Tree Sparsity in Multi-Task Learning: A Probabilistic Framework

• Lei Han
• Yu Zhang
• Guojie Song
• Kunqing Xie

Multi-task learning seeks to improve the generalization performance by sharing common information among multiple related tasks. A key assumption in most MTL algorithms is that all tasks are related, which, however, may not hold in many real-world applications. Existing techniques, which attempt to address this issue, aim to identify groups of related tasks using group sparsity. In this paper, we propose a probabilistic tree sparsity (PTS) model to utilize the tree structure to obtain the sparse solution instead of the group structure. Specifically, each model coefficient in the learning model is decomposed into a product of multiple component coefficients each of which corresponds to a node in the tree. Based on the decomposition, Gaussian and Cauchy distributions are placed on the component coefficients as priors to restrict the model complexity. We devise an efficient expectation maximization algorithm to learn the model parameters. Experiments conducted on both synthetic and real-world problems show the effectiveness of our model compared with state-of-the-art baselines.

NeurIPS Conference 2013 Conference Paper

Heterogeneous-Neighborhood-based Multi-Task Local Learning Algorithms

• Yu Zhang

All the existing multi-task local learning methods are defined on homogeneous neighborhood which consists of all data points from only one task. In this paper, different from existing methods, we propose local learning methods for multi-task classification and regression problems based on heterogeneous neighborhood which is defined on data points from all tasks. Specifically, we extend the k-nearest-neighbor classifier by formulating the decision function for each data point as a weighted voting among the neighbors from all tasks where the weights are task-specific. By defining a regularizer to enforce the task-specific weight matrix to approach a symmetric one, a regularized objective function is proposed and an efficient coordinate descent method is developed to solve it. For regression problems, we extend the kernel regression to multi-task setting in a similar way to the classification case. Experiments on some toy data and real-world datasets demonstrate the effectiveness of our proposed methods.

IJCAI Conference 2013 Conference Paper

Learning High-Order Task Relationships in Multi-Task Learning

• Yu Zhang
• Dit-Yan Yeung

Multi-task learning is a way of bringing inductive transfer studied in human learning to the machine learning community. A central issue in multi-task learning is to model the relationships between tasks appropriately and exploit them to aid the simultaneous learning of multiple tasks effectively. While some recent methods model and learn the task relationships from data automatically, only pairwise relationships can be represented by them. In this paper, we propose a new model, called Multi- Task High-Order relationship Learning (MTHOL), which extends in a novel way the use of pairwise task relationships to high-order task relationships. We first propose an alternative formulation of an existing multi-task learning method. Based on the new formulation, we propose a high-order generalization leading to a new prior for the model parameters of different tasks. We then propose a new probabilistic model for multi-task learning and validate it empirically on some benchmark datasets.

ICRA Conference 2013 Conference Paper

Multi-robot task scheduling

• Yu Zhang
• Lynne E. Parker

The scheduling problem has been studied extensively in the literature. Many algorithms have been developed to operate with different types of processors and tasks. In the robotics domain, when considering each robot as a processor, some of these algorithms can be directly adapted. However, most of the existing algorithms can only handle single-robot tasks, or multi-robot tasks that can be divided into single-robot tasks. As the task requirements may only be partially known, and the available (heterogeneous) robots can dynamically change, robots may be required to cooperate tightly to share different capabilities (i. e. , sensors and motors). In such cases, considering scheduling for individual robots is no longer sufficient, since the robots need to work at the coalition level. Although there exist a few algorithms that also support these more complex cases, they do not represent efficient solutions that can be adapted by various multi-robot systems in a convenient manner. In this paper, we propose heuristics to address the multi-robot task scheduling problem at the coalition level, which hides the details of robot specifications, thus allowing these heuristics to be incorporated straightforwardly. These heuristics are easy to implement and efficient enough to run in real time. We provide formal analyses and simulation results to demonstrate and compare their performances.

JAAMAS Journal 2012 Journal Article

Considering inter-task resource constraints in task allocation

• Yu Zhang
• Lynne E. Parker

Abstract This paper focuses on task allocation with single-task robots, multi-robot tasks and instantaneous assignment, which has been shown to be strongly NP-hard. Although this problem has been studied extensively, few efficient approximation algorithms have been provided due to its inherent complexity. In this paper, we first provide discussions and analyses for two natural greedy heuristics for solving this problem. Then, a new greedy heuristic is introduced, which considers inter-task resource constraints to approximate the influence between different assignments in task allocation. Instead of only looking at the utility of the assignment, our approach computes the expected loss of utility (due to the assigned robots and task) as an offset and uses the offset utility for making the greedy choice. A formal analysis is provided for the new heuristic, which reveals that the solution quality is bounded by two different factors. A new algorithm is then provided to approximate the new heuristic for performance improvement. Finally, for more complicated applications, we extend this problem to include general task dependencies and provide a result on the hardness of approximating this new formulation. Comparison results with the two natural heuristics in simulation are provided for both formulations, which show that the new approach achieves improved performance.

YNIMG Journal 2012 Journal Article

Linking white matter integrity loss to associated cortical regions using structural connectivity information in Alzheimer's disease and fronto-temporal dementia: The Loss in Connectivity (LoCo) score

• Amy Kuceyeski
• Yu Zhang
• Ashish Raj

It is well known that gray matter changes occur in neurodegenerative diseases like Alzheimer's (AD) and fronto-temporal dementia (FTD), and several studies have investigated their respective patterns of atrophy progression. Recent work, however, has revealed that diffusion MRI that is able to detect white matter integrity changes may be an earlier or more sensitive biomarker in both diseases. However, studies that examine white matter changes only are limited in that they do not provide the functional specificity of GM region-based analysis. In this study, we develop a new metric called the Loss in Connectivity (LoCo) score that gives the amount of structural network disruption incurred by a gray matter region for a particular pattern of white matter integrity loss. Leveraging the relative strengths of WM and GM markers, this metric links areas of WM integrity loss to their connected GM regions as a first step in understanding their functional implications. The LoCo score is calculated for three groups: 18AD, 18 FTD, and 19 age-matched normal controls. We show significant correlations of the LoCo with the respective atrophy patterns in AD (R=0. 51, p=2. 2×10−9) and FTD (R=0. 49, p=2. 5×10−8) for a standard 116 region gray matter atlas. In addition, we demonstrate that the LoCo outperforms a measure of gray matter atrophy when classifying individuals into AD, FTD, and normal groups.

AAAI Conference 2012 Conference Paper

Supervised Probabilistic Robust Embedding with Sparse Noise

• Yu Zhang
• Dit-Yan Yeung
• Eric Xing

Many noise models do not faithfully reflect the noise processes introduced during data collection in many real-world applications. In particular, we argue that a type of noise referred to as sparse noise is quite commonly found in many applications and many existing works have been proposed to model such sparse noise. However, all the existing works only focus on unsupervised learning without considering the supervised information, i. e. , label information. In this paper, we consider how to model and handle sparse noise in the context of embedding high-dimensional data under a probabilistic formulation for supervised learning. We propose a supervised probabilistic robust embedding (SPRE) model in which data are corrupted either by sparse noise or by a combination of Gaussian and sparse noises. By using the Laplace distribution as a prior to model sparse noise, we devise a twofold variational EM learning algorithm in which the update of model parameters has analytical solution. We report some classification experiments to compare SPRE with several related models.

ICRA Conference 2012 Conference Paper

Task allocation with executable coalitions in multirobot tasks

• Yu Zhang
• Lynne E. Parker

In our prior work, we proposed the IQ-ASyMTRe architecture with a measure of information quality to reason about forming coalitions in multirobot tasks. The formed coalitions are guaranteed to be executable, given the current configurations of the robots and environment. A cost and a quality measure are associated with each coalition to further determine its utility for the task. In this paper, we show that IQ-ASyMTRe-like architectures can be utilized to significantly reduce the overall complexity of task allocation by considering only executable coalitions. For implementation, we apply a layering technique such that most existing methods for task allocation can be easily incorporated. Furthermore, we introduce a general process to address situations in which no executable coalitions are available for certain tasks, and integrate it with IQ-ASyMTRe to achieve more autonomy. Such an approach is able to autonomously decompose unsatisfied preconditions of the required task behaviors into satisfiable components, in order to generate partial order plans for them accordingly. We show how this process can be implemented using a market-based approach. Simulation results are provided to demonstrate these techniques.

YNIMG Journal 2012 Journal Article

Tractography‐based parcellation of the human left inferior parietal lobule

• Jiaojian Wang
• Lingzhong Fan
• Yu Zhang
• Yong Liu
• Di Jiang
• Yuanchao Zhang
• Chunshui Yu
• Tianzi Jiang

The inferior parietal lobule (IPL) is a functionally and anatomically heterogeneous region. Much of the information about the anatomical connectivity and parcellation of this region was obtained from histological studies on non-human primates. However, whether these findings from non-human primates can be applied to the human inferior parietal lobule, especially the left inferior parietal lobule, which shows evidence of considerable evolution from primates to humans, remains unclear. In this study, diffusion MRI was employed to investigate the anatomical connectivities of the human left inferior parietal lobule. Using a new algorithm, spectral clustering with edge-weighted centroidal voronoi tessellations, to search for regional variations in the probabilistic connectivity profiles of all left inferior parietal lobule voxels with all the rest of the brain identified six subregions with distinctive connectivity properties in the left inferior parietal lobule. Consistent with cytoarchitectonic findings, four subregions were found in the left supramarginal gyrus and two subregions in the left angular gyrus. The specific connectivity patterns of each subregion of the left inferior parietal lobule were supported by both the anatomical and functional connectivity properties for each subregion, as calculated by a meta-analysis-based target method and by voxel-based whole brain anatomical and functional connectivity analyses. The proposed parcellation scheme for the human left inferior parietal lobule and the maximum probability map for each subregion may facilitate more detailed future studies of this brain area.

TIST Journal 2012 Journal Article

Transfer Metric Learning with Semi-Supervised Extension

• Yu Zhang
• Dit-Yan Yeung

Distance metric learning plays a very crucial role in many data mining algorithms because the performance of an algorithm relies heavily on choosing a good metric. However, the labeled data available in many applications is scarce, and hence the metrics learned are often unsatisfactory. In this article, we consider a transfer-learning setting in which some related source tasks with labeled data are available to help the learning of the target task. We first propose a convex formulation for multitask metric learning by modeling the task relationships in the form of a task covariance matrix. Then we regard transfer learning as a special case of multitask learning and adapt the formulation of multitask metric learning to the transfer-learning setting for our method, called transfer metric learning (TML). In TML, we learn the metric and the task covariances between the source tasks and the target task under a unified convex formulation. To solve the convex optimization problem, we use an alternating method in which each subproblem has an efficient solution. Moreover, in many applications, some unlabeled data is also available in the target task, and so we propose a semi-supervised extension of TML called STML to further improve the generalization performance by exploiting the unlabeled data based on the manifold assumption. Experimental results on some commonly used transfer-learning applications demonstrate the effectiveness of our method.

AAAI Conference 2011 Conference Paper

Multi-Task Learning in Heterogeneous Feature Spaces

• Yu Zhang
• Dit-Yan Yeung

Multi-task learning aims at improving the generalization performance of a learning task with the help of some other related tasks. Although many multi-task learning methods have been proposed, they are all based on the assumption that all tasks share the same data representation. This assumption is too restrictive for general applications. In this paper, we propose a multi-task extension of linear discriminant analysis (LDA), called multi-task discriminant analysis (MTDA), which can deal with learning tasks with different data representations. For each task, MTDA learns a separate transformation which consists of two parts, one specific to the task and one common to all tasks. A by-product of MTDA is that it can alleviate the labeled data deficiency problem of LDA. Moreover, unlike many existing multi-task learning methods, MTDA can handle binary and multi-class problems for each task in a generic way. Experimental results on face recognition show that MTDA consistently outperforms related methods.

YNIMG Journal 2011 Journal Article

Patterns of altered cortical perfusion and diminished subcortical integrity in posttraumatic stress disorder: An MRI study

• Norbert Schuff
• Yu Zhang
• Wang Zhan
• Maryann Lenoci
• Christopher Ching
• Lauren Boreta
• Susanne G. Mueller
• Zhen Wang

Posttraumatic stress disorder (PTSD) accounts for a substantial proportion of casualties among surviving soldiers of the Iraq and Afghanistan wars. Currently, the assessment of PTSD is based exclusively on symptoms, making it difficult to obtain an accurate diagnosis. This study aimed to find potential imaging markers for PTSD using structural, perfusion, and diffusion magnetic resonance imaging (MRI) together. Seventeen male veterans with PTSD (45±14years old) and 15 age-matched male veterans without PTSD had measurements of regional cerebral blood flow (rCBF) using arterial spin labeling (ASL) perfusion MRI. A slightly larger group had also measurements of white matter integrity using diffusion tensor imaging (DTI) with computations of regional fractional anisotropy (FA). The same subjects also had structural MRI of the hippocampal subfields as reported recently (W. Zhen et al. Arch Gen Psych 2010; 67(3): 296–303). On ASL-MRI, subjects with PTSD had increased rCBF in primarily right parietal and superior temporal cortices. On DTI, subjects with PTSD had FA reduction in white matter regions of the prefrontal lobe, including areas near the anterior cingulate cortex and prefrontal cortex as well as in the posterior angular gyrus. In conclusion, PTSD is associated with a systematic pattern of physiological and structural abnormalities in predominantly frontal lobe and limbic brain regions. Structural, perfusion, and diffusion MRI together may provide a signature for a PTSD marker.

ICRA Conference 2011 Conference Paper

Solution space reasoning to improve IQ-ASyMTRe in tightly-coupled multirobot tasks

• Yu Zhang
• Lynne E. Parker

In our prior work, we proposed the IQ-ASyMTRe architecture as a general method to combine coalition formation and execution for tightly-coupled multirobot tasks. IQ ASyMTRe extends the ASyMTRe architecture by introducing several new mechanisms to provide more flexibility for coalition formation as well as to facilitate coalition execution. On the other hand, these mechanisms also change the process of reasoning about solutions and further increase the complexity of the solution space. In this paper, we provide improvements for utilizing the IQ-ASyMTRe architecture based on reasoning about the solution space. We introduce a method in which the exponential growth of the number of potential solutions to be searched can be avoided; instead, the search space is only of linear size for certain tasks. Unnecessary potential solutions are removed to further increase online efficiency. Moreover, the relationships between the created solution space and the complete solution space are studied, and are utilized to provide more coverage of the complete solution space for arbitrary tasks. Although these improvements are discussed with respect to IQ-ASyMTRe, they are also applicable to architectures that approach the generality that IQ-ASyMTRe achieves. Robot simulation and experimental results are provided to demonstrate that the generation and searching of the solution space can be done online (which was impractical previously even for tasks with relatively modest complexities) for certain tasks, and to illustrate how our approach impacts the solution space.

ICRA Conference 2010 Conference Paper

A general information quality based approach for satisfying sensor constraints in multirobot tasks

• Yu Zhang
• Lynne E. Parker

Many architectures have been proposed to solve tightly-coupled multirobot tasks (MT) through coalitions of heterogeneous robots. However, several issues remain unaddressed. As coalitions are formed, sensor constraints among robots are also established. For example, in a leader-follower task, follower robots must keep leader robots within their sights, while in a box-pushing task, a supervisor robot needs to track the moving direction of the box and monitor the pushing path to the goal for obstacle avoidance. The question of how to keep these constraints satisfied during the entire execution, from initial configurations to completeness of the task, remains an open issue. In addition, environmental factors, both static and dynamic, can influence the maintenance of the constraints. Moreover, problems arise when the constraints are unsatisfiable given the current circumstances. For example, the sight of the leader might be blocked or there might be obstacles blocking the view of the box. This paper proposes a general method to address these issues for various applications with sensors having certain characteristics. Our approach combines the use of sensor models, environment sampling, measures of information quality, a motion model with sampling, and a constraint model. We believe that this approach offers the first generic formulation of robotic sensor constraints that can be applied to a wide variety of applications. To illustrate this method, we apply the approach to solve robot tracking and navigation tasks both in simulation and with physical robots. Experimental results illustrate the flexibility and robustness of the approach.

AAAI Conference 2010 Conference Paper

Adaptive Transfer Learning

• Bin Cao
• Sinno Jialin Pan
• Yu Zhang
• Dit-Yan Yeung
• Qiang Yang

Transfer learning aims at reusing the knowledge in some source tasks to improve the learning of a target task. Many transfer learning methods assume that the source tasks and the target task be related, even though many tasks are not related in reality. However, when two tasks are unrelated, the knowledge extracted from a source task may not help, and even hurt, the performance of a target task. Thus, how to avoid negative transfer and then ensure a “safe transfer” of knowledge is crucial in transfer learning. In this paper, we propose an Adaptive Transfer learning algorithm based on Gaussian Processes (AT-GP), which can be used to adapt the transfer learning schemes by automatically estimating the similarity between a source and a target task. The main contribution of our work is that we propose a new semi-parametric transfer kernel for transfer learning from a Bayesian perspective, and propose to learn the model with respect to the target task, rather than all tasks as in multi-task learning. We can formulate the transfer learning problem as a unified Gaussian Process (GP) model. The adaptive transfer ability of our approach is verified on both synthetic and real-world datasets.

IROS Conference 2010 Conference Paper

IQ-ASyMTRe: Synthesizing coalition formation and execution for tightly-coupled multirobot tasks

• Yu Zhang
• Lynne E. Parker

This paper presents the IQ-ASyMTRe architecture, which is aimed to address both coalition formation and execution for tightly-coupled multirobot tasks in a single framework. Many task allocation algorithms have been previously proposed without explicitly enabling the sharing of robot capabilities. Inspired by information invariant theory, ASyMTRe was introduced which enables the sharing of sensory and computational capabilities by allowing information to flow among different robots via communication. However, ASyMTRe does not provide a solution for how a coalition should satisfy sensor constraints introduced by the sharing of capabilities while executing the assigned task. Furthermore, conversions among different information types1 are hardcoded, which limits the flexibility of ASyMTRe. Moreover, relationships between entities (e. g. , robots) and information types are not explicitly captured, which may produce infeasible solutions from the start, as the defined information type may not correspond well to the current environment settings. The new architecture introduces a complete definition of information type to guarantee the feasibility of solutions; it also explicitly models information conversions. Inspired by our previous work, IQ-ASyMTRe uses measures of information quality to guide robot coalitions to satisfy sensor constraints (introduced by capability sharing) while executing tasks, thus providing a complete and general solution. We demonstrate the capability of the approach both in simulation and on physical robots to form and execute coalitions that share sensory information to achieve tightly-coupled tasks.

NeurIPS Conference 2010 Conference Paper

Probabilistic Multi-Task Feature Selection

• Yu Zhang
• Dit-Yan Yeung
• Qian Xu

Recently, some variants of the $l_1$ norm, particularly matrix norms such as the $l_{1, 2}$ and $l_{1, \infty}$ norms, have been widely used in multi-task learning, compressed sensing and other related areas to enforce sparsity via joint regularization. In this paper, we unify the $l_{1, 2}$ and $l_{1, \infty}$ norms by considering a family of $l_{1, q}$ norms for $1 < q\le\infty$ and study the problem of determining the most appropriate sparsity enforcing norm to use in the context of multi-task feature selection. Using the generalized normal distribution, we provide a probabilistic interpretation of the general multi-task feature selection problem using the $l_{1, q}$ norm. Based on this probabilistic interpretation, we develop a probabilistic model using the noninformative Jeffreys prior. We also extend the model to learn and exploit more general types of pairwise relationships between tasks. For both versions of the model, we devise expectation-maximization~(EM) algorithms to learn all model parameters, including $q$, automatically. Experiments have been conducted on two cancer classification applications using microarray gene expression data.

AAAI Conference 2010 Conference Paper

Transductive Learning on Adaptive Graphs

• Yan-Ming Zhang
• Yu Zhang
• Dit-Yan Yeung
• Cheng-Lin Liu
• Xinwen Hou

Graph-based semi-supervised learning methods are based on some smoothness assumption about the data. As a discrete approximation of the data manifold, the graph plays a crucial role in the success of such graphbased methods. In most existing methods, graph construction makes use of a predefined weighting function without utilizing label information even when it is available. In this work, by incorporating label information, we seek to enhance the performance of graph-based semi-supervised learning by learning the graph and label inference simultaneously. In particular, we consider a particular setting of semi-supervised learning called transductive learning. Using the LogDet divergence to define the objective function, we propose an iterative algorithm to solve the optimization problem which has closed-form solution in each step. We perform experiments on both synthetic and real data to demonstrate improvement in the graph and in terms of classification accuracy.

NeurIPS Conference 2010 Conference Paper

Worst-Case Linear Discriminant Analysis

• Yu Zhang
• Dit-Yan Yeung

Dimensionality reduction is often needed in many applications due to the high dimensionality of the data involved. In this paper, we first analyze the scatter measures used in the conventional linear discriminant analysis~(LDA) model and note that the formulation is based on the average-case view. Based on this analysis, we then propose a new dimensionality reduction method called worst-case linear discriminant analysis~(WLDA) by defining new between-class and within-class scatter measures. This new model adopts the worst-case view which arguably is more suitable for applications such as classification. When the number of training data points or the number of features is not very large, we relax the optimization problem involved and formulate it as a metric learning problem. Otherwise, we take a greedy approach by finding one direction of the transformation at a time. Moreover, we also analyze a special case of WLDA to show its relationship with conventional LDA. Experiments conducted on several benchmark datasets demonstrate the effectiveness of WLDA when compared with some related dimensionality reduction methods.

YNIMG Journal 2009 Journal Article

Characterization of white matter degeneration in elderly subjects by magnetic resonance diffusion and FLAIR imaging correlation

• Wang Zhan
• Yu Zhang
• Susanne G. Mueller
• Peter Lorenzen
• Stathis Hadjidemetriou
• Norbert Schuff
• Michael W. Weiner

Fluid attenuated inversion recovery (FLAIR) and diffusion tensor imaging (DTI) techniques have been widely used to evaluate white matter (WM) alterations associated with aging, dementia and cerebral vascular disease. The relationship between FLAIR detected WM lesions (WML) and DTI detected WM integrity changes, however, remains unclear. To investigate this association, voxelwise correlations between 4 Tesla DTI and FLAIR images from elderly subjects were performed by relating WML volume and intensity in FLAIR to fractional anisotropy (FA) and mean diffusivity (MD) in DTI. Significant DTI–FLAIR correlations were found in regions overlapping with the WML of moderate intensities in FLAIR. No significant correlations were detected in periventricular regions where the FLAIR intensities are particularly high. The findings are consistent with a transitional model for WM degeneration from normal WM to cerebrospinal fluid (CSF). The results show that the correlation between DTI and FLAIR disappears when the FLAIR intensity of WML reaches its maximum at a certain lesion severity, and that the correlations may remerge with reversed signs when the lesion severity is further increased. These results suggest that the different stages of WM degeneration in elderly subjects can be better characterized by regional DTI–FLAIR correlations than single modality alone.

TCS Journal 2008 Journal Article

Cryptographic logical relations

• Yu Zhang

Using contextual equivalence (a. k. a. observational equivalence) to specify security properties is an important idea in the field of formal verification of cryptographic protocols. While contextual equivalence is difficult to prove directly, one is usually able to deduce it using the so-called logical relations in typed λ -calculi. We apply this technique to the cryptographic metalanguage—an extension of Moggi’s computational λ -calculus, where we use Stark’s model for name creation to explore the difficult aspect of dynamic key generation. The categorical construction of logical relations for monadic types (by Goubault-Larrecq et al.) then allows us to derive logical relations over the category S e t I. Although S e t I is a perfectly adequate model of dynamic key generation, it lacks in some aspects when we study relations between programs in the metalanguage. This leads us to an interesting exploration of what should be the proper category to consider. We show that, to define logical relations in the cryptographic metalanguage, a better choice of category is S e t I → that we proposed in [Y. Zhang, D. Nowak, Logical relations for dynamic name creation, in: Proceedings of the 17th International Workshop of Computer Science Logic and the 8th Kurt Gödel Colloqium, CSL & KGL, in: Lecture Notes in Computer Science, vol. 2803, Springer-Verlag, 2003, pp. 575–588]. However, this category is still lacking in some subtler aspects and we propose a refined category S e t P I → to fix the flaws, but our final choice is S e t I × I, which is equivalent to S e t P I →. We define the contextual equivalence based on S e t I × I and show that the cryptographic logical relation derived over S e t I × I is sound and can be used to verify protocols in practice.

IJCAI Conference 2007 Conference Paper

• Shiqi Zhao
• Ting Liu
• Xincheng Yuan
• Sheng Li
• Yu Zhang

Lexical paraphrasing aims at acquiring word-level paraphrases. It is critical for many Natural Language Processing (NLP) applications, such as Question Answering (QA), Information Extraction (IE), and Machine Translation (MT). Since the meaning and usage of a word can vary in distinct contexts, different paraphrases should be acquired according to the contexts. However, most of the existing researches focus on constructing paraphrase corpora, in which little contextual constraints for paraphrase application are imposed. This paper presents a method that automatically acquires context-specific lexical paraphrases. In this method, the obtained paraphrases of a word depend on the specific sentence the word occurs in. Two stages are included, i. e. candidate paraphrase extraction and paraphrase validation, both of which are mainly based on web mining. Evaluations are conducted on a news title corpus and the presented method is compared with a paraphrasing method that exploits a Chinese thesaurus of synonyms -- Tongyi Cilin (Extended) (CilinE for short). Results show that the f-measure of our method (0. 4852) is significantly higher than that using CilinE (0. 1127). In addition, over 85% of the correct paraphrases derived by our method cannot be found in CilinE, which suggests that our method is effective in acquiring out-of-thesaurus paraphrases.

IROS Conference 2006 Conference Paper

Research on the Hierarchical Supervisory Control of Underwater Glider

• Yu Zhang
• Jiaping Tian
• Donghai Su
• Shijie Wang

An underwater glider is a buoyancy-propelled and fixed-wing vehicle with attitude controlled completely by means of internal mass redistribution. In order to independently accomplish complex missions in unstructured and unknown oceanic environment, intelligent control system is needed to provide the underwater glider with the ability of active autonomy. Based on the RW (Ramadge & Wonham) supervisory control theory of discrete event dynamic system (DEDS), a three-level hierarchical supervisory control architecture for underwater glider is presented. The DEDS formalism models of underwater glider in terms of finite state automata (FSA) are built, and the realization of the hierarchical supervisory control (HSC) system is brought forth in detail. The simulation result shows that the three-level supervisory control system can adapt to uncertain undersea environment, and makes reasonable planning

AAMAS Conference 2005 Conference Paper

Observation-Based Proactive Communication in Team Cooperation

• Yu Zhang
• Richard A. Volz
• Thomas R. Ioerger

CSL Conference 2004 Conference Paper

Complete Lax Logical Relations for Cryptographic Lambda-Calculi

• Jean Goubault-Larrecq
• Slawomir Lasota 0001
• David Nowak
• Yu Zhang

Abstract Security properties are profitably expressed using notions of contextual equivalence, and logical relations are a powerful proof technique to establish contextual equivalence in typed lambda calculi, see e. g. Sumii and Pierce’s logical relation for a cryptographic lambda-calculus. We clarify Sumii and Pierce’s approach, showing that the right tool is prelogical relations, or lax logical relations in general: relations should be lax at encryption types, notably. To explore the difficult aspect of fresh name creation, we use Moggi’s monadic lambda-calculus with constants for cryptographic primitives, and Stark’s name creation monad. We define logical relations which are lax at encryption and function types but strict (non-lax) at various other types, and show that they are sound and complete for contextual equivalence at all types.

CSL Conference 2003 Conference Paper

Logical Relations for Dynamic Name Creation

• Yu Zhang
• David Nowak

Abstract Pitts and Stark’s nu-calculus is a typed lambda-calculus which forms a basis for the study of interaction between higher-order functions and dynamically created names. A similar approach has received renewed attention recently through Sumii and Pierce’s cryptographic lambda-calculus, which deals with security protocols. Logical relations are a powerful tool to prove properties of such a calculus, notably observational equivalence. While Pitts and Stark construct a logical relation for the nu-calculus, it rests heavily on operational aspects of the calculus and is hard to be extended. We propose an alternative Kripke logical relation for the nu-calculus, which is derived naturally from the categorical model of the nu-calculus and the general notion of Kripke logical relation. This is also related to the Kripke logical relation for the name creation monad by Goubault-Larrecq et al. (CSL’2002), which the authors claimed had similarities with Pitts and Stark’s logical relation. We show that their Kripke logical relation for names is strictly weaker than Pitts and Stark’s. We also show that our Kripke logical relation, which extends the definition of Goubault-Larrecq et al. , is equivalent to Pitts and Stark’s up to first-order types; our definition rests on purely semantic constituents, and dispenses with the detours through operational semantics that Pitts and Stark use.