Wei Zhao

JBHI Journal 2026 Journal Article

A General Global and Local Pre-Training Framework for 3D Medical Image Segmentation

• Jianguo Ju
• Ziyu Guan
• Hao Lei
• Dandan Qiu
• Haoming Zhang
• Long Chen
• Fei Xie
• Wei Zhao

Accurate target segmentation from computed tomography (CT) scans is crucial for surgical robots to perform clinical surgeries successfully. However, the lack of medical image data and annotations has been the biggest obstacle to learning robust medical image segmentation models. Self-supervised learning can effectively address this problem by providing a strategy to pre-train a model with unlabeled data, and then fine-tune downstream tasks with limited labeled data. Existing self-supervised methods fail to simultaneously utilize the abundant global anatomical structure information and local feature differences in medical imaging. In this work, we propose a new strategy for the pre-training framework, which uses the three-dimensional anatomical structure of medical images and specific task and background cues to segment volumetric medical images with limited annotations. Specifically, we propose (1) learning intrinsic patterns of volumetric medical image structures through multiple sub-tasks, and (2) designing a multi-level background cube contrastive learning strategy to enhance the target feature representation by exploiting the differences between the specific target and background. We conduct extensive evaluations on two publicly available datasets. Under limited annotation settings, the proposed method yields significant improvements compared to other self-supervised learning techniques. The proposed method achieves within 6% of the baseline performance using only five labeled CT volumes for training.

YNIMG Journal 2026 Journal Article

Alpha-band temporal variability in the Fronto-parietal network facilitates active recovery from mental fatigue

• Maoqin Peng
• Wei Zhao
• Pengrui Li
• Haokai Zhang
• Liu Deng
• Yun Qin
• Tiejun Liu

Mental fatigue from prolonged cognitive tasks impairs performance, yet this decline is often non-linear. This suggests the brain engages active self-regulatory mechanisms to recover, though the neural basis of this recovery process remains unclear. We investigated this phenomenon using electroencephalography (EEG) during a continuous 120-minute boundary avoidance task. Behaviorally, we confirmed a non-linear performance curve that featured a significant recovery stage following an initial decline. The decline stage showed classic neural fatigue signatures, including increased delta and alpha power alongside reduced brain network efficiency. However, the recovery stage did not simply reverse these patterns. Instead, it involved a distinct increase in the temporal variability of alpha-band functional connectivity within the fronto-parietal network (FPN). We found that this heightened FPN variability, interacting with alpha power, directly predicted behavioral recovery. These findings expand our view of cognitive resilience. We propose that resilience is not merely passive resistance to fatigue but an active process of compensatory control driven by dynamic network flexibility. Thus, increased FPN variability likely serves as an intrinsic neural response that facilitates behavioral recovery.

AAAI Conference 2026 Conference Paper

GUIDER: Uncertainty Guided Dynamic Re-ranking for Large Language Models Based Recommender Systems

• Cai Xu
• Xujing Wang
• Ziyu Guan
• Wei Zhao
• Meng Yan

Large Language Models (LLMs) are increasingly integral to recommendation systems, offering sophisticated language understanding and generation capabilities. However, their practical application is often hindered by challenges such as data sparsity, the generation of unreliable or hallucinated recommendations, and a general lack of transparency in their decision-making processes. Existing mitigation strategies frequently introduce significant complexity or computational overhead. To address these limitations, particularly the critical gap in quantifying the confidence of LLM-generated recommendations, we propose GUIDER: Uncertainty Guided Dynamic Re-ranking for Large Language Models based Recommender Systems. This new framework innovatively leverages the logits produced by LLMs as evidence for recommended items. By employing a Dirichlet distribution, GUIDER decomposes the total predictive uncertainty into distinct Data Uncertainty (DU), reflecting inherent data ambiguity, and Model Uncertainty (MU), indicating the model's own conviction. This principled decomposition, achieved with a single inference pass, enhances transparency and trustworthiness. Based on the quantified DU and MU levels, our system dynamically adapts its recommendation strategy---adjusting output diversity---through a four-quadrant analysis that tailors responses to specific uncertainty profiles. Extensive experiments conducted in zero-shot recommendation settings validate the effectiveness of our approach. GUIDER consistently outperforms existing methods in reliability-aware scenarios, demonstrably improving recommendation quality. This framework not only advances the practical deployment of LLM-based recommenders by making them more dependable but also provides a robust foundation for future research into uncertainty-aware generative systems.

AAAI Conference 2026 Conference Paper

ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs

• Weigang Lu
• Ziyu Guan
• Wei Zhao
• Yaming Yang
• Yujie Sun
• Zheng Liang
• Yibing Zhan
• Dapeng Tao

GNN-to-MLP (G2M) methods have emerged as a promising approach to accelerate Graph Neural Networks (GNNs) by distilling their knowledge into simpler Multi-Layer Perceptrons (MLPs). These methods bridge the gap between the expressive power of GNNs and the computational efficiency of MLPs, making them well-suited for resource-constrained environments. However, existing G2M methods are limited by their inability to flexibly adjust inference cost and accuracy dynamically, a critical requirement for real-world applications where computational resources and time constraints can vary significantly. To address this, we introduce a Progressive framework designed to offer flexible and on-demand trade-offs between inference cost and accuracy for GNN-to-MLP knowledge distillation (ProGMLP). ProGMLP employs a Progressive Training Structure (PTS), where multiple MLP students are trained in sequence, each building on the previous one. Furthermore, ProGMLP incorporates Progressive Knowledge Distillation (PKD) to iteratively refine the distillation process from GNNs to MLPs, and Progressive Mixup Augmentation (PMA) to enhance generalization by progressively generating harder mixed samples. Our approach is validated through comprehensive experiments on eight real-world graph datasets, demonstrating that ProGMLP maintains high accuracy while dynamically adapting to varying runtime scenarios, making it highly effective for deployment in diverse application settings.

AAAI Conference 2026 Conference Paper

Universal EEG Epilepsy Detection via Evidential Multi-View De-Biasing

• Ziqi Wen
• Cai Xu
• Wanqing Zhao
• Jie Zhao
• Wei Zhao

Epilepsy is a widespread neurological disorder characterized by highly patient-specific EEG patterns. Existing EEG-based seizure detection methods either train individualized models for each patient or adapt models pre-trained on known patients to new ones. However, when encountering previously unseen patients, these methods typically require retraining or fine-tuning, which limits their practical utility in clinical settings. This limitation can be linked to biases caused by patient-specific variations, which obscure the underlying pathological patterns of seizures. To address this, we propose an evidential multi-view framework that reinforces the learning of core epileptic features by promoting consistency across multiple views and reducing reliance on high-uncertainty, patient-specific segments. Specifically, we introduce Bias-guided Fisher-Evidential Multi-View Learning (BF-EML) to guide the model toward discovering intrinsic seizure patterns. BF-EML employs a two-stage training architecture: In Stage 1, we use the Fisher Information Matrix to reorder EEG segments by uncertainty and deliberately train a biased feature generator on low-evidence segments. In Stage 2, we design a dual-branch network where the biased and unbiased branches are alternately trained, encouraging the unbiased branch to reduce its reliance on patient-specific biases. Finally, we introduce a shift-calibrated fusion strategy to enhance the consistency of pathogenic feature integration. Extensive experiments on public datasets and a clinical dataset demonstrate that our method achieves superior performance in both single- and multi-patient scenarios. Importantly, it generalizes well to unseen patients without the need for retraining.

AAAI Conference 2026 Conference Paper

VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model

• Yihao Wang
• Pengxiang Ding
• Lingxiao Li
• Can Cui
• Zirui Ge
• Xinyang Tong
• Wenxuan Song
• Han Zhao

Vision-Language-Action (VLA) models typically bridge the gap between perceptual and action spaces by pre-training a large-scale Vision-Language Model (VLM) on robotic data. While this approach greatly enhances performance, it also incurs significant training costs. In this paper, we investigate how to effectively bridge vision-language (VL) representations to action (A). We introduce VLA-Adapter, a novel paradigm designed to reduce the reliance of VLA models on large-scale VLMs and extensive pre-training. To this end, we first systematically analyze the effectiveness of various VL conditions and present key findings on which conditions are essential for bridging perception and action spaces. Based on these insights, we propose a lightweight Policy module with Bridge Attention, which autonomously injects the optimal condition into the action space. In this way, our method achieves high performance using only a 0.5B-parameter backbone, without any robotic data pre-training. Extensive experiments on both simulated and real-world robotic benchmarks show that VLA-Adapter not only achieves state-of-the-art level performance, but also offers the fast inference speed reported to date. Furthermore, thanks to the proposed advanced bridging paradigm, VLA-Adapter enables the training of a powerful VLA model on a single consumer-grade GPU, greatly lowering the barrier to deploying VLA model.

JBHI Journal 2025 Journal Article

A Deep Learning-Based Approach for the Diagnostic of Brucellar Spondylitis in Magnetic Resonance Images

• Dan Shao
• Jinquan Wei
• Binyang Wang
• Zhijun Wang
• Pengying Niu
• Lvlin Yang
• Guangzhao Zhang
• Pu Chen

Brucellar spondylitis (BS), a prevalent zoonotic disease caused by Brucella, poses a significant global health threat. Accurate and timely diagnosis of BS is crucial for effective treatment; however, no specialized deep learning model has been developed for detecting BS in MR images. In this study, we proposed Brucella Spondylitis MRI Diagnosis Network (BSMRINet), a fully automated diagnostic framework designed for the detection of BS from T2-weighted (T2W) MR images. The model was developed and validated using 582 cohorts collected from four hospitals between January 2018 and August 2023. The BSMRINet architecture comprised two key modules. The vertebral body lesion detection module was designed to detect BS in intact vertebral bodies by integrating a corner detection algorithm with a ResNet-based deep learning model. This module provided accurate identification and localization of potential lesions of Brucella and calculated intervertebral disc height (DH) values. The spine lesion detection module was specifically designed to detect BS in damaged vertebral bodies by utilizing a DenseNet architecture with modified squeeze-and-excitation (scSE) networks. This module further evaluated paravertebral injuries, including abscess formation, soft tissue swelling, and joint involvement. BSMRINet demonstrated strong robustness and generalization across both internal and external validation phases. Additionally, it outperformed two radiologists with 10 to 15 years of experience in diagnosing spinal MR images. The results suggested that BSMRINet can assist in the diagnostic process of BS and enhance the diagnostic capabilities of radiologists.

EAAI Journal 2025 Journal Article

A novel compositional zero-shot learning approach based on hierarchical multi-scale feature fusion

• Wenlong Du
• Xianglin Bao
• Wei Zhao
• Xiaofeng Xu
• Xingyu Lu
• Ruiheng Zhang

Compositional Zero-Shot Learning (CZSL) aims to recognize novel combinations of attributes and objects with pre-existing concepts. Most of existing CZSL methods encounter substantial challenges in managing the complex interplay between attributes and objects, particularly when compositions vary in subtle visual details or scale. Inspired by the point-wise convolution, in this work, we propose a novel Hierarchical Multi-Scale Feature Fusion approach for the compositional zero-shot learning task. The proposed CZSL approach incorporates a patch-aware feature selection mechanism to select informative patches from images, enhancing the model’s ability to capture fine-grained details. Subsequently, we design a hierarchical multi-scale feature fusion strategy that combines visual features from multiple scales, allowing the model to integrate local and global information effectively. The proposed feature fusion strategy works by enhancing the model’s ability to disentangle attributes and objects, thereby facilitating improved recognition of novel compositions. The proposed feature fusion strategy works by enhancing the model’s ability to disentangle attributes and objects for improved recognition of novel compositions. Through extensive experiments on standard CZSL benchmark datasets, the proposed approach demonstrates significant improvement over other state-of-the-art methods in both open-world and closed-world CZSL scenarios. This study not only improves accuracy and robustness in the compositional zero-shot learning task but also provides solutions for complex visual tasks in image understanding and robotics, promoting Artificial Intelligence development in visual data and semantic understanding.

EAAI Journal 2025 Journal Article

A parameter-free self-training algorithm based on the three successive confirmation rule

• Jikui Wang
• Wei Zhao
• Qingsheng Shang
• Feiping Nie

Semi-supervised learning is a popular research topic today, and self-training is a classical semi-supervised learning framework. How to select high-confidence samples in self-training is a critical step. However, the existing algorithms do not consider both global and local information of the data. In the paper, we propose a parameter-free self-training algorithm based on the three successive confirmation rule, which integrates global and local information to identify high-confidence samples. Concretely, the local information is obtained by using k nearest neighbors and global information is derived from the three successive confirmation rule. This dual selection strategy helps to improve the quality of high-confidence samples and further improve the performance of classification. We conduct experiments on 14 benchmark datasets, comparing our method with other self-training algorithms. We use accuracy and F-score as performance metrics. The experimental results demonstrate that our algorithm significantly improves classification performance, proving its effectiveness and superiority in semi-supervised learning.

AAAI Conference 2025 Conference Paper

AGMixup: Adaptive Graph Mixup for Semi-supervised Node Classification

• Weigang Lu
• Ziyu Guan
• Wei Zhao
• Yaming Yang
• Yibing Zhan
• Yiheng Lu
• Dapeng Tao

Mixup is a data augmentation technique that enhances model generalization by interpolating between data points using a mixing ratio lambda in the image domain. Recently, the concept of mixup has been adapted to the graph domain through node-centric interpolations. However, these approaches often fail to address the complexity of interconnected relationships, potentially damaging the graph's natural topology and undermining node interactions. Furthermore, current graph mixup methods employ a one-size-fits-all strategy with a randomly sampled lambda for all mixup pairs, ignoring the diverse needs of different pairs. This paper proposes an Adaptive Graph Mixup (AGMixup) framework for semi-supervised node classification. AGMixup introduces a subgraph-centric approach, which treats each subgraph similarly to how images are handled in Euclidean domains, thus facilitating a more natural integration of mixup into graph-based learning. We also propose an adaptive mechanism to tune the mixing ratio lambda for diverse mixup pairs, guided by the contextual similarity and uncertainty of the involved subgraphs. Extensive experiments across seven datasets on semi-supervised node classification benchmarks demonstrate AGMixup's superiority over state-of-the-art graph mixup methods.

NeurIPS Conference 2025 Conference Paper

Beyond Average Value Function in Precision Medicine: Maximum Probability-Driven Reinforcement Learning for Survival Analysis

• Jianqi Feng
• Chengchun Shi
• Zhenke Wu
• Xiaodong Yan
• Wei Zhao

Constructing multistage optimal decisions for alternating recurrent event data is critically important in medical and healthcare research. Current reinforcement learning (RL) algorithms have only been applied to time-to-event data, with the objective of maximizing expected survival time. However, alternating recurrent event data has a different structure, which motivates us to model the probability and frequency of event occurrences rather than a single terminal outcome. In this paper, we introduce an RL framework specifically designed for alternating recurrent event data. Our goal is to maximize the probability that the duration between consecutive events exceeds a clinically meaningful threshold. To achieve this, we identify a lower bound of this probability, which transforms the problem into maximizing a cumulative sum of log probabilities, thus enabling direct application of standard RL algorithms. We establish the theoretical properties of the resulting optimal policy and demonstrate through numerical experiments that our proposed algorithm yields a larger probability of that the time between events exceeds a critical threshold compared with existing state-of-the-art algorithms.

AAAI Conference 2025 Conference Paper

Biased Incomplete Multi-View Learning

• Haishun Chen
• Cai Xu
• Ziyu Guan
• Wei Zhao
• Jinlong Liu

Considering the ubiquitous phenomenon of missing views in multi-view data, incomplete multi-view learning is a crucial task in many applications. Existing methods usually follow an impute-then-predict strategy for handling this problem. However, they often assume that the view-missing patterns are uniformly random in multi-view data, which does not agree with real-world scenarios. In practice, view-missing patterns often vary across different classes. For example, in the medical field, patients with rare diseases would take more examinations than those with common diseases; in the financial field, high-risk customers tend to receive evaluations from more views than ordinary ones. Hence, we often observe that data-rich classes suffer limited views while data-poor classes suffer limited samples. Previous methods would typically fail due to such biased view-missing patterns. This motivates us to delve into a new biased incomplete multi-view learning problem. To this end, we develop a Reliable Incomplete Multi-view Learning (RIML) method. RIML is a simple yet effective learning-free imputation framework that goes beyond the conventional approaches by considering information from all classes, rather than just relying on individual views or within-class samples. Specifically, we utilize an inter-class association matrix that allows data-poor classes to refer the knowledge from data-rich classes. This enables the construction of more reliable view-specific distributions, from which we perform multiple samplings to recover missing views. Additionally, to obtain a reliable multi-view representation for downstream tasks, we develop an enhanced focal loss with a category-aware marginal term to learn a more distinguishable feature space. Experiments on five multi-view datasets demonstrate that RIML significantly outperforms existing methods in both accuracy and robustness.

AAAI Conference 2025 Conference Paper

Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference

• Han Zhao
• Min Zhang
• Wei Zhao
• Pengxiang Ding
• Siteng Huang
• Donglin Wang

In recent years, applying multi-modal large language models (MLLMs) in various fields has achieved remarkable success. However, as the foundation model for many downstream tasks, MLLMs comprise the well-known Transformer network, which has a less efficient quadratic computation complexity. In this study, we introduce Cobra, a multi-modal large-scale language model built upon a state-space model, which has demonstrated significant potential in efficiently handling long sequences with fast inference and linear scalability concerning sequence length. Specifically, Cobra involves replacing Transformer-based backbone models (e.g., LLaMA or Phi) with pre-trained Mamba language models. We then empirically explore effective strategies for aligning visual and textual modalities and integrating various pre-trained Mamba model variants with visual encoders. Experiments across various multi-modal benchmarks demonstrate that: (i) Cobra performs 3× ∼ 4× faster than the most computationally efficient state-of-the-art methods, e.g., LLaVA-Phi and MobileVLM v2. Additionally, its performance is significantly enhanced thanks to the implementation of linear sequential modeling. (ii) Cobra fine-tunes a small parameter (∼48% of model parameters), leading to a significant improvement in overall performance compared to LLaVA.

NeurIPS Conference 2025 Conference Paper

Defining and Discovering Hyper-meta-paths for Heterogeneous Hypergraphs

• Yaming Yang
• Ziyu Zheng
• Weigang Lu
• Zhe Wang
• Xinyan Huang
• Wei Zhao
• Ziyu Guan

Heterogeneous hypergraph is a kind of structural data that contains multiple types of nodes and multiple types of hyperedges. Each hyperedge type corresponds to a specific multi-ary relation (called hyper-relation) among subsets of nodes, which goes beyond traditional pair-wise relations in simple graphs. Existing representation learning methods for heterogeneous hypergraphs typically learn embeddings for nodes and hyperedges based on graph neural networks. Although achieving promising performance, they are still limited in capturing more complex structural features and richer semantics conveyed by the composition of various hyper-relations. To fill this research gap, in this work, we propose the concept of hyper-meta-path for heterogeneous hypergraphs, which is defined as the composition of a sequence of hyper-relations. Besides, we design an attention-based heterogeneous hypergraph neural network (HHNN) to automatically learn the importance of hyper-meta-paths. By exploiting useful ones, HHNN is able to capture more complex structural features to boost the model's performance, as well as leverage their conveyed semantics to improve the model's interpretability. Extensive experiments show that HHNN can achieve significantly better performance than state-of-the-art baselines, and the discovered hyper-meta-paths bring good interpretability for the model predictions. To facilitate the reproducibility of this work, we provide our dataset as well as anonymized source code at: https: //github. com/zhengziyu77/HHNN.

ICLR Conference 2025 Conference Paper

Democratic Training Against Universal Adversarial Perturbations

• Bing Sun
• Jun Sun 0001
• Wei Zhao

Despite their advances and success, real-world deep neural networks are known to be vulnerable to adversarial attacks. Universal adversarial perturbation, an input-agnostic attack, poses a serious threat for them to be deployed in security-sensitive systems. In this case, a single universal adversarial perturbation deceives the model on a range of clean inputs without requiring input-specific optimization, which makes it particularly threatening. In this work, we observe that universal adversarial perturbations usually lead to abnormal entropy spectrum in hidden layers, which suggests that the prediction is dominated by a small number of ``feature'' in such cases (rather than democratically by many features). Inspired by this, we propose an efficient yet effective defense method for mitigating UAPs called \emph{Democratic Training} by performing entropy-based model enhancement to suppress the effect of the universal adversarial perturbations in a given model. \emph{Democratic Training} is evaluated with 7 neural networks trained on 5 benchmark datasets and 5 types of state-of-the-art universal adversarial attack methods. The results show that it effectively reduces the attack success rate, improves model robustness and preserves the model accuracy on clean samples.

EAAI Journal 2025 Journal Article

Enhancing scene text image super-resolution via gradient-based graph attention network

• Xiangyuan Zhu
• Xuchong Liu
• Kehua Guo
• Wei Zhao

Scene text image super-resolution is crucial for enhancing text recognition in low-resolution real-world images. Existing methods usually overlook the structured and repetitive layout of text, which can serve as powerful prior knowledge for guiding reconstruction. In this work, we propose a novel framework that incorporates gradient-based graph attention to explicitly model patch-level text layout. The architecture combines a non-local group-wise attention module, a cascaded channel attention module, and a gradient-guided graph attention module to capture both global and local structural dependencies. This design enables more accurate restoration of text contours and layout consistency. Extensive experiments on the benchmark dataset demonstrate that our method achieves superior performance in both image quality and recognition accuracy, outperforming state-of-the-art methods. The code is available at: https: //github. com/cvzxy/TSANv2.

NeurIPS Conference 2025 Conference Paper

GD$^2$: Robust Graph Learning under Label Noise via Dual-View Prediction Discrepancy

• Kailai Li
• Jiong Lou
• Jiawei Sun
• Honghong Zeng
• Wen Li
• Chentao Wu
• Yuan Luo
• Wei Zhao

Graph Neural Networks (GNNs) achieve strong performance in node classification tasks but exhibit substantial performance degradation under label noise. Despite recent advances in noise-robust learning, a principled approach that exploits the node-neighbor interdependencies inherent in graph data for label noise detection remains underexplored. To address this gap, we propose GD$^2$, a noise-aware \underline{G}raph learning framework that detects label noise by leveraging \underline{D}ual-view prediction \underline{D}iscrepancies. The framework contrasts the \textit{ego-view}, constructed from node-specific features, with the \textit{structure-view}, derived through the aggregation of neighboring representations. The resulting discrepancy captures disruptions in semantic coherence between individual node representations and the structural context, enabling effective identification of mislabeled nodes. Building upon this insight, we further introduce a view-specific training strategy that enhances noise detection by amplifying prediction divergence through differentiated view-specific supervision. Extensive experiments on multiple datasets and noise settings demonstrate that \name~achieves superior performance over state-of-the-art baselines.

JBHI Journal 2025 Journal Article

LADDA: Latent Diffusion-based Domain-adaptive Feature Disentangling for Unsupervised Multi-modal Medical Image Registration

• Peng Yuan
• Jianmin Dong
• Wei Zhao
• Fei Lyu
• Cheng Xue
• Yudong Zhang
• Chunfeng Yang
• Zhan Wu

Deformable image registration (DIR) is critical for accurate clinical diagnosis and effective treatment planning. However, patient movement, significant intensity differences, and large breathing deformations hinder accurate anatomical alignment in multi-modal image registration. These factors exacerbate the entanglement of anatomical and modality-specific style information, thereby severely limiting the performance of multi-modal registration. To address this, we propose a novel LAtent Diffusion-based Domain-Adaptive feature disentangling (LADDA) framework for unsupervised multi-modal medical image registration, which explicitly addresses the representation disentanglement. First, LADDA extracts reliable anatomical priors from the Latent Diffusion Model (LDM), facilitating downstream content-style disentangled learning. A Domain-Adaptive Feature Disentangling (DAFD) module is proposed to promote anatomical structure alignment further. This module disentangles image features into content and style information, boosting the network to focus on cross-modal content information. Next, a Neighborhood-Preserving Hashing (NPH) is constructed to further perceive and integrate hierarchical content information through local neighbourhood encoding, thereby maintaining cross-modal structural consistency. Furthermore, a Unilateral-Query-Frozen Attention (UQFA) module is proposed to enhance the coupling between upstream prior and downstream content information. The feature interaction within intra-domain consistent structures improves the fine recovery of detailed textures. The proposed framework is extensively evaluated on large-scale multi-center datasets, demonstrating superior performance across diverse clinical scenarios and strong generalization on out-of-distribution (OOD) data.

IJCAI Conference 2025 Conference Paper

Leveraging Peer-Informed Label Consistency for Robust Graph Neural Networks with Noisy Labels

• Kailai Li
• Jiawei Sun
• Jiong Lou
• Zhanbo Feng
• Hefeng Zhou
• Chentao Wu
• Guangtao Xue
• Wei Zhao

Graph Neural Networks (GNNs) excel in many applications but struggle when trained with noisy labels, especially as noise can propagate through the graph structure. Despite recent progress in developing robust GNNs, few methods exploit the intrinsic properties of graph data to filter out noise. In this paper, we introduce ProCon, a novel framework that identifies mislabeled nodes by measuring label consistency among semantically similar peers, which are determined by feature similarity and graph adjacency. Mislabeled nodes typically exhibit lower consistency with these peers, a signal we measure using pseudo-labels derived from representational prototypes. A Gaussian Mixture Model is fitted to the consistency distribution to identify clean samples, which refine prototype quality in an iterative feedback loop. Experiments on multiple datasets demonstrate that ProCon significantly outperforms state-of-the-art methods, effectively mitigating label noise and enhancing GNN robustness.

AAAI Conference 2025 Conference Paper

Local Conditional Controlling for Text-to-Image Diffusion Models

• Yibo Zhao
• Liang Peng
• Yang Yang
• Zekai Luo
• Hengjia Li
• Yao Chen
• Zheng Yang
• Xiaofei He

Diffusion models have exhibited impressive prowess in the text-to-image task. Recent methods add image-level structure controls, e.g., edge and depth maps, to manipulate the generation process together with text prompts to obtain desired images. This controlling process is globally operated on the entire image, which limits the flexibility of control regions. In this paper, we explore a novel and practical task setting: local control. It focuses on controlling specific local region according to user-defined image conditions, while the remaining regions are only conditioned by the original text prompt. However, it is non-trivial to achieve it. The naive manner of directly adding local conditions may lead to the local control dominance problem, which forces the model to focus on the controlled region and neglect object generation in other regions. To mitigate this problem, we propose Regional Discriminate Loss to update the noised latents, aiming at enhanced object generation in non-control regions. Furthermore, the proposed Focused Token Response suppresses weaker attention scores which lack the strongest response to enhance object distinction and reduce duplication. Lastly, we adopt Feature Mask Constraint to reduce quality degradation in images caused by information differences across the local control region. All proposed strategies are operated at the inference stage. Extensive experiments demonstrate that our method can synthesize high-quality images aligned with the text prompt under local control conditions.

IROS Conference 2025 Conference Paper

PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decoding

• Wenxuan Song
• Jiayi Chen
• Pengxiang Ding
• Han Zhao 0008
• Wei Zhao
• Zhide Zhong
• Zongyuan Ge
• Zhijun Li

Vision-Language-Action (VLA) models demonstrate remarkable potential for generalizable robotic manipulation. The performance of VLA models can be improved by integrating with action chunking, a critical technique for effective control. However, action chunking linearly scales up action dimensions in VLA models with increased chunking sizes. This reduces the inference efficiency. Therefore, accelerating VLA integrated with action chunking is an urgent need. To tackle this problem, we propose PD-VLA, the first parallel decoding framework for VLA models integrated with action chunking. Our framework reformulates autoregressive decoding as a nonlinear system solved by parallel fixed-point iterations. This approach preserves model performance with mathematical guarantees while significantly improving decoding speed. In addition, it enables training-free acceleration without architectural changes, as well as seamless synergy with existing acceleration techniques. Extensive simulations validate that our PD-VLA maintains competitive success rates while achieving 2. 52× execution frequency on manipulators (with 7 degrees of freedom) compared with the fundamental VLA model. Furthermore, we experimentally identify the most effective settings for acceleration. Finally, real-world experiments validate its high applicability across different tasks.

ICML Conference 2025 Conference Paper

Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation

• Shuanghao Bai
• Wanqi Zhou
• Pengxiang Ding
• Wei Zhao
• Donglin Wang
• Badong Chen

Behavior Cloning (BC) is a widely adopted visual imitation learning method in robot manipulation. Current BC approaches often enhance generalization by leveraging large datasets and incorporating additional visual and textual modalities to capture more diverse information. However, these methods overlook whether the learned representations contain redundant information and lack a solid theoretical foundation to guide the learning process. To address these limitations, we adopt an information-theoretic perspective and introduce mutual information to quantify and mitigate redundancy in latent representations. Building on this, we incorporate the Information Bottleneck (IB) principle into BC, which extends the idea of reducing redundancy by providing a structured framework for compressing irrelevant information while preserving task-relevant features. This work presents the first comprehensive study on redundancy in latent representations across various methods, backbones, and experimental settings, while extending the generalizability of the IB to BC. Extensive experiments and analyses on the CortexBench and LIBERO benchmarks show consistent performance improvements with IB across various settings, underscoring the importance of reducing input data redundancy and highlighting its practical value for real-world applications.

NeurIPS Conference 2025 Conference Paper

Selftok-Zero: Reinforcement Learning for Visual Generation via Discrete and Autoregressive Visual Tokens

• Bohan Wang
• Mingze Zhou
• Zhongqi Yue
• Wang Lin
• Kaihang Pan
• Liyu Jia
• Wentao Hu
• Wei Zhao

Reinforcement learning (RL) has become an indispensable post-training step for unlocking the full potential of Large Language Models (LLMs). Its core motivation is to incentivize the model’s inference trajectory via a reward model, effectively balancing the exploration–exploitation trade-off in scenarios where collecting exhaustive input–output ground-truth pairs is infeasible. This motivation naturally extends to visual generation, where perfect alignment between an image and a textual prompt is inherently ambiguous and often unattainable. However, existing visual generative models are not yet ready for RL due to the following two fundamental drawbacks that undermine the foundations of RL: 1) For diffusion-based models, the actual generation trajectories of sampled images cannot be reliably rewarded, as diffusion inversion is notoriously difficult. 2) For autoregressive (AR) models, we show that the widely used spatial visual tokens do not satisfy the Bellman equation and thus violate the policy improvement theorem of RL. To this end, we propose to use Selftok (Self-consistency Tokenizer), which represents each image as a sequential 1D stream of discrete, autoregressive tokens. Together with language, we train a pure AR vision-language model (VLM) for visual generation. Impressively, without using any text-image training pairs, a simple policy gradient algorithm applied to Selftok tokens significantly boosts visual generation performance, surpassing existing models by a large margin. Implementation details are provided in the Appendix.

ICLR Conference 2025 Conference Paper

VLAS: Vision-Language-Action Model with Speech Instructions for Customized Robot Manipulation

• Wei Zhao
• Pengxiang Ding
• Min Zhang 0068
• Zhefei Gong
• Shuanghao Bai
• Han Zhao 0008
• Donglin Wang

Vision-language-action models (VLAs) have recently become highly prevalent in robot manipulation due to its end-to-end architecture and impressive performance. However, current VLAs are limited to processing human instructions in textual form, neglecting the more natural speech modality for human interaction. A typical approach of incorporating speech modality into VLA necessitates a separate speech recognition system to transcribe spoken instructions into text. Such a cascading pipeline raises two major concerns for robotic systems. First, the entire model grows in size and complexity, potentially resulting in redundant computations and increased memory consumption. Second, the transcription procedure would lose non-semantic information in the raw speech, such as voiceprint, which is crucial for a robot to successfully understand and complete customized tasks. To this end, we propose VLAS, the fisrt end-to-end policy model that seamlessly integrates speech modality for robot manipulation. We present a three-stage speech instruction tuning strategy leveraging multimodal datasets, including our manually curated SQA and CSI datasets. Furthermore, to facilitate personalized operations, we develop a voice retrieval-augmented generation (RAG) approach to enhance the robot's performance in tasks requiring individual-specific knowledge. Experimental results show that the proposed VLAS, following either textual or speech instructions, can achieve performance comparable to traditional VLAs on the CALVIN benchmark. In addition, we created a benchmark consisting of customization tasks, where our VLAS demonstrates absolute superiority by fully leveraging the auxiliary information in speech.

EAAI Journal 2024 Journal Article

Application of forecasting strategies and techniques to natural gas consumption: A comprehensive review and comparative study

• Ning Tian
• Bilin Shao
• Genqing Bian
• Huibin Zeng
• Xiaojun Li
• Wei Zhao

Accurate forecasting of natural gas consumption (NGC) plays an important role in energy supply, energy trading, economic effects and environmental sustainability. NGC forecasts can be used to adjust production and supply plans to improve gas efficiency and reduce carbon emissions and supply chain waste. This paper reviews the research progress on NGC in the past decade, analyzes the typical characteristics of different forecasting strategies, and highlights 163 studies in terms of the technical aspects of feature processing methods, data decomposition methods, forecasting models and optimization algorithms. It also systematically elaborates the application of statistical models, machine learning models, grey models, logistic regression and their combinations in predictive models. Bibliometric methods are also utilized to dissect research hotspots and summarize cutting-edge trends in the field. It is worth mentioning that in the terms of hybrid model structures, the application and performance of various model structures are described and evaluated. In this paper, the future development is discussed from spatiotemporal characteristics, studying reasonable data decomposition layers and fusion models, considering potential data privacy issues, and developing artificial intelligence-supporting models and interpretable frameworks. This paper is expected to provide a multi-technology reference for natural gas forecasting and help researchers to select and develop more accurate forecasting techniques and strategies.

JBHI Journal 2024 Journal Article

CDI-NSTSEG: A Clinical Diagnosis-Inspired Effective and Efficient Framework for Non-Salient Small Tumor Segmentation

• Jianguo Ju
• Dandan Qiu
• Hao Lei
• Shumin Ren
• Wei Zhao
• Pengfei Xu
• Xuesong Zhao
• Ziyu Guan

To accurately segment various clinical lesions from computed tomography(CT) images is a critical task for the diagnosis and treatment of many diseases. However, current segmentation frameworks are tailored to specific diseases, and limited frameworks can detect and segment different types of lesions. Besides, it is another challenging problem for current segmentation frameworks to segment visually inconspicuous and small-scale tumors (such as small intestinal stromal tumors and pancreatic tumors). Our proposed framework, CDI-NSTSEG, efficiently segments small non-salient tumors using multi-scale visual information and non-local target mining. CDI-NSTSEG follows the diagnostic process of clinicians, including preliminary screening, localization, refinement, and segmentation. Specifically, we first explore to extract the unique features at three different scales (1×, 0. 5×, and 1. 5×) based on the scale space theory. Our proposed scale fusion module (SFM) hierarchically fuses features to obtain a comprehensive representation, similar to preliminary screening in clinical diagnosis. The global localization module (GLM) is designed with a non-local attention mechanism. It captures the long-range semantic dependencies of channels and spatial locations from the fused features. GLM enables us to locate the tumor from a global perspective and output the initial prediction results. Finally, we design the layer focusing module (LFM) to gradually refine the initial results. LFM mainly conducts context exploration based on foreground and background features, focuses on suspicious areas layer-by-layer, and performs element-by-element addition and subtraction to eliminate errors. Our framework achieves state-of-the-art segmentation performance on small intestinal stromal tumor and pancreatic tumor datasets.

IS Journal 2024 Journal Article

Edge-Weight-Embedding Graph Convolutional Network for Person Reidentification

• Yuanhai Lv
• Gexuan Wang
• Wanqing Zhao
• Wei Zhao
• Ziyu Guan

Person reidentification (re-ID) aims to accurately identify the same person in images from a large dataset captured by nonoverlapping cameras. Recently, local-scale features of person representation have been shown to be effective in improving the performance of re-ID. However, most previous methods have overlooked the inherent and potential relationships among the joint parts of the human skeletal structure. There are inherent differences in the human skeletal structure, such as the bone length between joints, which can be considered a highly distinguishable feature for re-ID. To address this, we propose a novel graph-convolutional-network-based method that embeds the relationships between human joints and bones into a high-level representation for re-ID. In our method, the relationships between human joints and bones are represented by the biological information of the human skeletal and encoded into a learned adjacency matrix by an edge score predictor module. Our proposed method achieves competitive results on several benchmark datasets (Market-1501, DukeMTMC-ReID, and CUHK03), demonstrating its effectiveness.

AAAI Conference 2024 Conference Paper

Entropy Induced Pruning Framework for Convolutional Neural Networks

• Yiheng Lu
• Ziyu Guan
• Yaming Yang
• Wei Zhao
• Maoguo Gong
• Cai Xu

Structured pruning techniques have achieved great compression performance on convolutional neural networks for image classification tasks. However, the majority of existing methods are sensitive with respect to the model parameters, and their pruning results may be unsatisfactory when the original model is trained poorly. That is, they need the original model to be fully trained, to obtain useful weight information. This is time-consuming, and makes the effectiveness of the pruning results dependent on the degree of model optimization. To address the above issue, we propose a novel metric named Average Filter Information Entropy (AFIE). It decomposes the weight matrix of each layer into a low-rank space, and quantifies the filter importance based on the distribution of the normalized eigenvalues. Intuitively, the eigenvalues capture the covariance among filters, and therefore could be a good guide for pruning. Since the distribution of eigenvalues is robust to the updating of parameters, AFIE can yield a stable evaluation for the importance of each filter no matter whether the original model is trained fully. We implement our AFIE-based pruning method for three popular CNN models of AlexNet, VGG-16, and ResNet-50, and test them on three widely-used image datasets MNIST, CIFAR-10, and ImageNet, respectively. The experimental results are encouraging. We surprisingly observe that for our methods, even when the original model is trained with only one epoch, the AFIE score of each filter keeps identical to the results when the model is fully-trained. This fully indicates the effectiveness of the proposed pruning method.

AAAI Conference 2024 Conference Paper

NodeMixup: Tackling Under-Reaching for Graph Neural Networks

• Weigang Lu
• Ziyu Guan
• Wei Zhao
• Yaming Yang
• Long Jin

Graph Neural Networks (GNNs) have become mainstream methods for solving the semi-supervised node classification problem. However, due to the uneven location distribution of labeled nodes in the graph, labeled nodes are only accessible to a small portion of unlabeled nodes, leading to the under-reaching issue. In this study, we firstly reveal under-reaching by conducting an empirical investigation on various well-known graphs. Then, we demonstrate that under-reaching results in unsatisfactory distribution alignment between labeled and unlabeled nodes through systematic experimental analysis, significantly degrading GNNs' performance. To tackle under-reaching for GNNs, we propose an architecture-agnostic method dubbed NodeMixup. The fundamental idea is to (1) increase the reachability of labeled nodes by labeled-unlabeled pairs mixup, (2) leverage graph structures via fusing the neighbor connections of intra-class node pairs to improve performance gains of mixup, and (3) use neighbor label distribution similarity incorporating node degrees to determine sampling weights for node mixup. Extensive experiments demonstrate the efficacy of NodeMixup in assisting GNNs in handling under-reaching. The source code is available at https://github.com/WeigangLu/NodeMixup.

TMLR Journal 2024 Journal Article

Normed Spaces for Graph Embedding

• Diaaeldin Taha
• Wei Zhao
• J. Maxwell Riestenberg
• Michael Strube

Theoretical results from discrete geometry suggest that normed spaces can abstractly embed finite metric spaces with surprisingly low theoretical bounds on distortion in low dimensions. Inspired by this theoretical insight, we highlight in this paper normed spaces as a more flexible and computationally efficient alternative to several popular Riemannian manifolds for learning graph embeddings. Normed space embeddings significantly outperform several popular manifolds on a large range of synthetic and real-world graph reconstruction benchmark datasets while requiring significantly fewer computational resources. We also empirically verify the superiority of normed space embeddings on growing families of graphs associated with negative, zero, and positive curvature, further reinforcing the flexibility of normed spaces in capturing diverse graph structures as graph sizes increase. Lastly, we demonstrate the utility of normed space embeddings on two applied graph embedding tasks, namely, link prediction and recommender systems. Our work highlights the potential of normed spaces for geometric graph representation learning, raises new research questions, and offers a valuable tool for experimental mathematics in the field of finite metric space embeddings. We make our code and data publically available \footnote{\url{https://github.com/andyweizhao/graphs-normed-spaces}}.

AAAI Conference 2024 Conference Paper

Reliable Conflictive Multi-View Learning

• Cai Xu
• Jiajun Si
• Ziyu Guan
• Wei Zhao
• Yue Wu
• Xiyue Gao

Multi-view learning aims to combine multiple features to achieve more comprehensive descriptions of data. Most previous works assume that multiple views are strictly aligned. However, real-world multi-view data may contain low-quality conflictive instances, which show conflictive information in different views. Previous methods for this problem mainly focus on eliminating the conflictive data instances by removing them or replacing conflictive views. Nevertheless, real-world applications usually require making decisions for conflictive instances rather than only eliminating them. To solve this, we point out a new Reliable Conflictive Multi-view Learning (RCML) problem, which requires the model to provide decision results and attached reliabilities for conflictive multi-view data. We develop an Evidential Conflictive Multi-view Learning (ECML) method for this problem. ECML first learns view-specific evidence, which could be termed as the amount of support to each category collected from data. Then, we can construct view-specific opinions consisting of decision results and reliability. In the multi-view fusion stage, we propose a conflictive opinion aggregation strategy and theoretically prove this strategy can exactly model the relation of multi-view common and view-specific reliabilities. Experiments performed on 6 datasets verify the effectiveness of ECML. The code is released at https://github.com/jiajunsi/RCML.

JMLR Journal 2024 Journal Article

Towards Explainable Evaluation Metrics for Machine Translation

• Christoph Leiter
• Piyawat Lertvittayakumjorn
• Marina Fomicheva
• Wei Zhao
• Yang Gao
• Steffen Eger

Unlike classical lexical overlap metrics such as BLEU, most current evaluation metrics for machine translation (for example, COMET or BERTScore) are based on black-box large language models. They often achieve strong correlations with human judgments, but recent research indicates that the lower-quality classical metrics remain dominant, one of the potential reasons being that their decision processes are more transparent. To foster more widespread acceptance of novel high-quality metrics, explainability thus becomes crucial. In this concept paper, we identify key properties as well as key goals of explainable machine translation metrics and provide a comprehensive synthesis of recent techniques, relating them to our established goals and properties. In this context, we also discuss the latest state-of-the-art approaches to explainable metrics based on generative models such as ChatGPT and GPT4. Finally, we contribute a vision of next-generation approaches, including natural language explanations. We hope that our work can help catalyze and guide future research on explainable evaluation metrics and, mediately, also contribute to better and more transparent machine translation systems. [abs] [ pdf ][ bib ] &copy JMLR 2024. ( edit, beta )

IJCAI Conference 2024 Conference Paper

Trusted Multi-view Learning with Label Noise

• Cai Xu
• Yilin Zhang
• Ziyu Guan
• Wei Zhao

Multi-view learning methods often focus on improving decision accuracy while neglecting the decision uncertainty, which significantly restricts their applications in safety-critical applications. To address this issue, researchers propose trusted multi-view methods that learn the class distribution for each instance, enabling the estimation of classification probabilities and uncertainty. However, these methods heavily rely on high-quality ground-truth labels. This motivates us to delve into a new generalized trusted multi-view learning problem: how to develop a reliable multi-view learning model under the guidance of noisy labels? We propose a trusted multi-view noise refining method to solve this problem. We first construct view-opinions using evidential deep neural networks, which consist of belief mass vectors and uncertainty estimates. Subsequently, we design view-specific noise correlation matrices that transform the original opinions into noisy opinions aligned with the noisy labels. Considering label noises originating from low-quality data features and easily-confused classes, we ensure that the diagonal elements of these matrices are inversely proportional to the uncertainty, while incorporating class relations into the off-diagonal elements. Finally, we aggregate the noisy opinions and employ a generalized maximum likelihood loss on the aggregated opinion for model training, guided by the noisy labels. We empirically compare TMNR with state-of-the-art trusted multi-view learning and label noise learning baselines on 5 publicly available datasets. Experiment results show that TMNR outperforms baseline methods on accuracy, reliability and robustness. The code and appendix are released at https: //github. com/YilinZhang107/TMNR.

LORI Conference 2023 Conference Paper

Belief Revision with Satisfaction Measure

• Wei Zhao

Abstract In this paper we propose a satisfaction measure for the theory of belief revision, which should be an alternative to the classical distance measure for defining revision operators. We present the idea of satisfaction measure as a monotonic condition, and characterize how it affects the revision operation by proving an extension of the AGM representation theorem (H. Katsuno and A. Mendelzon, 1991). A unique syntactical form, the disjunction of all prime implicants, is used to compile the belief bases. Then we develop a method to generate revision operators meet that monotonic condition. Lastly, we compare such an operator to four operators of other kinds with an example, then highlight its novelty and advantages.

EAAI Journal 2023 Journal Article

Multi-spatio-temporal Fusion Graph Recurrent Network for Traffic Forecasting

• Wei Zhao
• Shiqi Zhang
• Bing Zhou
• Bei Wang

Accurate traffic forecasting is crucial for smart city development in the new era. However, the intricate spatial and temporal dependencies in traffic data present significant challenges for prediction accuracy. Existing methods often rely on predefined adjacency matrices to capture Spatio-temporal dependencies, which may not adapt well to the dynamic nature of road traffic. To address these challenges, we propose the Multi-Spatio-temporal Fusion Graph Recurrent Network (MSTFGRN). This innovative approach introduces a data-driven method for generating a weighted adjacency matrix, effectively capturing real-time spatial dependencies that are not adequately captured by predefined matrices. The MSTFGRN also incorporates a novel two-way Spatio-temporal fusion operation to learn hidden dependencies between parallel Spatio-temporal relations at different time points. Additionally, a global attention mechanism is integrated into the Spatio-temporal fusion module, enabling the simultaneous capture of global Spatio-temporal dependencies. Through extensive trials on publicly available highway traffic datasets, our method demonstrates state-of-the-art performance compared to alternative baselines.

AAAI Conference 2023 Conference Paper

Progressive Deep Multi-View Comprehensive Representation Learning

• Cai Xu
• Wei Zhao
• Jinglong Zhao
• Ziyu Guan
• Yaming Yang
• Long Chen
• Xiangyu Song

Multi-view Comprehensive Representation Learning (MCRL) aims to synthesize information from multiple views to learn comprehensive representations of data items. Prevalent deep MCRL methods typically concatenate synergistic view-specific representations or average aligned view-specific representations in the fusion stage. However, the performance of synergistic fusion methods inevitably degenerate or even fail when partial views are missing in real-world applications; the aligned based fusion methods usually cannot fully exploit the complementarity of multi-view data. To eliminate all these drawbacks, in this work we present a Progressive Deep Multi-view Fusion (PDMF) method. Considering the multi-view comprehensive representation should contain complete information and the view-specific data contain partial information, we deem that it is unstable to directly learn the mapping from partial information to complete information. Hence, PDMF employs a progressive learning strategy, which contains the pre-training and fine-tuning stages. In the pre-training stage, PDMF decodes the auxiliary comprehensive representation to the view-specific data. It also captures the consistency and complementarity by learning the relations between the dimensions of the auxiliary comprehensive representation and all views. In the fine-tuning stage, PDMF learns the mapping from the original data to the comprehensive representation with the help of the auxiliary comprehensive representation and relations. Experiments conducted on a synthetic toy dataset and 4 real-world datasets show that PDMF outperforms state-of-the-art baseline methods. The code is released at https://github.com/winterant/PDMF.

EAAI Journal 2023 Journal Article

Transferable visual pattern memory network for domain adaptation in anomaly detection

• Cangning Fan
• Ye Jin
• Peng Liu
• Wei Zhao

Anomaly detection transfer aims to utilize knowledge learned from source anomaly detection task to improve the performance of target anomaly detection task. Conventional methods typically assume that labeled normal or abnormal data are available in the source or target domain. However, many real-world applications do not satisfy this assumption because such labels are hard to collect. This study focuses on the case where anomalous labels are unavailable. More specifically, a rarely studied scenario in which the target domain contains unlabeled normal and abnormal instances, whereas only normal instances are available in the source domain, is addressed. To this end, a transferable visual pattern memory network was designed to transfer knowledge for anomaly detection tasks. The network comprises an adversarial domain adaptation method to extract transferable visual patterns, and a memory module utilized to store these patterns. The model utilizes transferable patterns stored in memory to identify anomalous samples. Moreover, a self-supervised objective is integrated to enhance the discriminability of target abnormal instances, thereby improving the anomaly detection performance. The results of extensive experiments conducted on publicly available anomaly-detection datasets verified the efficacy of the proposed approach.

IJCAI Conference 2022 Conference Paper

Charge Prediction by Constitutive Elements Matching of Crimes

• Jie Zhao
• Ziyu Guan
• Cai Xu
• Wei Zhao
• Enze Chen

Charge prediction is to automatically predict the judgemental charges for legal cases. To convict a person/unit of a charge, the case description must contain matching instances of the constitutive elements (CEs) of that charge. This knowledge of CEs is a valuable guide for the judge in making final decisions. However, it is far from fully exploited for charge prediction in the literature. In this paper we propose a novel method named Constitutive Elements-guided Charge Prediction (CECP). CECP mimics human's charge identification process to extract potential instances of CEs and generate predictions accordingly. It avoids laborious labeling of matching instances of CEs by a novel reinforcement learning module which progressively selects potentially matching sentences for CEs and evaluates their relevance. The final prediction is generated based on the selected sentences and their relevant CEs. Experiments on two real-world datasets show the superiority of CECP over competitive baselines.

YNIMG Journal 2022 Journal Article

Discovering hidden brain network responses to naturalistic stimuli via tensor component analysis of multi-subject fMRI data

• Guoqiang Hu
• Huanjie Li
• Wei Zhao
• Yuxing Hao
• Zonglei Bai
• Lisa D. Nickerson
• Fengyu Cong

The study of brain network interactions during naturalistic stimuli facilitates a deeper understanding of human brain function. To estimate large-scale brain networks evoked with naturalistic stimuli, a tensor component analysis (TCA) based framework was used to characterize shared spatio-temporal patterns across subjects in a purely data-driven manner. In this framework, a third-order tensor is constructed from the timeseries extracted from all brain regions from a given parcellation, for all participants, with modes of the tensor corresponding to spatial distribution, time series and participants. TCA then reveals spatially and temporally shared components, i.e., evoked networks with the naturalistic stimuli, their time courses of activity and subject loadings of each component. To enhance the reproducibility of the estimation with the adaptive TCA algorithm, a novel spectral clustering method, tensor spectral clustering, was proposed and applied to evaluate the stability of the TCA algorithm. We demonstrated the effectiveness of the proposed framework via simulations and real fMRI data collected during a motor task with a traditional fMRI study design. We also applied the proposed framework to fMRI data collected during passive movie watching to illustrate how reproducible brain networks are evoked by naturalistic movie viewing.

NeurIPS Conference 2022 Conference Paper

Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering

• Yaming Yang
• Ziyu Guan
• Zhe Wang
• Wei Zhao
• Cai Xu
• Weigang Lu
• Jianbin Huang

Recent self-supervised pre-training methods on Heterogeneous Information Networks (HINs) have shown promising competitiveness over traditional semi-supervised Heterogeneous Graph Neural Networks (HGNNs). Unfortunately, their performance heavily depends on careful customization of various strategies for generating high-quality positive examples and negative examples, which notably limits their flexibility and generalization ability. In this work, we present SHGP, a novel Self-supervised Heterogeneous Graph Pre-training approach, which does not need to generate any positive examples or negative examples. It consists of two modules that share the same attention-aggregation scheme. In each iteration, the Att-LPA module produces pseudo-labels through structural clustering, which serve as the self-supervision signals to guide the Att-HGNN module to learn object embeddings and attention coefficients. The two modules can effectively utilize and enhance each other, promoting the model to learn discriminative embeddings. Extensive experiments on four real-world datasets demonstrate the superior effectiveness of SHGP against state-of-the-art unsupervised baselines and even semi-supervised baselines. We release our source code at: https: //github. com/kepsail/SHGP.

NeurIPS Conference 2021 Conference Paper

AP-10K: A Benchmark for Animal Pose Estimation in the Wild

• Hang Yu
• Yufei Xu
• Jing Zhang
• Wei Zhao
• Ziyu Guan
• Dacheng Tao

Accurate animal pose estimation is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. Previous works only focus on specific animals while ignoring the diversity of animal species, limiting the generalization ability. In this paper, we propose AP-10K, the first large-scale benchmark for general animal pose estimation, to facilitate the research in animal pose estimation. AP-10K consists of 10, 015 images collected and filtered from 23 animal families and 54 species following the taxonomic rank and high-quality keypoint annotations labeled and checked manually. Based on AP-10K, we benchmark representative pose estimation models on the following three tracks: (1) supervised learning for animal pose estimation, (2) cross-domain transfer learning from human pose estimation to animal pose estimation, and (3) intra- and inter-family domain generalization for unseen animals. The experimental results provide sound empirical evidence on the superiority of learning from diverse animals species in terms of both accuracy and generalization ability. It opens new directions for facilitating future research in animal pose estimation. AP-10k is publicly available at https: //github. com/AlexTheBad/AP10K.

IS Journal 2021 Journal Article

Toward Aspect-Level Sentiment Modification Without Parallel Data

• Qingnan Jiang
• Lei Chen
• Wei Zhao
• Min Yang

This article takes the lead to study aspect-level sentiment modification (ALSM) without parallel data. Given a sentence, the task of ALSM needs to reverse the sentiment with respect to the given aspect while preserving other content. The main challenge is reversing the sentiment of the given aspect without affecting the sentiments of other aspects in the sentences. To handle this problem, we propose a joint aspect-level sentiment modification (called JASM) model. JASM is a multitask system, which jointly trains two coupled modules: aspect-specific sentiment words extraction and aspect-level sentiment transformation. Besides, we propose a novel memory mechanism to learn aspect-aware sentiment representation and a gating mechanism to dynamically select aspect-aware sentiment information or content information for generating the next words. Experiments show that the proposed model substantially outperforms the compared methods in both aspect-level sentiment transformation and content preservation. For applications, we conduct data augmentation for aspect-based sentiment analysis (ABSA) through generating plausible training data with the trained ALSM model. Experiments show that data augmentation with generated data boosts the performance of a broad range of ABSA models.

AAAI Conference 2019 Conference Paper

A Human-Like Semantic Cognition Network for Aspect-Level Sentiment Classification

• Zeyang Lei
• Yujiu Yang
• Min Yang
• Wei Zhao
• Jun Guo
• Yi Liu

In this paper, we propose a novel Human-like Semantic Cognition Network (HSCN) for aspect-level sentiment classification, motivated by the principles of human beings’ reading cognitive process (pre-reading, active reading, post-reading). We first design a word-level interactive perception module to capture the correlation between context words and the given target words, which can be regarded as pre-reading. Second, to mimic the process of active reading, we propose a targetaware semantic distillation module to produce the targetspecific context representation for aspect-level sentiment prediction. Third, we further devise a semantic deviation metric module to measure the semantic deviation between the targetspecific context representation and the given target, which evaluates the degree we understand the target-specific context semantics. The measured semantic deviation is then used to fine-tune the above active reading process in a feedback regulation way. To verify the effectiveness of our approach, we conduct extensive experiments on three widely used datasets. The experiments demonstrate that HSCN achieves impressive results compared to other strong competitors.

IJCAI Conference 2019 Conference Paper

Adversarial Incomplete Multi-view Clustering

• Cai Xu
• Ziyu Guan
• Wei Zhao
• Hongchang Wu
• Yunfei Niu
• Beilei Ling

Multi-view clustering aims to leverage information from multiple views to improve clustering. Most previous works assumed that each view has complete data. However, in real-world datasets, it is often the case that a view may contain some missing data, resulting in the incomplete multi-view clustering problem. Previous methods for this problem have at least one of the following drawbacks: (1) employing shallow models, which cannot well handle the dependence and discrepancy among different views; (2) ignoring the hidden information of the missing data; (3) dedicated to the two-view case. To eliminate all these drawbacks, in this work we present an Adversarial Incomplete Multi-view Clustering (AIMC) method. Unlike most existing methods which only learn a new representation with existing views, AIMC seeks the common latent space of multi-view data and performs missing data inference simultaneously. In particular, the element-wise reconstruction and the generative adversarial network (GAN) are integrated to infer the missing data. They aim to capture overall structure and get a deeper semantic understanding respectively. Moreover, an aligned clustering loss is designed to obtain a better clustering structure. Experiments conducted on three datasets show that AIMC performs well and outperforms baseline methods.

AAAI Conference 2019 Conference Paper

Answer Identification from Product Reviews for User Questions by Multi-Task Attentive Networks

• Long Chen
• Ziyu Guan
• Wei Zhao
• Wanqing Zhao
• Xiaopeng Wang
• Zhou Zhao
• Huan Sun

Online Shopping has become a part of our daily routine, but it still cannot offer intuitive experience as store shopping. Nowadays, most e-commerce Websites offer a Question Answering (QA) system that allows users to consult other users who have purchased the product. However, users still need to wait patiently for others’ replies. In this paper, we investigate how to provide a quick response to the asker by plausible answer identification from product reviews. By analyzing the similarity and discrepancy between explicit answers and reviews that can be answers, a novel multi-task deep learning method with carefully designed attention mechanisms is developed. The method can well exploit large amounts of user generated QA data and a few manually labeled review data to address the problem. Experiments on data collected from Amazon demonstrate its effectiveness and superiority over competitive baselines.

IJCAI Conference 2018 Conference Paper

A Multi-task Learning Approach for Image Captioning

• Wei Zhao
• Benyou Wang
• Jianbo Ye
• Min Yang
• Zhou Zhao
• Ruotian Luo
• Yu Qiao

In this paper, we propose a Multi-task Learning Approach for Image Captioning (MLAIC ), motivated by the fact that humans have no difficulty performing such task because they possess capabilities of multiple domains. Specifically, MLAIC consists of three key components: (i) A multi-object classification model that learns rich category-aware image representations using a CNN image encoder; (ii) A syntax generation model that learns better syntax-aware LSTM based decoder; (iii) An image captioning model that generates image descriptions in text, sharing its CNN encoder and LSTM decoder with the object classification task and the syntax generation task, respectively. In particular, the image captioning model can benefit from the additional object categorization and syntax knowledge. To verify the effectiveness of our approach, we conduct extensive experiments on MS-COCO dataset. The experimental results demonstrate that our model achieves impressive results compared to other strong competitors.

AAAI Conference 2018 Short Paper

A Semi-Supervised Network Embedding Model for Protein Complexes Detection

• Wei Zhao
• Jia Zhu
• Min Yang
• Danyang Xiao
• Gabriel Pui Cheong Fung
• Xiaojun Chen

Protein complex is a group of associated polypeptide chains which plays essential roles in biological process. Given a graph representing protein-protein interactions (PPI) network, it is critical but non-trivial to detect protein complexes. In this paper, we propose a semi-supervised network embedding model by adopting graph convolutional networks to effectively detect densely connected subgraphs. We conduct extensive experiment on two popular PPI networks with various data sizes and densities. The experimental results show our approach achieves state-of-the-art performance.

JBHI Journal 2018 Journal Article

Automatic Side Branch Ostium Detection and Main Vascular Segmentation in Intravascular Optical Coherence Tomography Images

• Yihui Cao
• Qinhua Jin
• Yundai Chen
• Qinye Yin
• Xianjing Qin
• Jianan Li
• Rui Zhu
• Wei Zhao

Intravascular optical coherence tomography is the state-of-the-art imaging modality in percutaneous coronary interventionplanning and evaluation, in which side branch ostium and main vascular measurements play critical roles. However, manual measurement is time consuming and labor intensive. In this paper, we propose a fully automatic method for side branch ostium detection and main vascular segmentation to make up manual deficiency. In our method, side branch ostium points are first detected and subsequently used to divide the lumen contour into side branch and main vascular regions. Based on the division, main vascular contour is then smoothly fitted for segmentation. In side branch ostium detection, our algorithm creatively converts the definition of curvature into the calculation of the signed included angles in global view, and originally applies a differential filter to highlight the feature of side branch ostium points. A total of 4618 images from 22 pullback runs were used to evaluate the performance of the presented method. The validation results of side branch detection were TPR = 82. 8%, TNR = 98. 7%, PPV = 86. 8%, NPV = 98. 7%. The average ostial distance error (ODE) was 0. 22 mm, and the DSC of main vascular segmentation was 0. 96. In conclusion, the qualitative and quantitative evaluation indicated that the presented method is effective and accurate.

YNICL Journal 2018 Journal Article

Callosal and subcortical white matter alterations in schizophrenia: A diffusion tensor imaging study at multiple levels

• Wei Zhao
• Shuixia Guo
• Ningning He
• Albert C. Yang
• Ching-Po Lin
• Shih-Jen Tsai

Diffusion tensor imaging and its distinct capability to detect micro-structural changes in vivo allows the exploration of white matter (WM) abnormalities in patients who have been diagnosed with schizophrenia; however, the results regarding the anatomical positions and degree of abnormalities are inconsistent. In order to obtain more robust and stable findings, we conducted a multi-level analysis to investigate WM disruption in a relatively large sample size (142 schizophrenia patients and 163 healthy subjects). Specifically, we evaluated the univariate fractional anisotropy (FA) in voxel level; the bivariate pairwise structural connectivity between regions using deterministic tractography as the network node defined by the Human Brainnetome Atlas; and the multivariate network topological properties, including the network hub, efficiency, small-worldness, and strength. Our data demonstrated callosal and subcortical WM alterations in patients with schizophrenia. These disruptions were evident in both voxel and connectivity levels and further supported by associations between FA values and illness duration. Based on the findings regarding topological properties, the structural network showed weaker global integration in patients with schizophrenia than in healthy subjects, while brain network hubs showed decreased functionality. We replicated these findings using an automated anatomical labeling atlas to define the network node. Our study indicates that callosal and subcortical WM disruptions are biomarkers for chronic schizophrenia.

IJCAI Conference 2018 Conference Paper

Deep Multi-View Concept Learning

• Cai Xu
• Ziyu Guan
• Wei Zhao
• Yunfei Niu
• Quan Wang
• Zhiheng Wang

Multi-view data is common in real-world datasets, where different views describe distinct perspectives. To better summarize the consistent and complementary information in multi-view data, researchers have proposed various multi-view representation learning algorithms, typically based on factorization models. However, most previous methods were focused on shallow factorization models which cannot capture the complex hierarchical information. Although a deep multi-view factorization model has been proposed recently, it fails to explicitly discern consistent and complementary information in multi-view data and does not consider conceptual labels. In this work we present a semi-supervised deep multi-view factorization method, named Deep Multi-view Concept Learning (DMCL). DMCL performs nonnegative factorization of the data hierarchically, and tries to capture semantic structures and explicitly model consistent and complementary information in multi-view data at the highest abstraction level. We develop a block coordinate descent algorithm for DMCL. Experiments conducted on image and document datasets show that DMCL performs well and outperforms baseline methods.

IJCAI Conference 2018 Conference Paper

PLASTIC: Prioritize Long and Short-term Information in Top-n Recommendation using Adversarial Training

• Wei Zhao
• Benyou Wang
• Jianbo Ye
• Yongqiang Gao
• Min Yang
• Xiaojun Chen

Recommender systems provide users with ranked lists of items based on individual's preferences and constraints. Two types of models are commonly used to generate ranking results: long-term models and session-based models. While long-term models represent the interactions between users and items that are supposed to change slowly across time, session-based models encode the information of users' interests and changing dynamics of items' attributes in short terms. In this paper, we propose a PLASTIC model, Prioritizing Long And Short-Term Information in top-n reCommendation using adversarial training. In the adversarial process, we train a generator as an agent of reinforcement learning which recommends the next item to a user sequentially. We also train a discriminator which attempts to distinguish the generated list of items from the real list recorded. Extensive experiments show that our model exhibits significantly better performances on two widely used real-world datasets.

AAAI Conference 2018 Conference Paper

Substructure Assembling Network for Graph Classification

• Xiaohan Zhao
• Bo Zong
• Ziyu Guan
• Kai Zhang
• Wei Zhao

Graphs are natural data structures adopted to represent realworld data of complex relationships. In recent years, a surge of interest has been received to build predictive models over graphs, with prominent examples in chemistry, computational biology, and social networks. The overwhelming complexity of graph space often makes it challenging to extract interpretable and discriminative structural features for classification tasks. In this work, we propose a novel neural network structure called Substructure Assembling Network (SAN) to extract graph features and improve the generalization performance of graph classification. The key innovation of our work is a unified substructure assembling unit, which is a variant of Recurrent Neural Network (RNN) designed to hierarchically assemble useful pieces of graph components so as to fabricate discriminative substructures. SAN adopts a sequential, probabilistic decision process, and therefore it can tune substructure features in a finer granularity. Meanwhile, the parameterized soft decisions can be continuously improved with supervised learning through back-propagation, leading to optimizable search trajectories. Overall, SAN embraces both the flexibility of combinatorial pattern search and the strong optimizability of deep learning, and delivers promising results as well as interpretable structural features in graph classification against state-of-the-art techniques.

IJCAI Conference 2018 Conference Paper

Tag-based Weakly-supervised Hashing for Image Retrieval

• Ziyu Guan
• Fei Xie
• Wanqing Zhao
• Xiaopeng Wang
• Long Chen
• Wei Zhao
• Jinye Peng

We are concerned with using user-tagged images to learn proper hashing functions for image retrieval. The benefits are two-fold: (1) we could obtain abundant training data for deep hashing models; (2) tagging data possesses richer semantic information which could help better characterize similarity relationships between images. However, tagging data suffers from noises, vagueness and incompleteness. Different from previous unsupervised or supervised hashing learning, we propose a novel weakly-supervised deep hashing framework which consists of two stages: weakly-supervised pre-training and supervised fine-tuning. The second stage is as usual. In the first stage, rather than performing supervision on tags, the framework introduces a semantic embedding vector (sem-vector) for each image and performs learning of hashing and sem-vectors jointly. By carefully designing the optimization problem, it can well leverage tagging information and image content for hashing learning. The framework is general and does not depend on specific deep hashing methods. Empirical results on real world datasets show that when it is integrated with state-of-art deep hashing methods, the performance increases by 8-10%.

YNICL Journal 2016 Journal Article

The volumetric and shape changes of the putamen and thalamus in first episode, untreated major depressive disorder

• Yi Lu
• Hongmin Liang
• Dan Han
• Yin Mo
• Zongfang Li
• Yuqi Cheng
• Xiufeng Xu
• Zonglin Shen

Previous MRI studies confirmed abnormalities in the limbic-cortical-striatal-pallidal-thalamic (LCSPT) network or limbic-cortico-striatal-thalamic-cortical (LCSTC) circuits in patients with major depressive disorder (MDD), but few studies have investigated the subcortical structural abnormalities. Therefore, we sought to determine whether focal subcortical grey matter (GM) changes might be present in MDD at an early stage. We recruited 30 first episode, untreated patients with major depressive disorder (MDD) and 26 healthy control subjects. Voxel-based morphometry was used to evaluate cortical grey matter changes, and automated volumetric and shape analyses were used to assess volume and shape changes of the subcortical GM structures, respectively. In addition, probabilistic tractography methods were used to demonstrate the relationship between the subcortical and the cortical GM. Compared to healthy controls, MDD patients had significant volume reductions in the bilateral putamen and left thalamus (FWE-corrected, p < 0.05). Meanwhile, the vertex-based shape analysis showed regionally contracted areas on the dorsolateral and ventromedial aspects of the bilateral putamen, and on the dorsal and ventral aspects of left thalamus in MDD patients (FWE-corrected, p < 0.05). Additionally, a negative correlation was found between local atrophy in the dorsal aspects of the left thalamus and clinical variables representing severity. Furthermore, probabilistic tractography demonstrated that the area of shape deformation of the bilateral putamen and left thalamus have connections with the frontal and temporal lobes, which were found to be related to major depression. Our results suggested that structural abnormalities in the putamen and thalamus might be present in the early stages of MDD, which support the role of subcortical structure in the pathophysiology of MDD. Meanwhile, the present study showed that these subcortical structural abnormalities might be the potential trait markers of MDD.

IJCAI Conference 2016 Conference Paper

Weakly-Supervised Deep Learning for Customer Review Sentiment Classification

• Ziyu Guan
• Long Chen
• Wei Zhao
• Yi Zheng
• Shulong Tan
• Deng Cai

Sentiment analysis is one of the key challenges for mining online user generated content. In this work, we focus on customer reviews which are an important form of opinionated content. The goal is to identify each sentence's semantic orientation (e. g. positive or negative) of a review. Traditional sentiment classification methods often involve substantial human efforts, e. g. lexicon construction, feature engineering. In recent years, deep learning has emerged as an effective means for solving sentiment classification problems. A neural network intrinsically learns a useful representation automatically without human efforts. However, the success of deep learning highly relies on the availability of large-scale training data. In this paper, we propose a novel deep learning framework for review sentiment classification which employs prevalently available ratings as weak supervision signals. The framework consists of two steps: (1) learn a high level representation (embedding space) which captures the general sentiment distribution of sentences through rating information; (2) add a classification layer on top of the embedding layer and use labeled sentences for supervised fine-tuning. Experiments on review data obtained from Amazon show the efficacy of our method and its superiority over baseline methods.

JBHI Journal 2015 Journal Article

Sparsity-Inspired Nonparametric Probability Characterization for Radio Propagation in Body Area Networks

• Xiaodong Yang
• Shuyuan Yang
• Qammer Hussain Abbasi
• Zhiya Zhang
• Aifeng Ren
• Wei Zhao
• Akram Alomainy

Parametric probability models are common references for channel characterization. However, the limited number of samples and uncertainty of the propagation scenario affect the characterization accuracy of parametric models for body area networks. In this paper, we propose a sparse nonparametric probability model for body area wireless channel characterization. The path loss and root-mean-square delay, which are significant wireless channel parameters, can be learned from this nonparametric model. A comparison with available parametric models shows that the proposed model is very feasible for the body area propagation environment and can be seen as a significant supplement to parametric approaches.

JBHI Journal 2015 Journal Article

The Effects of Cell Asynchrony on Gene Expression Levels: Analysis and Application to <italic>Plasmodium Falciparum</italic>

• Wei Zhao
• Justin Dauwels
• Jianshu Cao

To investigate the intraerythrocytic developmental cycle of Plasmodium falciparum, time-series gene expression data is commonly measured of infected red blood cells. However, the observed data are usually blurred due to cell asynchrony during experiments. In this paper, the effect of cell asynchrony is investigated by conducting numerical experiments. The simulation results suggest that cell asynchrony has varying effects on different intrinsic expression patterns. Specifically, the intrinsic patterns with high expression around the late life stage are more likely to be affected by cell asynchrony. It is also investigated how the effect of cell asynchrony depends on the experimental conditions. From this analysis, the burst rate r% in infection period and the standard deviation σ of growth rate are identified to have a strong impact on the blurring due to cell asynchrony. Consequently, it is important to measure these two parameters during biological experiments in order to deblur time-series gene expression data.

AIIM Journal 2012 Journal Article

Translation and localization of SNOMED CT in China: A pilot study

• Yanhui Zhu
• Huiting Pan
• Lei Zhou
• Wei Zhao
• Ana Chen
• Ulrich Andersen
• Shuxiang Pan
• Lixin Tian