Arrow Research search

Author name cluster

Ke Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

30 papers
2 author rows

Possible papers

30

JBHI Journal 2026 Journal Article

FIGNet: A Robust and Interpretable Fuzzy-Irreversible Gated Network for Auditory Brainstem Response Classification

  • Ke Zhang
  • Chunrui Zhao
  • Zenan Li
  • Caiwei Li
  • Desheng Jia
  • Yongchao Chen
  • Shang Yan
  • Xin Wang

Auditory brainstem response (ABR) is an important tool for newborn hearing screening and neurological assessment. However, its signals are often difficult to be accurately resolved due to noise interference and weak waveforms, and the need for repeated measurements under multiple sound intensity conditions results in time-consuming data acquisition. Therefore, there is an urgent need to develop an automatic classification model with high accuracy, robustness and good interpretability to achieve stable and effective recognition performance with minimal ABR data. This study presents FIGNet, a new deep learning model that combines type-2 fuzzy logic with a time-irreversible attention mechanism to address uncertainty and temporal direction in ABR signals. Fuzzy attention helps reduce the impact of noise, while the irreversible attention models the one-way nature of neural responses. Experiments on real ABR datasets show that FIGNet outperforms existing models in both binary and five-class classification tasks. It achieves 93. 72% accuracy in binary classification and 84. 42% accuracy in five-class classification. Visualization results—including confusion matrices, and accuracy curves under different noise levels—further confirm that FIGNet can focus on key waveform areas and stay reliable even in noisy conditions. These findings demonstrate that FIGNet offers fast, interpretable, and robust performance for clinical ABR analysis, achieving high classification accuracy under both clean and noisy conditions.

Details DOI

AAAI Conference 2026 Conference Paper

ProCrop: Learning Aesthetic Image Cropping from Professional Compositions

  • Ke Zhang
  • Tianyu Ding
  • Jiachen Jiang
  • Tianyi Chen
  • Ilya Zharkov
  • Vishal M. Patel
  • Luming Liang

Image cropping is crucial for enhancing the visual appeal and narrative impact of photographs, yet existing rule-based and data-driven approaches often lack diversity or require annotated training data. We introduce ProCrop, a retrieval-based method that leverages professional photography to guide cropping decisions. By fusing features from professional photographs with those of the query image, ProCrop learns from professional compositions, significantly boosting performance. Additionally, we present a large-scale dataset of 242K weakly-annotated images, generated by out-painting professional images and iteratively refining diverse crop proposals. This composition-aware dataset generation offers diverse high-quality crop proposals guided by aesthetic principles and becomes the largest publicly available dataset for image cropping. Extensive experiments show that ProCrop significantly outperforms existing methods in both supervised and weakly-supervised settings. Notably, when trained on the new dataset, our ProCrop surpasses previous weakly-supervised methods and even matches fully supervised approaches.

PDF Details DOI

JBHI Journal 2026 Journal Article

WGB-GLFI: A Novel Graph-Based Global-Local Feature Interaction Framework for Automated Seizure Detection

  • Xiang Li
  • Mingxing Zhu
  • Chuqi Yang
  • Ke Zhang
  • Xin Wang
  • Sunday Timothy Aboyeji
  • Fei Chen
  • Chen Yao

Epilepsy detection faces significant challenges due to unpredictable seizures, ranging from brief awareness lapses to severe convulsions, posing risks to patients' safety and quality of life. In recent years, deep learning has become a mainstream approach in this field, leveraging advanced computational resources and EEG datasets. However, a key challenge remains: existing methods often lack unified spatial modeling and struggle to effectively handle local detailed features, thereby limiting their accuracy and robustness. To address these issues, we propose the Weighted Graph Building Global-Local Feature Interaction (WGB-GLFI) framework, which integrates spatial connectivity and dynamic patterns through a Weighted Graph Building (WGB) module and a Global-Local Feature Interaction (GLFI) module. This approach excels by comprehensively capturing the dynamic spatial relationships during epileptic seizures and achieving seamless global-local feature integration, significantly enhancing seizure detection performance. Its effectiveness has been validated across multiple datasets, including CHB-MIT, Siena Scalp, and private datasets, demonstrating robust and reliable results. Evaluated on these datasets, our model achieves accuracy rates of 99. 28%, 99. 21%, and 99. 30%, respectively. The reliability and robustness of our framework provide epilepsy patients with faster and more reliable seizure detection, which helps to intervene in a timely manner and improve the quality of life of patients.

Details DOI

EAAI Journal 2025 Journal Article

A ground-based cloud image classification method for photovoltaic power prediction based on Convolutional Neural Networks and Vision Transformer

  • Chaojun Shi
  • Mengyu Zhang
  • Hongyin Xiang
  • Ke Zhang
  • Sihao Ju
  • Xiaoyun Zhang
  • Leile Han

Cloud image classification plays a crucial role in accurately predicting solar radiation attenuation, which significantly impacts ultra-short-term photovoltaic power predictions. Despite recent advancements in cloud image classification using Transformer and convolutional neural networks, challenges remain, particularly in handling rapidly evolving cumuliform clouds. To address this, we propose CloudMViT, an improved model derived from a model that combines Convolutional Neural Networks and Vision Transformer (MobileViT). CloudMViT introduces the CloudMobileNetV2 (CMV2) Block, building upon the Mobile Network architecture (MobileNetV2). This block incorporates a triple-branch inverted residual structure consisting of depthwise separable convolution, standard convolution, and shortcut branches, along with a New Multi-scale Channel Attention Module (NMS-CAM). Simultaneously, CloudMViT proposes the CloudMViT Block, which integrates adaptive convolution to enhance local feature representation. CloudMViT improves local and global feature extraction, boosting cloud image classification accuracy. The proposed model was evaluated through ablation studies and comparative experiments on the Tianjin-Normal-University-Ground-based-Cloud-Dataset (GCD) and Tianjin-Normal-University-Ground-based-Remote-Sensing-Cloud-Database (GRSCD). The ablation experiments demonstrate that the modules introduced in CloudMViT significantly improve cloud image classification accuracy. Comparative results show that CloudMViT achieves higher accuracy than other state-of-the-art methods, reaching 91. 40% on GCD and 98. 25% on GRSCD datasets. Finally, CloudMViT was validated through experiments in conjunction with a photovoltaic power prediction model, achieving a prediction accuracy of 96. 70%, which surpasses that of the original model. This further demonstrates that utilizing CloudMViT cloud image classification results enhances the accuracy of photovoltaic power prediction. Moreover, it verifies that improving cloud image classification accuracy can effectively enhance the precision of photovoltaic power forecasting.

Details DOI

JBHI Journal 2025 Journal Article

Adapter-Enhanced Hierarchical Cross-Modal Pre-Training for Lightweight Medical Report Generation

  • Ting Yu
  • Wangwen Lu
  • Yan Yang
  • Weidong Han
  • Qingming Huang
  • Jun Yu
  • Ke Zhang

Automatic medical report generation is an emerging field that aims to transform medical images into descriptive, clinically relevant narratives, potentially reducing the workload for radiologists significantly. Despite substantial progress, the increasing model parameter size and corresponding marginal performance gains have limited further development and application. To address this challenge, we introduce an Adapter-enhanced Hierarchical cross-modal Pre-training (AHP) strategy for lightweight medical report generation. This approach significantly reduces the pre-trained model's parameter size while maintaining superior report generation performance through our proposed spatial adapters. To further address the issue of inadequate representation of visual space details, we employ a convolutional stem combined with hierarchical injectors and extractors, fully integrating with traditional Vision Transformers to achieve more comprehensive visual representations. Additionally, our cross-modal pre-training model effectively handles the inherent complex visual-textual relationships in medical imaging. Extensive experiments on multiple datasets, including IU X-Ray, MIMIC-CXR, and bladder pathology, demonstrate our model's exceptional generalization and transfer performance in downstream medical report generation tasks, highlighting AHP's potential in significantly reducing model parameters while enhancing report generation accuracy and efficiency.

Details DOI

EAAI Journal 2025 Journal Article

Anti-vibration hammer defect detection based on structural knowledge representation

  • Zhenbing Zhao
  • Guangxue Guo
  • Yitian Pan
  • Ke Zhang
  • Yongjie Zhai
  • Wenqing Zhao

Anti-vibration hammer defects pose a significant risk to the safe operation of power transmission lines. Addressing the issues posed by the various manifestations of identical category defects and the similarities among different defects, this paper introduces a defect detection algorithm for anti-vibration hammers based on structural knowledge representation. Initially, a Structural Knowledge Enhancement (StKE) Module is proposed to conduct a statistical analysis of the aspect ratios of different defects in anti-vibration hammers, effectively extracting the structural features of the anti-vibration hammers and their corresponding defects. Subsequently, a Structural Knowledge Representation (StKR) Module is introduced, which bolsters the model’s ability to precisely locate defects that disrupt structural symmetry. The model incorporates the Coordinate Attention (CA) mechanism to acquire contextual information, thereby enhancing detection accuracy. Experimental results demonstrate that the improved model achieves a 6. 1% increase in mean detection precision over the baseline model, and a notable improvement in detection accuracy compared to other advanced algorithms.

Details DOI

ICLR Conference 2025 Conference Paper

Drama: Mamba-Enabled Model-Based Reinforcement Learning Is Sample and Parameter Efficient

  • Wenlong Wang
  • Ivana Dusparic
  • Yucheng Shi
  • Ke Zhang
  • Vinny Cahill

Model-based reinforcement learning (RL) offers a solution to the data inefficiency that plagues most model-free RL algorithms. However, learning a robust world model often requires complex and deep architectures, which are computationally expensive and challenging to train. Within the world model, sequence models play a critical role in accurate predictions, and various architectures have been explored, each with its own challenges. Currently, recurrent neural network (RNN)-based world models struggle with vanishing gradients and capturing long-term dependencies. Transformers, on the other hand, suffer from the quadratic memory and computational complexity of self-attention mechanisms, scaling as $O(n^2)$, where $n$ is the sequence length. To address these challenges, we propose a state space model (SSM)-based world model, Drama, specifically leveraging Mamba, that achieves $O(n)$ memory and computational complexity while effectively capturing long-term dependencies and enabling efficient training with longer sequences. We also introduce a novel sampling method to mitigate the suboptimality caused by an incorrect world model in the early training stages. Combining these techniques, Drama achieves a normalised score on the Atari100k benchmark that is competitive with other state-of-the-art (SOTA) model-based RL algorithms, using only a 7 million-parameter world model. Drama is accessible and trainable on off-the-shelf hardware, such as a standard laptop. Our code is available at https://github.com/realwenlongwang/Drama.git.

Details

AAAI Conference 2025 Conference Paper

Fine-grained Adaptive Visual Prompt for Generative Medical Visual Question Answering

  • Ting Yu
  • Zixuan Tong
  • Jun Yu
  • Ke Zhang

Medical Visual Question Answering (MedVQA) serves as an automated medical assistant, capable of answering patient queries and aiding physician diagnoses based on medical images and questions. Recent advancements have shown that incorporating Large Language Models (LLMs) into MedVQA tasks significantly enhances the capability for answer generation. However, for tasks requiring fine-grained organ-level precise localization, relying solely on language prompts struggles to accurately locate relevant regions within medical images due to substantial background noise. To address this challenge, we explore the use of visual prompts in MedVQA tasks for the first time and propose fine-grained adaptive visual prompts to enhance generative MedVQA. Specifically, we introduce an Adaptive Visual Prompt Creator that adaptively generates region-level visual prompts based on image characteristics of various organs, providing fine-grained references for LLMs during answer retrieval and generation from the medical domain, thereby improving the model's precise cross-modal localization capabilities on original images. Furthermore, we incorporate a Hierarchical Answer Generator with Parameter-Efficient Fine-Tuning (PEFT) techniques, significantly enhancing the model's understanding of spatial and contextual information with minimal parameter increase, promoting the alignment of representation learning with the medical space. Extensive experiments on VQA-RAD, SLAKE, and DME datasets validate the effectiveness of our proposed method, demonstrating its potential in generative MedVQA.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Rethinking Pseudo-Label Guided Learning for Weakly Supervised Temporal Action Localization from the Perspective of Noise Correction

  • Quan Zhang
  • Yuxin Qi
  • Xi Tang
  • Rui Yuan
  • Xi Lin
  • Ke Zhang
  • Chun Yuan

Pseudo-label learning methods have been widely applied in weakly-supervised temporal action localization. Existing works directly utilize weakly-supervised base model to generate instance-level pseudo-labels for training the fully-supervised detection head. We argue that the noise in pseudo-labels would interfere with the learning of fully-supervised detection head, leading to significant performance leakage. Issues with noisy labels include:(1) inaccurate boundary localization; (2) undetected short action clips; (3) multiple adjacent segments incorrectly detected as one segment. To target these issues, we introduce a two-stage noisy label learning strategy to harness every potential useful signal in noisy labels. First, we propose a frame-level pseudo-label generation model with a context-aware denoising algorithm to refine the boundaries. Second, we introduce an online-revised teacher-student framework with a missing instance compensation module and an ambiguous instance correction module to solve the short-action-missing and many-to-one problems. Besides, we apply a high-quality pseudo-label mining loss in our online-revised teacher-student framework to add different weights to the noisy labels to train more effectively. Our model outperforms the previous state-of-the-art method in detection accuracy and inference speed greatly upon the THUMOS14 and ActivityNet v1.2 benchmarks.

PDF Details DOI

ICML Conference 2025 Conference Paper

Super Deep Contrastive Information Bottleneck for Multi-modal Clustering

  • Zhengzheng Lou
  • Ke Zhang
  • Yucong Wu
  • Shizhe Hu

In an era of increasingly diverse information sources, multi-modal clustering (MMC) has become a key technology for processing multi-modal data. It can apply and integrate the feature information and potential relationships of different modalities. Although there is a wealth of research on MMC, due to the complexity of datasets, a major challenge remains in how to deeply explore the complex latent information and interdependencies between modalities. To address this issue, this paper proposes a method called super deep contrastive information bottleneck (SDCIB) for MMC, which aims to explore and utilize all types of latent information to the fullest extent. Specifically, the proposed SDCIB explicitly introduces the rich information contained in the encoder’s hidden layers into the loss function for the first time, thoroughly mining both modal features and the hidden relationships between modalities. Moreover, the proposed SDCIB performs dual optimization by simultaneously considering consistency information from both the feature distribution and clustering assignment perspectives, the proposed SDCIB significantly improves clustering accuracy and robustness. We conducted experiments on 4 multi-modal datasets and the accuracy of the method on the ESP dataset improved by 9. 3%. The results demonstrate the superiority and clever design of the proposed SDCIB. The source code is available on https: //github. com/ShizheHu.

Details

NeurIPS Conference 2025 Conference Paper

TANDEM: Bi-Level Data Mixture Optimization with Twin Networks

  • Jiaxing Wang
  • Deping Xiang
  • Jin Xu
  • Mingyang Yi
  • Guoqiang Gong
  • Zicheng Zhang
  • Haoran Li
  • Pengzhang Liu

The capabilities of large language models (LLMs) significantly depend on training data drawn from various domains. Optimizing domain-specific mixture ratios can be modeled as a bi-level optimization problem, which we simplify into a single-level penalized form and solve with twin networks: a proxy model trained on primary data and a dynamically updated reference model trained with additional data. Our proposed method, Twin Networks for bi-level DatA mixturE optiMization (TANDEM), measures the data efficacy through the difference between the twin models and up-weights domains that benefit more from the additional data. TANDEM provides theoretical guarantees and wider applicability, compared to prior approaches. Furthermore, our bi-level perspective suggests new settings to study domain reweighting such as data-restricted scenarios and supervised fine-tuning, where optimized mixture ratios significantly improve the performance. Extensive experiments validate TANDEM's effectiveness in all scenarios.

PDF Details

EAAI Journal 2025 Journal Article

TSD-DETR: A lightweight real-time detection transformer of traffic sign detection for long-range perception of autonomous driving

  • Lili Zhang
  • Kang Yang
  • Yucheng Han
  • Jing Li
  • Wei Wei
  • Hongxin Tan
  • Pei Yu
  • Ke Zhang

The key to accurate perception and efficient decision making of autonomous driving is the long-range detection of traffic signs. Long-range detection of traffic signs has the problems of small traffic sign size and complex background. In order to solve these problems, this paper proposes a lightweight model for traffic sign detection based on real-time detection transformer (TSD-DETR). Firstly, the feature extraction module is constructed using multiple types of convolutional modules. The model extracts multi-scale features of different levels to enhance feature extraction ability. Then, small object detection module and detection head are designed to extract and detect shallow features. It can improve the detection of small traffic signs. Finally, Efficient Multi-Scale Attention is introduced to adjust the channel weights. It aggregates the output features of three parallel branches interactively. TSD-DETR achieves a mean average precision (mAp) of 96. 8% on Tsinghua-Tencent 100K dataset. It is improved by 2. 5% compared with real-time detection transformer. In small object detection, mAp improved by 9%. TSD-DETR achieves 99. 4% mAp on the Changsha University of Science and Technology Chinese Traffic Sign Detection Benchmark dataset, with an improvement of 0. 6%. The experimental results show that TSD-DETR reduces the number of parameters by 9. 06M by optimizing the model structure. On the premise of ensuring the real-time performance of the model, the detection accuracy of the model is improved greatly. The results of ablation experiments show that the feature extraction module and small object detection module proposed in this paper are conducive to improving the detection accuracy.

Details DOI

ICRA Conference 2025 Conference Paper

Ultrasound-Guided Robotic Blood Drawing and In Vivo Studies on Submillimetre Vessels of Rats

  • Shuaiqi Jing
  • Tianliang Yao
  • Ke Zhang
  • Di Wu 0053
  • Qiulin Wang
  • Zixi Chen
  • Ke Chen
  • Peng Qi 0001

Billions of vascular access procedures are performed annually worldwide, serving as a crucial first step in various clinical diagnostic and therapeutic procedures. For pediatric or elderly individuals, whose vessels are small in size (typically 2 to 3 mm in diameter for adults and <1 mm in children), vascular access can be highly challenging. This study presents an image-guided robotic system aimed at enhancing the accuracy of difficult vascular access procedures. The system integrates a 6-DoF (Degrees of Freedom) robotic arm with a 3-DoF end-effector, ensuring precise navigation and needle insertion. Multi-modal imaging and sensing technologies have been utilized to endow the medical robot with precision and safety, while ultrasound (US) imaging guidance is specifically evaluated in this study. To evaluate in vivo vascular access in submillimeter vessels, we conducted ultrasound-guided robotic blood drawing on the tail veins (with a diameter of 0. 7 ± 0. 2 mm) of 40 rats. The results demonstrate that the system achieved a first-attempt success rate of 95%. The high first-attempt success rate in intravenous vascular access, even with small blood vessels, demonstrates the system's effectiveness in performing these procedures. This capability reduces the risk of failed attempts, minimizes patient discomfort, and enhances clinical efficiency.

Details

EAAI Journal 2024 Journal Article

A hybrid complex spectral conjugate gradient learning algorithm for complex-valued data processing

  • Ke Zhang
  • Huisheng Zhang
  • Xue Wang

Complex-valued neural networks (CVNNs) have become a powerful modelling tool for complex-valued data processing. Because most of the critical points of CVNNs are saddle points, the gradient-based learning algorithms for CVNNs enjoy more chances to reach the global minima while suffering from slow convergence. To this end, we propose a hybrid complex spectral conjugate gradient learning algorithm for fast training CVNNs in this paper. The proposed algorithm combines the scaled negative gradient with a Barzilai–Borwein stepsize and an optimized conjugate term to define a new training direction, thus providing an accurate approximation of the second-order curvature of the objective function. The complex Wolfe conditions are employed to adaptively determine the optimal training stepsize. Under mild conditions, the descent property of the training direction and the convergence of the proposed algorithm are theoretically established. Simulation results on a number of benchmark complex-valued data processing problems demonstrate the efficiency of the proposed algorithm.

Details DOI

JBHI Journal 2024 Journal Article

Method for Incomplete and Imbalanced Data Based on Multivariate Imputation by Chained Equations and Ensemble Learning

  • Jiaxi Li
  • Zhelong Wang
  • Lina Wu
  • Sen Qiu
  • Hongyu Zhao
  • Fang Lin
  • Ke Zhang

The classification analysis of incomplete and imbalanced data is still a challenging task since these issues could negatively impact the training of classifiers, which were also found in our study on the physical fitness assessments of patients. And in fields such as healthcare, there are higher requirements for the accuracy of the generated imputation values. To train a high-performance classifier and pursue high accuracy, we attempted to resolve any potential negative impact by using a novel algorithmic approach based on the combination of multivariate imputation by chained equations and the ensemble learning method (MICEEN), which can solve the two problems simultaneously. We used multivariate imputation by chained equations to generate more accurate imputation values for the training set passed to ensemble learning to build a predictor. On the other hand, missing values were introduced into minority classes and used them to generate new samples belonging to the minority classes in order to balance the distribution of classes. On real-world datasets, we perform extensive experiments to assess our method and compare it to other state-of-the-art approaches. The advantages of the proposed method are demonstrated by experimental results for the benchmark datasets and self-collected datasets of physical fitness assessment of tumor patients with varying missing rates.

Details DOI

ICML Conference 2024 Conference Paper

Nearest Neighbour Score Estimators for Diffusion Generative Models

  • Matthew Niedoba
  • Dylan Green
  • Saeid Naderiparizi
  • Vasileios Lioutas
  • Jonathan Wilder Lavington
  • Xiaoxuan Liang 0001
  • Yunpeng Liu 0007
  • Ke Zhang

Score function estimation is the cornerstone of both training and sampling from diffusion generative models. Despite this fact, the most commonly used estimators are either biased neural network approximations or high variance Monte Carlo estimators based on the conditional score. We introduce a novel nearest neighbour score function estimator which utilizes multiple samples from the training set to dramatically decrease estimator variance. We leverage our low variance estimator in two compelling applications. Training consistency models with our estimator, we report a significant increase in both convergence speed and sample quality. In diffusion models, we show that our estimator can replace a learned network for probability-flow ODE integration, opening promising new avenues of future research. Code will be released upon paper acceptance.

Details

JBHI Journal 2024 Journal Article

Spatio-Temporal Classification of Lung Ventilation Patterns Using 3D EIT Images: A General Approach for Individualized Lung Function Evaluation

  • Shuzhe Chen
  • Li Li
  • Zhichao Lin
  • Ke Zhang
  • Ying Gong
  • Lu Wang
  • Xu Wu
  • Maokun Li

The Pulmonary Function Test (PFT) is a widely utilized and rigorous classification test for evaluating lung function, serving as a comprehensive diagnostic tool for lung conditions. Meanwhile, Electrical Impedance Tomography (EIT) is a rapidly advancing clinical technique that visualizes conductivity distribution induced by ventilation. EIT provides additional spatial and temporal information on lung ventilation beyond traditional PFT. However, relying solely on conventional isolated interpretations of PFT results and EIT images overlooks the continuous dynamic aspects of lung ventilation. This study aims to classify lung ventilation patterns by extracting spatial and temporal features from the 3D EIT image series. The study uses a Variational Autoencoder (VAE) with a MultiRes block to compress the spatial distribution in a 3D image into a one-dimensional vector. These vectors are then stacked to create a feature map for the exhibition of temporal features. A simple convolutional neural network is used for classification. Data from 137 subjects were utilized for the training phase. Initially, the model underwent validation through a leave-one-out cross-validation process. During this validation, the model achieved an accuracy and sensitivity of 0. 96 and 1. 00, respectively, with an f1-score of 0. 98 when identifying the normal subjects. To assess pipeline reliability and feasibility, we tested it on 9 newly recruited subjects, with accurate ventilation mode predictions for 8 out of 9. In addition, we included 2D EIT results for comparison and conducted ablation experiments to validate the effectiveness of the VAE. The study demonstrates the potential of using image series for lung ventilation mode classification, providing a feasible method for patient prescreening and presenting an alternative form of PFT.

Details DOI

NeurIPS Conference 2023 Conference Paper

AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix

  • Yun Yue
  • Zhiling Ye
  • Jiadi Jiang
  • Yongchao Liu
  • Ke Zhang

Adaptive optimizers, such as Adam, have achieved remarkable success in deep learning. A key component of these optimizers is the so-called preconditioning matrix, providing enhanced gradient information and regulating the step size of each gradient direction. In this paper, we propose a novel approach to designing the preconditioning matrix by utilizing the gradient difference between two successive steps as the diagonal elements. These diagonal elements are closely related to the Hessian and can be perceived as an approximation of the inner product between the Hessian row vectors and difference of the adjacent parameter vectors. Additionally, we introduce an auto-switching function that enables the preconditioning matrix to switch dynamically between Stochastic Gradient Descent (SGD) and the adaptive optimizer. Based on these two techniques, we develop a new optimizer named AGD that enhances the generalization performance. We evaluate AGD on public datasets of Natural Language Processing (NLP), Computer Vision (CV), and Recommendation Systems (RecSys). Our experimental results demonstrate that AGD outperforms the state-of-the-art (SOTA) optimizers, achieving highly competitive or significantly better predictive performance. Furthermore, we analyze how AGD is able to switch automatically between SGD and the adaptive optimizer and its actual effects on various scenarios. The code is available at https: //github. com/intelligent-machine-learning/dlrover/tree/master/atorch/atorch/optimizers.

PDF Details

EAAI Journal 2023 Journal Article

Feature pre-inpainting enhanced transformer for video inpainting

  • Guanxiao Li
  • Ke Zhang
  • Yu Su
  • Jingyu Wang

Transformer-based video inpainting methods aggregate coherent contents into missing regions by learning dependencies spatial–temporally. However, existing methods suffer from the inaccurate self-attention calculation and excessive quadratic computational complexity, due to uninformative representations of missing regions and inefficient global self-attention mechanisms, respectively. To mitigate these problems, we propose a Feature pre-Inpainting enhanced Transformer (FITer) video inpainting method, in which the feature pre-inpainting network (FPNet) and local–global interleaving Transformer are designed. The FPNet pre-inpaints missing features before the Transformer by exploiting spatial context, and the representations of missing regions are thus enhanced with more informative content. Therefore, the interleaving Transformer can calculate more accurate self-attention weights and learns more effective dependencies between missing and valid regions. Since the interleaving Transformer involves both global and window-based local self-attention mechanisms, the proposed FITer method can effectively aggregate spatial–temporal features into missing regions while improving efficiency. Experiments on YouTube-VOS and DAVIS datasets demonstrate that the FITer method outperforms previous methods qualitatively and quantitatively.

Details DOI

NeurIPS Conference 2023 Conference Paper

RH-BrainFS: Regional Heterogeneous Multimodal Brain Networks Fusion Strategy

  • Hongting Ye
  • Yalu Zheng
  • Yueying Li
  • Ke Zhang
  • Youyong Kong
  • Yonggui Yuan

Multimodal fusion has become an important research technique in neuroscience that completes downstream tasks by extracting complementary information from multiple modalities. Existing multimodal research on brain networks mainly focuses on two modalities, structural connectivity (SC) and functional connectivity (FC). Recently, extensive literature has shown that the relationship between SC and FC is complex and not a simple one-to-one mapping. The coupling of structure and function at the regional level is heterogeneous. However, all previous studies have neglected the modal regional heterogeneity between SC and FC and fused their representations via "simple patterns", which are inefficient ways of multimodal fusion and affect the overall performance of the model. In this paper, to alleviate the issue of regional heterogeneity of multimodal brain networks, we propose a novel Regional Heterogeneous multimodal Brain networks Fusion Strategy (RH-BrainFS). Briefly, we introduce a brain subgraph networks module to extract regional characteristics of brain networks, and further use a new transformer-based fusion bottleneck module to alleviate the issue of regional heterogeneity between SC and FC. To the best of our knowledge, this is the first paper to explicitly state the issue of structural-functional modal regional heterogeneity and to propose asolution. Extensive experiments demonstrate that the proposed method outperforms several state-of-the-art methods in a variety of neuroscience tasks.

PDF Details

JMLR Journal 2023 Journal Article

SQLFlow: An Extensible Toolkit Integrating DB and AI

  • Jun Zhou
  • Ke Zhang
  • Lin Wang
  • Hua Wu
  • Yi Wang
  • Chaochao Chen

Integrating AI algorithms into databases is an ongoing effort in both academia and industry. We introduce SQLFlow, a toolkit seamlessly combining data manipulations and AI operations that can be run locally or remotely. SQLFlow extends SQL syntax to support typical AI tasks including model training, inference, interpretation, and mathematical optimization. It is compatible with a variety of database management systems (DBMS) and AI engines, including MySQL, TiDB, MaxCompute, and Hive, as well as TensorFlow, scikit-learn, and XGBoost. Documentations and case studies are available at https://sqlflow.org. The source code and additional details can be found at https://github.com/sql-machine-learning/sqlflow. &copy JMLR 2023. ( edit, beta )

PDF Details

IJCAI Conference 2022 Conference Paper

Hierarchical Diffusion Scattering Graph Neural Network

  • Ke Zhang
  • Xinyan Pu
  • Jiaxing Li
  • Jiasong Wu
  • Huazhong Shu
  • Youyong Kong

Graph neural network (GNN) is popular now to solve the tasks in non-Euclidean space and most of them learn deep embeddings by aggregating the neighboring nodes. However, these methods are prone to some problems such as over-smoothing because of the single-scale perspective field and the nature of low-pass filter. To address these limitations, we introduce diffusion scattering network (DSN) to exploit high-order patterns. With observing the complementary relationship between multi-layer GNN and DSN, we propose Hierarchical Diffusion Scattering Graph Neural Network (HDS-GNN) to efficiently bridge DSN and GNN layer by layer to supplement GNN with multi-scale information and band-pass signals. Our model extracts node-level scattering representations by intercepting the low-pass filtering, and adaptively tunes the different scales to regularize multi-scale information. Then we apply hierarchical representation enhancement to improve GNN with the scattering features. We benchmark our model on nine real-world networks on the transductive semi-supervised node classification task. The experimental results demonstrate the effectiveness of our method.

PDF Details DOI

EAAI Journal 2022 Journal Article

MRRNet: Learning multiple region representation for video person re-identification

  • Hui Fu
  • Ke Zhang
  • Haoyu Li
  • Jingyu Wang

Video person re-identification is a crucial component of a robust surveillance system. Within a video clip, different human regions exhibit unique stability characteristics, which would be harmful to generating a discriminative representation. Unfortunately, prior works cannot effectively deal with the stability characteristics of different regions. To tackle this problem, we propose a Multiple Region Representation Network (MRRNet) that aims to discover the discriminative information from different human regions. Firstly, a Stable Region Representation (SRR) layer is proposed to capture important clues from the stable regions and exchange temporal information by cross-relation aware operation. Secondly, a Multiple Region Representation (MRR) layer is designed to address the unstable regions and preserve the attention on stable regions. Thirdly, SRR and MRR can be conveniently inserted into multiple stages of the deep residual networks and significantly improve the performance of the network. Comprehensive experiments validate the effectiveness of our network. Particularly, MRRNet achieves 86. 7% mAP and 91. 1% Rank-1 accuracy on the MARS dataset, which outperforms state-of-the-arts.

Details DOI

IJCAI Conference 2021 Conference Paper

Secure Deep Graph Generation with Link Differential Privacy

  • Carl Yang
  • Haonan Wang
  • Ke Zhang
  • Liang Chen
  • Lichao Sun

Many data mining and analytical tasks rely on the abstraction of networks (graphs) to summarize relational structures among individuals (nodes). Since relational data are often sensitive, we aim to seek effective approaches to generate utility-preserved yet privacy-protected structured data. In this paper, we leverage the differential privacy (DP) framework to formulate and enforce rigorous privacy constraints on deep graph generation models, with a focus on edge-DP to guarantee individual link privacy. In particular, we enforce edge-DP by injecting designated noise to the gradients of a link reconstruction based graph generation model, while ensuring data utility by improving structure learning with structure-oriented graph discrimination. Extensive experiments on two real-world network datasets show that our proposed DPGGAN model is able to generate graphs with effectively preserved global structure and rigorously protected individual link privacy.

PDF Details DOI

IJCAI Conference 2021 Conference Paper

Structure Guided Lane Detection

  • Jinming Su
  • Chao Chen
  • Ke Zhang
  • Junfeng Luo
  • Xiaoming Wei
  • Xiaolin Wei

Recently, lane detection has made great progress with the rapid development of deep neural networks and autonomous driving. However, there exist three mainly problems including characterizing lanes, modeling the structural relationship between scenes and lanes, and supporting more attributes (e. g. , instance and type) of lanes. In this paper, we propose a novel structure guided framework to solve these problems simultaneously. In the framework, we first introduce a new lane representation to characterize each instance. Then a top-down vanishing point guided anchoring mechanism is proposed to produce intensive anchors, which efficiently capture various lanes. Next, multi-level structural constraints are used to improve the perception of lanes. In the process, pixel-level perception with binary segmentation is introduced to promote features around anchors and restore lane details from bottom up, a lane-level relation is put forward to model structures (i. e. , parallel) around lanes, and an image-level attention is used to adaptively attend different regions of the image from the perspective of scenes. With the help of structural guidance, anchors are effectively classified and regressed to obtain precise locations and shapes. Extensive experiments on public benchmark datasets show that the proposed approach outperforms state-of-the-art methods with 117 FPS on a single GPU.

PDF Details DOI

NeurIPS Conference 2021 Conference Paper

Subgraph Federated Learning with Missing Neighbor Generation

  • Ke Zhang
  • Carl Yang
  • Xiaoxiao Li
  • Lichao Sun
  • Siu Ming Yiu

Graphs have been widely used in data mining and machine learning due to their unique representation of real-world objects and their interactions. As graphs are getting bigger and bigger nowadays, it is common to see their subgraphs separately collected and stored in multiple local systems. Therefore, it is natural to consider the subgraph federated learning setting, where each local system holds a small subgraph that may be biased from the distribution of the whole graph. Hence, the subgraph federated learning aims to collaboratively train a powerful and generalizable graph mining model without directly sharing their graph data. In this work, towards the novel yet realistic setting of subgraph federated learning, we propose two major techniques: (1) FedSage, which trains a GraphSage model based on FedAvg to integrate node features, link structures, and task labels on multiple local subgraphs; (2) FedSage+, which trains a missing neighbor generator along FedSage to deal with missing links across local subgraphs. Empirical results on four real-world graph datasets with synthesized subgraph federated learning settings demonstrate the effectiveness and efficiency of our proposed techniques. At the same time, consistent theoretical implications are made towards their generalization ability on the global graphs.

PDF Details

AAAI Conference 2020 Conference Paper

Unsupervised Detection of Sub-Events in Large Scale Disasters

  • Chidubem Arachie
  • Manas Gaur
  • Sam Anzaroot
  • William Groves
  • Ke Zhang
  • Alejandro Jaimes

Social media plays a major role during and after major natural disasters (e. g. , hurricanes, large-scale fires, etc.), as people “on the ground” post useful information on what is actually happening. Given the large amounts of posts, a major challenge is identifying the information that is useful and actionable. Emergency responders are largely interested in finding out what events are taking place so they can properly plan and deploy resources. In this paper we address the problem of automatically identifying important sub-events (within a large-scale emergency “event”, such as a hurricane). In particular, we present a novel, unsupervised learning framework to detect sub-events in Tweets for retrospective crisis analysis. We first extract noun-verb pairs and phrases from raw tweets as sub-event candidates. Then, we learn a semantic embedding of extracted noun-verb pairs and phrases, and rank them against a crisis-specific ontology. We filter out noisy and irrelevant information then cluster the noun-verb pairs and phrases so that the top-ranked ones describe the most important sub-events. Through quantitative experiments on two large crisis data sets (Hurricane Harvey and the 2015 Nepal Earthquake), we demonstrate the effectiveness of our approach over the state-of-the-art. Our qualitative evaluation shows better performance compared to our baseline.

PDF Details

AAAI Conference 2014 Conference Paper

Semantic Segmentation Using Multiple Graphs with Block-Diagonal Constraints

  • Ke Zhang
  • Wei Zhang
  • Sheng Zeng
  • Xiangyang Xue

In this paper we propose a novel method for image semantic segmentation using multiple graphs. The multiview affinity graph is constructed by leveraging the consistency between semantic space and multiple visual spaces. With block-diagonal constraints, we enforce the affinity matrix to be sparse such that the pairwise potential for dissimilar superpixels is close to zero. By a divide-and-conquer strategy, the optimization for learning affinity matrix is decomposed into several subproblems that can be solved in parallel. Using the neighborhood relationship between superpixels and the consistency between affinity matrix and labelconfidence matrix, we infer the semantic label for each superpixel of unlabeled images by minimizing an objective whose closed form solution can be easily obtained. Experimental results on two real-world image datasets demonstrate the effectiveness of our method.

PDF Details

IJCAI Conference 2013 Conference Paper

Multi-View Embedding Learning for Incompletely Labeled Data

  • Wei Zhang
  • Ke Zhang
  • Pan Gu
  • Xiangyang Xue

In many applications, the data may be high dimensional, represented by multiple features, and associated with more than one labels. Embedding learning is an effective strategy for dimensionality reduction and for nearest neighbor search in massive datasets. We propose a novel method to seek compact embedding that allows efficient retrieval with incompletely-labeled multi-view data. Based on multi-graph Laplacian, we achieve the optimal combination of heterogeneous features to effectively describe data, which exploits the feature correlations between different views. We learn the embedding that preserves the neighborhood context in the original spaces, and obtain the complete labels simultaneously. Inter-label correlations are sufficiently leveraged in the proposed framework. Our goal is to find the maps from multiple input spaces to the compact embedding space and to the semantic concept space at the same time. There is semantic gap between the input multi-view feature spaces and the semantic concept space; and the compact embedding space can be looked on as the bridge between the above spaces. Experimental evaluation on three real-world datasets demonstrates the effectiveness of the proposed method.

PDF Details DOI

IJCAI Conference 2013 Conference Paper

Sparse Reconstruction for Weakly Supervised Semantic Segmentation

  • Ke Zhang
  • Wei Zhang
  • Yingbin Zheng
  • Xiangyang Xue

We propose a novel approach to semantic segmentation using weakly supervised labels. In traditional fully supervised methods, superpixel labels are available for training; however, it is not easy to obtain enough labeled superpixels to learn a satisfying model for semantic segmentation. By contrast, only image-level labels are necessary in weakly supervised methods, which makes them more practical in real applications. In this paper we develop a new way of evaluating classification models for semantic segmentation given weekly supervised labels. For a certain category, provided the classi- fication model parameter, we firstly learn the basis superpixels by sparse reconstruction, and then evaluate the parameters by measuring the reconstruction errors among negative and positive superpixels. Based on Gaussian Mixture Models, we use Iterative Merging Update (IMU) algorithm to obtain the best parameters for the classification models. Experimental results on two real-world datasets show that the proposed approach outperforms the existing weakly supervised methods, and it also competes with state-of-the-art fully supervised methods.

PDF Details DOI
v2026.09.13