Arrow Research search

Author name cluster

Han Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

76 papers
2 author rows

Possible papers

76

EAAI Journal 2026 Journal Article

Accurate detection and characterization of sub-millimeter cracks using nonlinear ultrasonics-informed parallel multi-branch convolutional neural network

  • Lu Wang
  • Zhengpan Qi
  • Xiangyan Ding
  • Ning Hu
  • Xiaoyang Bi
  • Han Zhang
  • Libin Zhao

Conventional ultrasonic testing struggles to inspect sub-millimeter cracks and identify multiple characteristics. To overcome these limitations, this study proposes a parallel multi-branch convolutional neural network (PMCNN) to simultaneously and accurately detect the depth, length, and orientation of sub-millimeter cracks. The artificial intelligence (AI) innovation lies in the PMCNN's branch-specific kernels and cross-task isolation layers that effectively decouple overlapping nonlinear ultrasonic signals. First, ultrasonic non-destructive testing was performed on micro-crack specimens to obtain essential data for verifying both the finite element (FE) model and PMCNN. Subsequently, an FE model was established to systematically analyze the coupling effects of depth, length, and orientation on the signals and to generate a comprehensive dataset for PMCNN training. The primary engineering application is the provision of an effective solution for quantitative micro-crack assessment in complex operational environments through experimental validation with practical inspection signals. Results reveal that while harmonic amplitudes correlated with individual parameter variations, their sensitivity significantly decreases under multiparameter conditions. Interpretability analysis confirms the distinct feature selectivity of each branch network, while a hybrid data training strategy maintains robust accuracy (above 90 %) under noisy conditions. Experimental validation demonstrates that the proposed method achieves stable and reliable performance, bridging advanced AI techniques with practical structural health monitoring needs.

Details DOI

AIJ Journal 2026 Journal Article

Approximate multi-objective search

  • Han Zhang
  • Oren Salzman
  • T. K. Satish Kumar
  • Ariel Felner
  • Carlos Hernández Ulloa
  • Sven Koenig

In multi-objective search, we consider a graph whose edges are annotated with multiple cost components. A typical task is to compute the Pareto frontier, i. e. , the set of all undominated paths from a given start state to a given goal state of the graph. However, the size of the Pareto frontier can be exponential in the size of the graph, and computing the entire Pareto frontier can be very time-consuming. Therefore, in this paper, we study how to find an approximate frontier for a user-provided approximation factor. Such a frontier can be significantly smaller than the Pareto frontier, enabling the design of efficient approximate multi-objective search algorithms. We present such an algorithm called A*pex, which uses an efficient, albeit approximate, representation of paths with similar costs to compute an approximate frontier. During the search, it avoids storing all paths explicitly, thereby reducing the search effort. We show that A*pex can be used as an algorithmic building block to solve additional problems by presenting (1) an anytime variant of A*pex, which computes an initial approximate frontier quickly and then works to find better approximate frontiers until eventually finding the entire Pareto frontier and (2) a variant of A*pex for approximately solving the Weight-Constrained Shortest Path (WCSP) problem, a problem that is closely related to multi-objective search. For evaluation, we use road networks from the 9th DIMACS Implementation Challenge: Shortest Path for evaluation. Our experimental results show that A*pex and its WCSP variant can outperform respective state-of-the-art algorithms by orders of magnitude in terms of runtime. We also show that the anytime variant of A*pex often computes better approximate frontiers than state-of-the-art algorithms, given a limited runtime.

Details DOI

AAAI Conference 2026 Conference Paper

FACTGUARD: Event-Centric and Commonsense-Guided Fake News Detection

  • Jing He
  • Han Zhang
  • Yuanhui Xiao
  • Wei Guo
  • Shaowen Yao
  • Renyang Liu

Fake news detection methods based on writing style have achieved remarkable progress. However, as adversaries increasingly imitate the style of authentic news, the effectiveness of such approaches is gradually diminishing. Recent research has explored incorporating large language models (LLMs) to enhance fake news detection. Yet, despite their transformative potential, LLMs remain an untapped goldmine for fake news detection, with their real-world adoption hampered by shallow functionality exploration, ambiguous usability, and prohibitive inference costs. In this paper, we propose a novel fake news detection framework, dubbed FACTGUARD, that leverages LLMs to extract event-centric content, thereby reducing the impact of writing style on detection performance. Furthermore, our approach introduces a dynamic usability mechanism that identifies contradictions and ambiguous cases in factual reasoning, adaptively incorporating LLM advice to improve decision reliability. To ensure efficiency and practical deployment, we employ knowledge distillation to derive FACTGUARD-D, enabling the framework to operate effectively in cold-start and resource-constrained scenarios. Comprehensive experiments on two benchmark datasets demonstrate that our approach consistently outperforms existing methods in both robustness and accuracy, effectively addressing the challenges of style sensitivity and LLM usability in fake news detection.

PDF Details DOI

JBHI Journal 2026 Journal Article

Image2Gene: A Minimalist and Weakly-Supervised Framework for Morphology-Aligned Gene Expression Prediction From Histology Images

  • Weiqi Fu
  • Xiongwen Quan
  • Shuang Bai
  • Han Zhang

Gene expression prediction from histological images offers a promising approach for spatial transcriptome analysis without expensive sequencing. We present Image2Gene, a simple yet effective weakly supervised contrastive learning framework that predicts gene expression profiles directly from tissue morphology using only an image encoder and multiple fully connected layers. Rather than relying on complex modules or gene expression-based embedding spaces, our approach does not rely on complex modules or gene expression-based embedding spaces. Instead, it encodes spatial coordinates via a learnable embedding and uses an image encoder to extract histological image features. We then propose a novel contrastive loss function that minimizes the difference between the cosine self-similarity of image embeddings and the Pearson autocorrelation of corresponding gene expression profiles to learn a structured image representation that reflects gene expression variations. Unlike previous methods for cross-heterogeneous modality matching, our approach aligns samples solely in image space, enabling more robust and biologically meaningful similarity learning. Finally, we perform gene expression inference via k-Nearest Neighbor interpolation in the learned image embedding space. Despite its simple architecture, extensive experiments on HER+ and cSCC spatial transcriptome datasets demonstrate that Image2Gene achieves highly competitive performance, highlighting its potential as a scalable, annotation-free alternative for inferring transcriptome patterns directly from histological sections.

Details DOI

AAAI Conference 2026 Conference Paper

Out-of-Context Misinformation Detection via Variational Domain-Invariant Learning with Test-Time Training

  • Xi Yang
  • Han Zhang
  • Zhijian Lin
  • Yibiao Hu
  • Hong Han

Out-of-context misinformation (OOC) is a low-cost form of misinformation in news reports, which refers to place authentic images into out-of-context or fabricated image-text pairings. This problem has attracted significant attention from researchers in recent years. Current methods focus on assessing image-text consistency or generating explanations. However, these approaches assume that the training and test data are drawn from the same distribution. When encountering novel news domains, models tend to perform poorly due to the lack of prior knowledge. To address this challenge, we propose Variational Domain-Invariant Learning with Test-Time Training (VDT) framework to enhance the domain adaptation capability for OOC misinformation detection. Domain-Invariant Variational Align module is employed to jointly encodes source and target domain data to learn a separable distributional space and domain-invariant features. For preserving semantic integrity, we utilize domain consistency constraint module to reconstruct the source and target domain latent distribution. During testing phase, we adopt the test-time training strategy and confidence-variance filtering module to dynamically updating the VAE encoder and classifier, facilitating the model's adaptation to the target domain distribution. Extensive experiments conducted on the benchmark dataset NewsCLIPpings demonstrate that our method outperforms state-of-the-art baselines under most domain adaptation settings.

PDF Details DOI

JBHI Journal 2026 Journal Article

SMFusion: Semantic-Preserving Fusion of Multimodal Medical Images for Enhanced Clinical Diagnosis

  • Haozhe Xiang
  • Han Zhang
  • Yu Cheng
  • Xiongwen Quan
  • Wanwan Huang

Multimodal medical image fusion plays a crucial role in medical diagnosis by integrating complementary information from different modalities to enhance image readability and clinical applicability. However, existing methods mainly follow computer vision standards for feature extraction and fusion strategy formulation, overlooking the rich semantic information inherent in medical images. To address this limitation, we propose a novel semantic-guided medical image fusion approach that, for the first time, incorporates medical prior knowledge into the fusion process. Specifically, we construct a publicly available multimodal medical image-text dataset, upon which text descriptions generated by BiomedGPT are encoded and semantically aligned with image features in a high-dimensional space via a semantic interaction alignment module. During this process, a cross attention based linear transformation automatically maps the relationship between textual and visual features to facilitate comprehensive learning. The aligned features are then embedded into a text-injection module for further feature-level fusion. Unlike traditional methods, we further generate diagnostic reports from the fused images to assess the preservation of medical information. Additionally, we design a medical semantic loss function to enhance the retention of textual cues from the source images. Experimental results on test datasets demonstrate that the proposed method achieves superior performance in both qualitative and quantitative evaluations while preserving more critical medical information.

Details DOI

AAAI Conference 2026 Conference Paper

Spikingformer: A Key Foundation Model for Spiking Neural Networks

  • Chenlin Zhou
  • Liutao Yu
  • Zhaokun Zhou
  • Han Zhang
  • Jiaqi Wang
  • Huihui Zhou
  • Zhengyu Ma
  • Yonghong Tian

Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation. However, some foundation SNN backbones (including Spikformer and SEW ResNet) suffer from non-spike computations (integer-float multiplications) caused by the structure of their residual connections. These non-spike computations increase SNNs' power consumption and make them unsuitable for deployment on mainstream neuromorphic hardware. In this paper, we analyze the spike-driven behavior of the residual connection methods in SNNs. We then present Spikingformer, a novel spiking transformer backbone that merges the MS Residual connection with Self-Attention in a biologically plausible way to address the non-spike computation challenge in Spikformer while maintaining global modeling capabilities. We evaluate Spikingformer across 13 datasets spanning large static images, neuromorphic data, and natural language tasks, and demonstrate the effectiveness and universality of Spikingformer, setting a vital benchmark for spiking neural networks. In addition, with the spike-driven features and global modeling capabilities, Spikingformer is expected to become a more efficient general-purpose SNN backbone towards energy-efficient artificial intelligence.

PDF Details DOI

AAAI Conference 2026 Conference Paper

STELAR-VISION: Self-Topology-Aware Efficient Learning for Aligned Reasoning in Vision

  • Chen Li
  • Han Zhang
  • Zhantao Yang
  • Fangyi Chen
  • Zihan Wang
  • Anudeepsekhar Bolimera
  • Marios Savvides

Vision-language models (VLMs) have made significant strides in reasoning, yet they often struggle with complex multimodal tasks and tend to generate overly verbose outputs. A key limitation is their reliance on chain-of-thought (CoT) reasoning, despite many tasks benefiting from alternative topologies like trees or graphs. To address this, we introduce STELAR-Vision, a training framework for topology-aware reasoning. At its core is TopoAug, a synthetic data pipeline that enriches training with diverse topological structures. Using supervised fine-tuning and reinforcement learning, we post-train Qwen2VL models with both accuracy and efficiency in mind. Additionally, we propose Frugal Learning, which reduces output length with minimal accuracy loss. On MATH-V and VLM_S2H, STELAR-Vision improves accuracy by 9.7% over its base model and surpasses the larger Qwen2VL-72B-Instruct by 7.3%. On five out-of-distribution benchmarks, it outperforms Phi-4-Multimodal-Instruct by up to 28.4% and LLaMA-3.2-11B-Vision-Instruct by up to 13.2%, demonstrating strong generalization. Compared to Chain-Only training, our approach achieves 4.3% higher overall accuracy on in-distribution datasets and consistently outperforms across all OOD benchmarks.

PDF Details DOI

IROS Conference 2025 Conference Paper

A 4D Radar Camera Extrinsic Calibration Tool Based on 3D Uncertainty Perspective N Points

  • Chuan Cao
  • Xiaoning Wang
  • Wenqian Xi
  • Han Zhang
  • Weidong Chen
  • Jingchuan Wang

4D imaging radar is a type of low-cost millimeter-wave radar(costing merely 10-20% of lidar systems) capable of providing range, azimuth, elevation, and Doppler velocity information. Accurate extrinsic calibration between millimeter-wave radar and camera systems is critical for robust multimodal perception in robotics, yet remains challenging due to inherent sensor noise characteristics and complex error propagation. This paper presents a systematic calibration framework to address critical challenges through a spatial 3d uncertainty-aware PnP algorithm (3DUPnP) that explicitly models spherical coordinate noise propagation in radar measurements, then compensating for non-zero error expectations during coordinate transformations. Finally, experimental validation demonstrates significant performance improvements over state-of-the-art CPnP baseline, including improved consistency in simulations and enhanced precision in physical experiments. This study provides a robust calibration solution for robotic systems equipped with millimeter-wave radar and cameras, tailored specifically for autonomous driving and robotic perception applications.

Details

EAAI Journal 2025 Journal Article

A survey on learning with noisy labels in Natural Language Processing: How to train models with label noise

  • Han Zhang
  • Yazhou Zhang
  • Jiajun Li
  • Junxiu Liu
  • Lixia Ji

When applying deep neural network language models to related systems (e. g. , question answering systems, chatbots, and intelligent assistants), many datasets contain different types or degrees of label noise. Label noise can lead to a decline in model performance and an increase in resource consumption. Therefore, learning with noisy labels is becoming an important task in Natural Language Processing (NLP). This paper aims to collect, analyze, and evaluate methods for learning with label noise in NLP. First, we analyze the relationship between data feature extraction, prediction output, and optimization in the context of noise robustness to help researchers understand the mechanisms behind noise generation. Based on this, we classified the noise processing methods into five types according to the training process: feature vector, transition matrix, prediction confidence, loss improvement, and data weighting. We analyze each method and conduct a systematic evaluation across six metrics. In addition, we summarized the commonly used resources such as datasets, open source codes, etc. Finally, we also analyzed the challenges faced in current research and the potential opportunities. As a comprehensive survey, this work will help researchers and industry developers to understand the current state of research and unique challenges facing label-noise learning, which facilitate the selection and combination of different methods in applications to further advancements.

Details DOI

JBHI Journal 2025 Journal Article

Automatic Multi-Task Segmentation and Vulnerability Assessment of Carotid Plaque on Contrast-Enhanced Ultrasound Images and Videos via Deep Learning

  • Bokai Hu
  • Han Zhang
  • Caixia Jia
  • Ke Chen
  • Xiangjiang Tang
  • Da He
  • Luni Zhang
  • Shiyao Gu

Intraplaque neovascularization (IPN) within carotid plaque is a crucial indicator of plaque vulnerability. Contrast-enhanced ultrasound (CEUS) is a valuable tool for assessing IPN by evaluating the location and quantity of microbubbles within the carotid plaque. However, this task is typically performed by experienced radiologists. Here we propose a deep learning-based multi-task model for the automatic segmentation and IPN grade classification of carotid plaque on CEUS images and videos. We also compare the performance of our model with that of radiologists. To simulate the clinical practice of radiologists, who often use CEUS videos with dynamic imaging to track microbubble flow and identify IPN, we develop a workflow for plaque vulnerability assessment using CEUS videos. Our multi-task model outperformed individually trained segmentation and classification models, achieving superior performance in IPN grade classification based on CEUS images. Specifically, our model achieved a high segmentation Dice coefficient of 84. 64% and a high classification accuracy of 81. 67%. Moreover, our model surpassed the performance of junior and medium-level radiologists, providing more accurate IPN grading of carotid plaque on CEUS images. For CEUS videos, our model achieved a classification accuracy of 80. 00% in IPN grading. Overall, our multi-task model demonstrates great performance in the automatic, accurate, objective, and efficient IPN grading in both CEUS images and videos. This work holds significant promise for enhancing the clinical diagnosis of plaque vulnerability associated with IPN in CEUS evaluations.

Details DOI

AAAI Conference 2025 Conference Paper

BeyondGender: A Multifaceted Bilingual Dataset for Practical Sexism Detection

  • Xuan Luo
  • Li Yang
  • Han Zhang
  • Geng Tu
  • Qianlong Wang
  • Keyang Ding
  • Chuang Fan
  • Jing Li

Sexism affects both women and men, yet research often overlooks misandry and suffers from overly broad annotations that limit AI applications. To address this, we introduce BeyondGender, a dataset meticulously annotated according to the latest definitions of misogyny and misandry. It features innovative multifaceted labels encompassing aspects of sexism, gender, phrasing, misogyny, and misandry. The dataset includes 6K English and 1.7K Chinese sexism instances, alongside 13K non-sexism examples. Our evaluations of masked language models and large language models reveal that they detect misogyny in English and misandry in Chinese more effectively, with F1-scores of 0.87 and 0.62, respectively. However, they frequently misclassify hostile and mild comments, underscoring the complexity of sexism detection. Parallel corpus experiments suggest promising data augmentation strategies to enhance AI systems for nuanced sexism detection, and our dataset can be leveraged to improve value alignment in large language models.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Correcting Large Language Model Behavior via Influence Function

  • Han Zhang
  • Zhuo Zhang
  • Yi Zhang
  • Yuanzhao Zhai
  • Hanyang Peng
  • Yu Lei
  • Yue Yu
  • Hui Wang

Recent advancements in AI alignment techniques have significantly improved the alignment of large language models (LLMs) with static human preferences. However, the dynamic nature of human preferences can render some prior training data outdated or even erroneous, ultimately causing LLMs to deviate from contemporary human preferences and societal norms. Existing methodologies, either curation of new data for continual alignment or manual correction of outdated data for re-alignment, demand costly human resources. To address this, we propose a novel approach, LLM BehAvior Correction with INfluence FunCtion REcall and Post-Training (LANCET), which needs no human involvement. LANCET consists of two phases: (1) using a new method LinFAC to efficiently identify the training data that significantly impact undesirable model outputs, and (2) applying an novel Influence-driven Bregman Optimization (IBO) technique to adjust the model’s outputs based on these influence distributions. Our experiments show that LANCET effectively and efficiently corrects inappropriate behaviors of LLMs while preserving model utility. Further more, LANCET exhibits stronger generalization ability than all baselines under out-of-distribution harmful prompts, offering better interpretability and compatibility with real-world applications of LLMs.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Inheriting Generalized Learngene for Efficient Knowledge Transfer across Multiple Tasks

  • Yuankun Zu
  • Shiyu Xia
  • Xu Yang
  • Qiufeng Wang
  • Han Zhang
  • Xin Geng

In practical applications, it is often necessary to transfer knowledge from large pretrained models to small ones with various architectures for tackling different tasks. The Learngene framework, proposed recently, firstly extracts one compact module termed as learngene from a large well-trained model, after which learngene is used to build descendant models for handling diverse tasks. In this paper, we aim to explore extracting and inheriting learngene which can be generalized across different model architectures and tasks, remaining understudied in previous works. Inspired by the existing observations that large kernel convolutional neural networks (CNNs) exhibit significant generalization potential across various architectures and tasks, we propose a novel two-stage Learngene method termed CLKG (Convolutional Learngene for Knowledge Generalization), which inherits convolutional kernels containing generalized knowledge as learngene to build diverse models for multiple tasks. Specifically, we construct an auxiliary model comprised of small kernels and train it through dense feature distillation to inherit the feature extraction ability from large kernel CNNs. After distillation, we select certain kernels from the auxiliary model as learngene based on three criteria: direct kernel extraction, priority to edge kernels, and continuous kernel selection. Subsequently, we adapt learngene according to the width of the descendant models and use it to initialize the backbone part of descendant models. Experiments on diverse vision tasks such as image classification, object detection and semantic segmentation demonstrate the superiority of CLKG. For example, compared with from scratch training, it brings 2.89% improvements on VOC12+SBD, and reduces around 2x training data volume and training epochs to achieve better results. Furthermore, compared to knowledge distillation method, CLKG significantly reduces negative transfer on certain datasets, e.g., achieves 1.88% performance improvements on NAO dataset despite domain differences.

PDF Details DOI

AAAI Conference 2025 Conference Paper

MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement

  • Xu He
  • Zhiyong Wu
  • Xiaoyu Li
  • Di Kang
  • Chaopeng Zhang
  • Jiangnan Ye
  • Liyang Chen
  • Xiangjun Gao

Existing works in single-image human reconstruction suffer from weak generalizability due to insufficient training data or 3D inconsistencies for a lack of comprehensive multi-view knowledge. In this paper, we introduce MagicMan, a human-specific multi-view diffusion model to generate high-quality novel views from a single reference image. As its core, we leverage a pre-trained 2D diffusion model as the generative prior for generalizability, with the parametric SMPL-X model as the 3D body prior to promote 3D awareness. To maintain consistency while generating denser views for improved 3D human reconstruction, we introduce hybrid multi-view attention to facilitate efficient and thorough information interchange across views. Besides, we present a geometry-aware dual branch to perform concurrent generation in both RGB and normal domains, further enhancing consistency via geometry cues. Last but not least, to address ill-shaped issues arising from inaccurate SMPL-X estimation, we propose a novel iterative refinement strategy, which progressively optimizes SMPL-X accuracy while enhancing the quality and consistency of the generated multi-views. Extensive experimental results demonstrate that our method significantly outperforms existing approaches in both novel view synthesis and subsequent 3D human reconstruction tasks.

PDF Details DOI

AAAI Conference 2025 Conference Paper

MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body Reconstruction

  • Yitao Zhu
  • Sheng Wang
  • Mengjie Xu
  • Zixu Zhuang
  • Zhixin Wang
  • Kaidong Wang
  • Han Zhang
  • Qian Wang

Multiple cameras can provide comprehensive multi-view video coverage of a person. Fusing this multi-view data is crucial for tasks like behavioral analysis, although it traditionally requires camera calibration—a process that is often complex. Moreover, previous studies have overlooked the challenges posed by self-occlusion under multiple views and the continuity of human body shape estimation. In this study, we introduce a method to reconstruct the 3D human body from multiple uncalibrated camera views. Initially, we utilize a pre-trained human body encoder to process each camera view individually, enabling the reconstruction of human body models and parameters for each view along with predicted camera positions. Rather than merely averaging the models across views, we develop a neural network trained to assign weights to individual views for all human body joints, based on the estimated distribution of joint distances from each camera. Additionally, we focus on the mesh surface of the human body for dynamic fusion, allowing for the seamless integration of facial expressions and body shape into a unified human body model. Our method has shown excellent performance in reconstructing the human body on two public datasets, advancing beyond previous work from the SMPL model to the SMPL-X model. This extension incorporates more complex hand poses and facial expressions, enhancing the detail and accuracy of the reconstructions. Crucially, it supports the flexible ad-hoc deployment of any number of cameras, offering significant potential for various applications.

PDF Details DOI

JBHI Journal 2025 Journal Article

Multi-Scale Spatio-Temporal Attention Network for Epileptic Seizure Prediction

  • Qiulei Dong
  • Han Zhang
  • Jun Xiao
  • Jiayin Sun

Epilepticseizure prediction from electroencephalogram (EEG) data has attracted much attention in the clinical diagnosis and treatment of epilepsy. Most of the existing methods in literature extract either spatial or temporal features at a single scale from EEG data, however, their learned features are generally less discriminative since the EEG data is complex and severely noisy in general, leading to low-accuracy predictions. To address this problem, we propose a Multi-scale Spatio-temporal Attention Network to learn discriminative features for seizure prediction, called MSAN, which contains a backbone module, a spatial pyramid module, and a multi-scale sequential aggregation module. The backbone module is to extract initial spatial features from the input EEG spectrograms, and the pyramid module is introduced to learn multi-scale features from the initial features. Then by taking these multi-scale features as input temporal features, the sequential aggregation module employs multiple Long Short-Term Memory(LSTM) blocks to aggregate these features. In addition, a dual-loss function is introduced to alleviate the class imbalance problem. The proposed method achieves an average sensitivity of 96. 27% with a mean false prediction rate of 0. 00/h on the CHB-MIT dataset and an average sensitivity of 93. 57% with a mean false prediction rate of 0. 044/h on the Kaggle dataset. The comparative results demonstrate that the proposed method outperforms 10 state-of-the-art epileptic seizure prediction models.

Details DOI

EAAI Journal 2025 Journal Article

Nonlinear excavation load prediction of hydraulic excavator based on gated recurrent unit neural network

  • Jinshi Chen
  • Yue Yu
  • Dongyang Huo
  • Han Zhang
  • Jingyan Wang

Excavator arms are integral to the mining and construction industries, where real-time excavation load prediction is a critical element for the advancement of automated excavation technology. This study presents a novel Physics-guided Neural Network (PGNN) designed to predict the excavating force of hydraulic cylinders used in earthwork excavation. The PGNN model synergizes the physical load model of excavators with a Gated Recurrent Unit (GRU) neural network and is optimized using the Hyperband algorithm to attain both high-speed and precise forecasting. Through comparative experiments, the study validates the PGNN model's ability to achieve optimal response speed and precision in predicting excavation loads. Additionally, the predictive performance of the PGNN model is assessed via a Hardware-in-the-loop (HIL) test, conducted within the context of an actual excavation experiment. This research introduces a promising approach that seamlessly integrates physics-based modeling with machine learning techniques, facilitating real-time load forecasting for excavators. The findings pave the way for more efficient and precise excavation processes, with implications for the broader fields of mining and construction automation.

Details DOI

NeurIPS Conference 2025 Conference Paper

S$^2$M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection

  • Jiaqi Wang
  • Zhengyu Ma
  • Xiongri Shen
  • Chenlin Zhou
  • Leilei Zhao
  • Han Zhang
  • Yi Zhong
  • Siqi Cai

Auditory attention detection (AAD) aims to decode listeners' focus in complex auditory environments from electroencephalography (EEG) recordings, which is crucial for developing neuro-steered hearing devices. Despite recent advancements, EEG-based AAD remains hindered by the absence of synergistic frameworks that can fully leverage complementary EEG features under energy-efficiency constraints. We propose ***S$^2$M-Former***, a novel ***s***piking ***s***ymmetric ***m***ixing framework to address this limitation through two key innovations: i) Presenting a spike-driven symmetric architecture composed of parallel spatial and frequency branches with mirrored modular design, leveraging biologically plausible token-channel mixers to enhance complementary learning across branches; ii) Introducing lightweight 1D token sequences to replace conventional 3D operations, reducing parameters by 14. 7$\times$. The brain-inspired spiking architecture further reduces power consumption, achieving a 5. 8$\times$ energy reduction compared to recent ANN methods, while also surpassing existing SNN baselines in terms of parameter efficiency and performance. Comprehensive experiments on three AAD benchmarks (KUL, DTU and AV-GC-AAD) across three settings (within-trial, cross-trial and cross-subject) demonstrate that S$^2$M-Former achieves comparable state-of-the-art (SOTA) decoding accuracy, making it a promising low-power, high-performance solution for AAD tasks. Code is available at https: //github. com/JackieWang9811/S2M-Former.

PDF Details

EAAI Journal 2025 Journal Article

Spatial–temporal intention representation with multi-agent reinforcement learning for unmanned surface vehicles strategies learning in asset guarding task

  • Yang Li
  • Shaorong Xie
  • Hang Yu
  • Han Zhang
  • Zhenyu Zhang
  • Xiangfeng Luo

As a typical application of artificial intelligence, autonomous and intelligent Unmanned Surface Vehicles (USVs) hold significant practical value in asset guarding tasks, as they can ensure target security while significantly reducing costs. Each USV is tasked with inferring adversary intentions based on locally observable information and proactively intercepting intruding boats to maximize asset survival time. Most current methods primarily identify and represent intentions through either prior rule matching or low-dimensional behavioral features. However, rule-based methods struggle to handle dynamically changing intents, and behavioral features can easily introduce uncertainty in intention recognition. In this paper, we propose a Spatial–Temporal Intention Representation (STIR) model that effectively conveys the dynamic intentions of various boats in asset guarding tasks, enhancing the learning efficiency of USVs’ strategies. First, utilizing local observation information from USVs, we construct an intention recognition tree that correlates intentions with a priori task background knowledge, thereby reducing uncertainty in intention recognition. Second, we develop a spatial–temporal attention network to dynamically represent intentions in both spatial and temporal dimensions, improving the USVs’ understanding of local scene dynamics. Third, we combine STIR with multi-agent reinforcement learning to train the interception strategies of USVs. In the experiments, we discuss the positive impact of STIR on adversarial strategy learning in the asset guarding task. Simulation results illustrate the advantages of our approach in terms of learning speed and strategy effectiveness.

Details DOI

NeurIPS Conference 2025 Conference Paper

The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control

  • Ruili Feng
  • Han Zhang
  • Zhilei Shu
  • Zhantao Yang
  • Longxiang Tang
  • Zhicai Wang
  • Andy Zheng
  • Jie Xiao

We present The Matrix, a foundational realistic world simulator capable of generating infinitely long 720p high-fidelity real-scene video streams with real-time, responsive control in both first- and third-person perspectives. Trained on limited supervised data from video games like Forza Horizon 5 and Cyberpunk 2077, complemented by large-scale unsupervised footage from real-world settings like Tokyo streets, The Matrix allows users to traverse diverse terrains—deserts, grasslands, water bodies, and urban landscapes—in continuous, uncut hour-long sequences. With speeds of up to 16 FPS, the system supports real-time interactivity and demonstrates zero-shot generalization, translating virtual game environments to real-world contexts where collecting continuous movement data is often infeasible. For example, The Matrix can simulate a BMW X3 driving through an office setting—an environment present in neither gaming data nor real-world sources. This approach showcases the potential of game data to advance robust world models, bridging the gap between simulations and real-world applications in scenarios with limited data.

PDF Details

IJCAI Conference 2025 Conference Paper

Towards Comprehensive and Prerequisite-Free Explainer for Graph Neural Networks

  • Han Zhang
  • Yan Wang
  • Guanfeng Liu
  • Pengfei Ding
  • Huaxiong Wang
  • Kwok-Yan Lam

To enhance the reliability and credibility of graph neural networks (GNNs) and improve the transparency of their decision logic, a new field of explainability of GNNs (XGNN) has emerged. However, two major limitations severely degrade the performance and hinder the generalizability of existing XGNN methods: they (a) fail to capture the complete decision logic of GNNs across diverse distributions in the entire dataset's sample space, and (b) impose strict prerequisites on edge properties and GNN internal accessibility. To address these limitations, we propose OPEN, a novel cOmprehensive and Prerequisite-free Explainer for GNNs. OPEN, as the first work in the literature, can infer and partition the entire dataset's sample space into multiple environments, each containing graphs that follow a distinct distribution. OPEN further learns the decision logic of GNNs across different distributions by sampling subgraphs from each environment and analyzing their predictions, thus eliminating the need for strict prerequisites. Experimental results demonstrate that OPEN captures nearly complete decision logic of GNNs, outperforms state-of-the-art methods in fidelity while maintaining similar efficiency, and enhances robustness in real-world scenarios.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks

  • Xuan Tang
  • Han Zhang
  • Yuan Cao
  • Difan Zou

Adam is a popular and widely used adaptive gradient method in deep learning, which has also received tremendous focus in theoretical research. However, most existing theoretical work primarily analyzes its full-batch version, which differs fundamentally from the stochastic variant used in practice. Unlike SGD, stochastic Adam does not converge to its full-batch counterpart even with infinitesimal learning rates. We present the first theoretical characterization of how batch size affects Adam's generalization, analyzing two-layer over-parameterized CNNs on image data. Our results reveal that while both Adam and AdamW with proper weight decay $\lambda$ converge to poor test error solutions, their mini-batch variants can achieve near-zero test error. We further prove Adam has a strictly smaller effective weight decay bound than AdamW, theoretically explaining why Adam requires more sensitive $\lambda$ tuning. Extensive experiments validate our findings, demonstrating the critical role of batch size and weight decay in Adam's generalization performance.

PDF Details

NeurIPS Conference 2024 Conference Paper

4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on RDBs

  • Minjie Wang
  • Quan Gan
  • David Wipf
  • Zhenkun Cai
  • Ning Li
  • Jianheng Tang
  • Yanlin Zhang
  • Zizhao Zhang

Given a relational database (RDB), how can we predict missing column values in some target table of interest? Although RDBs store vast amounts of rich, informative data spread across interconnected tables, the progress of predictive machine learning models as applied to such tasks arguably falls well behind advances in other domains such as computer vision or natural language processing. This deficit stems, at least in part, from the lack of established/public RDB benchmarks as needed for training and evaluation purposes. As a result, related model development thus far often defaults to tabular approaches trained on ubiquitous single-table benchmarks, or on the relational side, graph-based alternatives such as GNNs applied to a completely different set of graph datasets devoid of tabular characteristics. To more precisely target RDBs lying at the nexus of these two complementary regimes, we explore a broad class of baseline models predicated on: (i) converting multi-table datasets into graphs using various strategies equipped with efficient subsampling, while preserving tabular characteristics; and (ii) trainable models with well-matched inductive biases that output predictions based on these input subgraphs. Then, to address the dearth of suitable public benchmarks and reduce siloed comparisons, we assemble a diverse collection of (i) large-scale RDB datasets and (ii) coincident predictive tasks. From a delivery standpoint, we operationalize the above four dimensions (4D) of exploration within a unified, scalable open-source toolbox called 4DBInfer; please see https: //github. com/awslabs/multi-table-benchmark.

PDF Details DOI

JBHI Journal 2024 Journal Article

A Dual-Modality Complex-Valued Fusion Method for Predicting Side Effects of Drug-Drug Interactions Based on Graph Neural Network

  • Chuanze Kang
  • Han Zhang
  • Yanbin Yin

Predicting potential side effects of drug-drug interactions (DDIs), which is a major concern in clinical treatment, can increase therapeutic efficacy. In recent studies, how to use the multi-modal drug features is important for DDI prediction. Thus, it remains a challenge to explore an efficient computational method to achieve the feature fusion cross- and intra-modality. In this paper, we propose a dual-modality complex-valued fusion method (DMCF-DDI) for predicting the side effects of DDIs, using the form and properties of complex-vector to enhance the representations of DDIs. Firstly, DMCF-DDI applies two Graph Convolutional Network (GCN) encoders to learn molecular structure and topological features from fingerprint and knowledge graphs, respectively. Secondly, an asymmetric skip connection (ASC) uses distinct semantic-level features to construct the complex-valued drug pair representations (DPRs). Then, the complex-vector multiplication is used as a fusion operator to obtain the fine-grained DPRs. Finally, we calculate the prediction probability of DDIs by Hermitian inner product in the complex space. Compared with other methods, DMCF-DDI achieves superior performance in all situations using a fusion operator with the lowest parameter numbers. For the case study, we select six diseases and common side effects in clinical treatment to verify identification ability of our model. We also prove the advantage of ASC and complex-valued fusion can achieve to align the cross-modal fused positive DPRs through a comprehensive analysis on the phase-modulus distribution histogram of DPRs. In the end, we explain the reason for alignment based on the similarity of features and node neighbors.

Details DOI

ICLR Conference 2024 Conference Paper

A Multi-Level Framework for Accelerating Training Transformer Models

  • Longwei Zou
  • Han Zhang
  • Yangdong Deng

The fast growing capabilities of large-scale deep learning models, such as Bert, GPT and ViT, are revolutionizing the landscape of NLP, CV and many other domains. Training such models, however, poses an unprecedented demand for computing power, which incurs exponentially increasing energy cost and carbon dioxide emissions. It is thus critical to develop efficient training solutions to reduce the training costs. Motivated by a set of key observations of inter- and intra-layer similarities among feature maps and attentions that can be identified from typical training processes, we propose a multi-level framework for training acceleration. Specifically, the framework is based on three basic operators, Coalescing, De-coalescing and Interpolation, which can be orchestrated to build a multi-level training framework. The framework consists of a V-cycle training process, which progressively down- and up-scales the model size and projects the parameters between adjacent levels of models via coalescing and de-coalescing. The key idea is that a smaller model that can be trained for fast convergence and the trained parameters provides high-qualities intermediate solutions for the next level larger network. The interpolation operator is designed to break the symmetry of neurons incurred by de-coalescing for better convergence performance. Our experiments on transformer-based language models (e.g. Bert, GPT) as well as a vision model (e.g. DeiT) prove that the proposed framework reduces the computational cost by about 20% on training BERT/GPT-Base models and up to 51.6% on training the BERT-Large model while preserving the performance.

Details

JBHI Journal 2024 Journal Article

A Unified Multi-Modality Fusion Framework for Deep Spatio-Spectral-Temporal Feature Learning in Resting-State fMRI Denoising

  • Minjoo Lim
  • Keun-Soo Heo
  • Jun-Mo Kim
  • Bogyeong Kang
  • Weili Lin
  • Han Zhang
  • Dinggang Shen
  • Tae-Eui Kam

Resting-state functional magnetic resonance imaging (rs-fMRI) is a commonly used functional neuroimaging technique to investigate the functional brain networks. However, rs-fMRI data are often contaminated with noise and artifacts that adversely affect the results of rs-fMRI studies. Several machine/deep learning methods have achieved impressive performance to automatically regress the noise-related components decomposed from rs-fMRI data, which are expressed as the pairs of a spatial map and its associated time series. However, most of the previous methods individually analyze each modality of the noise-related components and simply aggregate the decision-level information (or knowledge) extracted from each modality to make a final decision. Moreover, these approaches consider only the limited modalities making it difficult to explore class-discriminative spectral information of noise-related components. To overcome these limitations, we propose a unified deep attentive spatio-spectral-temporal feature fusion framework. We first adopt a learnable wavelet transform module at the input-level of the framework to elaborately explore the spectral information in subsequent processes. We then construct a feature-level multi-modality fusion module to efficiently exchange the information from multi-modality inputs in the feature space. Finally, we design confidence-based voting strategies for decision-level fusion at the end of the framework to make a robust final decision. In our experiments, the proposed method achieved remarkable performance for noise-related component detection on various rs-fMRI datasets.

Details DOI

ICLR Conference 2024 Conference Paper

BatteryML: An Open-source Platform for Machine Learning on Battery Degradation

  • Han Zhang
  • Xiaofan Gui
  • Shun Zheng 0001
  • Ziheng Lu
  • Yuqi Li
  • Jiang Bian 0002

Battery degradation remains a pivotal concern in the energy storage domain, with machine learning emerging as a potent tool to drive forward insights and solutions. However, this intersection of electrochemical science and machine learning poses complex challenges. Machine learning experts often grapple with the intricacies of battery science, while battery researchers face hurdles in adapting intricate models tailored to specific datasets. Beyond this, a cohesive standard for battery degradation modeling, inclusive of data formats and evaluative benchmarks, is conspicuously absent. Recognizing these impediments, we present BatteryML—a one-step, all-encompass, and open-source platform designed to unify data preprocessing, feature extraction, and the implementation of both traditional and state-of-the-art models. This streamlined approach promises to enhance the practicality and efficiency of research applications. BatteryML seeks to fill this void, fostering an environment where experts from diverse specializations can collaboratively contribute, thus elevating the collective understanding and advancement of battery research.

Details

IJCAI Conference 2024 Conference Paper

Efficient Multi-view Unsupervised Feature Selection with Adaptive Structure Learning and Inference

  • Chenglong Zhang
  • Yang Fang
  • Xinyan Liang
  • Han Zhang
  • Peng Zhou
  • Xingyu Wu
  • Jie Yang
  • Bingbing Jiang

As data with diverse representations become high-dimensional, multi-view unsupervised feature selection has been an important learning paradigm. Generally, existing methods encounter the following challenges: (i) traditional solutions either concatenate different views or introduce extra parameters to weight them, affecting the performance and applicability; (ii) emphasis is typically placed on graph construction, yet disregarding the clustering information of data; (iii) exploring the similarity structure of all samples from the original features is suboptimal and extremely time-consuming. To solve this dilemma, we propose an efficient multi-view unsupervised feature selection (EMUFS) to construct bipartite graphs between samples and anchors. Specifically, a parameter-free manner is devised to collaboratively fuse the membership matrices and graphs to learn the compatible structure information across all views, naturally balancing different views. Moreover, EMUFS leverages the similarity relations of data in the feature subspace induced by l2, 0-norm to dynamically update the graph. Accordingly, the cluster information of anchors can be accurately propagated to samples via the graph structure and further guide feature selection, enhancing the quality of selected features and the computational costs in solution processes. A convergent optimization is developed to solve the formulated problem, and experiments demonstrate the effectiveness and efficiency of EMUFS.

PDF Details DOI

JBHI Journal 2024 Journal Article

Memory-Based Cross-Modal Semantic Alignment Network for Radiology Report Generation

  • Yitian Tao
  • Liyan Ma
  • Jing Yu
  • Han Zhang

Generating radiology reports automatically reduces the workload of radiologists and helps the diagnoses of specific diseases. Many existing methods take this task as modality transfer process. However, since the key information related to disease accounts for a small proportion in both image and report, it is hard for the model to learn the latent relation between the radiology image and its report, thus failing to generate fluent and accurate radiology reports. To tackle this problem, we propose a memory-based cross-modal semantic alignment model (MCSAM) following an encoder-decoder paradigm. MCSAM includes a well initialized long-term clinical memory bank to learn disease-related representations as well as prior knowledge for different modalities to retrieve and use the retrieved memory to perform feature consolidation. To ensure the semantic consistency of the retrieved cross modal prior knowledge, a cross-modal semantic alignment module (SAM) is proposed. SAM is also able to generate semantic visual feature embeddings which can be added to the decoder and benefits report generation. More importantly, to memorize the state and additional information while generating reports with the decoder, we use learnable memory tokens which can be seen as prompts. Extensive experiments demonstrate the promising performance of our proposed method which generates state-of-the-art performance on the MIMIC-CXR dataset.

Details DOI

EAAI Journal 2024 Journal Article

MMH-Net: A novel multi-modal hybrid learning network for accurate mass estimation of acoustic levitated objects

  • Yingwei Wang
  • Liangxu Jiang
  • Ziyi Chen
  • Meiqi Sun
  • Han Zhang
  • Xinbo Li

The acoustic levitation technology expands the possibility of non-contact mass measurements, avoiding contact contamination and loss, particularly for tiny objects. Current acoustic levitated object mass estimation methods focus on the mechanism models between object mass and oscillating frequency. However, the same object exhibits different oscillating behaviors in the acoustic field, especially for non-spherical objects, resulting in obvious variability and non-stationarity in oscillating signals, which limits the estimation accuracy and versatility of mass. To address these challenges, this paper proposes a novel multi-modal hybrid network (MMH-Net) method for accurate mass estimation of levitated objects. This method directly correlates the object’s actual mass with the oscillatory signal. Firstly, to reveal the inherent patterns of the oscillating signals, the variational mode decomposition (VMD) is introduced to decompose the variable oscillating signals into different frequency mode sub-windows. Secondly, a global module and a feature refinement module are constructed to extract the long-term dependencies of the original oscillatory signals and refine the local regularity patterns of the sub-modal signals, respectively. In addition, a weighted fusion strategy is adopted to integrate global and refinement features better. The specially designed dual spectrum self-attention module can assign reasonable weights to information from different sources, facilitating the extraction of representative features and effectively suppressing negative migration. Experimental results demonstrate that the proposed network model performs outstandingly in acoustic levitated object mass estimation. The proposed method is suitable to mass measurements of spherical and non-spherical objects, further promoting the development of acoustic levitation systems in research and application fields.

Details DOI

NeurIPS Conference 2024 Conference Paper

QKFormer: Hierarchical Spiking Transformer using Q-K Attention

  • Chenlin Zhou
  • Han Zhang
  • Zhaokun Zhou
  • Liutao Yu
  • Liwei Huang
  • Xiaopeng Fan
  • Li Yuan
  • Zhengyu Ma

Spiking Transformers, which integrate Spiking Neural Networks (SNNs) with Transformer architectures, have attracted significant attention due to their potential for low energy consumption and high performance. However, there remains a substantial gap in performance between SNNs and Artificial Neural Networks (ANNs). To narrow this gap, we have developed QKFormer, a direct training spiking transformer with the following features: i) Linear complexity and high energy efficiency, the novel spike-form Q-K attention module efficiently models the token or channel attention through binary vectors and enables the construction of larger models. ii) Multi-scale spiking representation, achieved by a hierarchical structure with the different numbers of tokens across blocks. iii) Spiking Patch Embedding with Deformed Shortcut (SPEDS), enhances spiking information transmission and integration, thus improving overall performance. It is shown that QKFormer achieves significantly superior performance over existing state-of-the-art SNN models on various mainstream datasets. Notably, with comparable size to Spikformer (66. 34 M, 74. 81\%), QKFormer (64. 96 M) achieves a groundbreaking top-1 accuracy of 85. 65\% on ImageNet-1k, substantially outperforming Spikformer by 10. 84\%. To our best knowledge, this is the first time that directly training SNNs have exceeded 85\% accuracy on ImageNet-1K.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Theoretical Study on Multi-objective Heuristic Search

  • Shawn Skyler
  • Shahaf Shperberg
  • Dor Atzmon
  • Ariel Felner
  • Oren Salzman
  • Shao-Hung Chan
  • Han Zhang
  • Sven Koenig

This paper provides a theoretical study on Multi-Objective Heuristic Search. We first classify states in the state space into must-expand, maybe-expand, and never-expand states and then transfer these definitions to nodes in the search tree. We then formalize a framework that generalizes A* to Multi-Objective Search. We study different ways to order nodes under this framework and their relation to traditional tie-breaking policies and provide theoretical findings. Finally, we study and empirically compare different ordering functions.

PDF Details DOI

JBHI Journal 2023 Journal Article

Contactless Sensing-Aided Respiration Signal Acquisition Using Improved Empirical Wavelet Transform for Rhythm Detection

  • Baoxian Yu
  • Yue Hou
  • Zhiqiang Pang
  • Han Zhang

Respiration is one of the most important vital signs indicating physical condition, while the signal detection is challenging due to the complex rhythm and effort in practical scenarios. In this paper, we propose a contactless sensing-aided respiration signal acquisition technique, which can adaptively extract the desired signal under time-varying respiration rhythms within a wide range. To be specific, respiration is perceived by piezoelectric ceramics sensors along with ballistocardiography and other interference in a contactless manner, and the proposed improved empirical wavelet transform (IEWT) performs spectrum division and recognition based on upper envelop and principal component criteria, respectively, to adaptively extract the respiration spectrum for signal reconstruction. For validations, we extracted respiration signals from 8 healthy individuals in lab breathing at specified rhythms from 0. 2 Hz to 0. 6 Hz as well as 38 in-patients suffering from sleep-disordered-breathing with reference of polysomnogram in practical clinic scenario. The results showed that the detected respiration rhythms perfectly fitted the ones in experimental lab dataset with a correlation coefficient of 0. 98, which validated the effectiveness of the respiration spectrum extraction of the proposed IEWT method. Besides, in practical clinical dataset, the proposed IEWT method could yield mean absolute and relative errors of respiration intervals of 0. 4 and 0. 05 seconds, respectively, achieving significant improvement in comparison with conventional ones. Meanwhile, the performance of IEWT was robust to rhythm variation, individual difference and breathing cycle detection techniques, which demonstrated the feasibility and superiority of the proposed IEWT method for practical respiration monitoring.

Details DOI

NeurIPS Conference 2023 Conference Paper

Decision Tree for Locally Private Estimation with Public Data

  • Yuheng Ma
  • Han Zhang
  • Yuchao Cai
  • Hanfang Yang

We propose conducting locally differentially private (LDP) estimation with the aid of a small amount of public data to enhance the performance of private estimation. Specifically, we introduce an efficient algorithm called Locally differentially Private Decision Tree (LPDT) for LDP regression. We first use the public data to grow a decision tree partition and then fit an estimator according to the partition privately. From a theoretical perspective, we show that LPDT is $\varepsilon$-LDP and has a mini-max optimal convergence rate under a mild assumption of similarity between public and private data, whereas the lower bound of the convergence rate of LPDT without public data is strictly slower, which implies that the public data helps to improve the convergence rates of LDP estimation. We conduct experiments on both synthetic and real-world data to demonstrate the superior performance of LPDT compared with other state-of-the-art LDP regression methods. Moreover, we show that LPDT remains effective despite considerable disparities between public and private data.

PDF Details

NeurIPS Conference 2023 Conference Paper

Diversify Your Vision Datasets with Automatic Diffusion-based Augmentation

  • Lisa Dunlap
  • Alyssa Umino
  • Han Zhang
  • Jiezhi Yang
  • Joseph E. Gonzalez
  • Trevor Darrell

Many fine-grained classification tasks, like rare animal identification, have limited training data and consequently classifiers trained on these datasets often fail to generalize to variations in the domain like changes in weather or location. As such, we explore how natural language descriptions of the domains seen in training data can be used with large vision models trained on diverse pretraining datasets to generate useful variations of the training data. We introduce ALIA (Automated Language-guided Image Augmentation), a method which utilizes large vision and language models to automatically generate natural language descriptions of a dataset's domains and augment the training data via language-guided image editing. To maintain data integrity, a model trained on the original dataset filters out minimal image edits and those which corrupt class-relevant information. The resulting dataset is visually consistent with the original training data and offers significantly enhanced diversity. We show that ALIA is able to surpasses traditional data augmentation and text-to-image generated data on fine-grained classification tasks, including cases of domain generalization and contextual bias. Code is available at https: //github. com/lisadunlap/ALIA.

PDF Details

IJCAI Conference 2023 Conference Paper

Heuristic-Search Approaches for the Multi-Objective Shortest-Path Problem: Progress and Research Opportunities

  • Oren Salzman
  • Ariel Felner
  • Carlos Hernández
  • Han Zhang
  • Shao-Hung Chan
  • Sven Koenig

In the multi-objective shortest-path problem we are interested in computing a path, or a set of paths that simultaneously balance multiple cost functions. This problem is important for a diverse range of applications such as transporting hazardous materials considering travel distance and risk. This family of problems is not new with results dating back to the 1970's. Nevertheless, the significant progress made in the field of heuristic search resulted in a new and growing interest in the sub-field of multi-objective search. Consequently, in this paper we review the fundamental problems and techniques common to most algorithms and provide a general overview of the field. We then continue to describe recent work with an emphasis on new challenges that emerged and the resulting research opportunities.

PDF Details DOI

JBHI Journal 2023 Journal Article

Individualized Assessment of Brain Aβ Deposition With fMRI Using Deep Learning

  • Chaolin Li
  • Mianxin Liu
  • Jing Xia
  • Lang Mei
  • Qing Yang
  • Feng Shi
  • Han Zhang
  • Dinggang Shen

PET-based Alzheimer's disease (AD) assessment has many limitations in large-scale screening. Non-invasive techniques such as resting-state functional magnetic resonance imaging (rs-fMRI) have been proven valuable in early AD diagnosis. This study investigated feasibility of using rs-fMRI, especially functional connectivity (FC), for individualized assessment of brain amyloid-β deposition derived from PET. We designed a graph convolutional networks (GCNs) and random forest (RF) based integrated framework for using rs-fMRI-derived multi-level FC networks to predict amyloid-β PET patterns with the OASIS-3 (N = 258) and ADNI-2 (N = 291) datasets. Our method achieved satisfactory accuracy not only in Aβ-PET grade classification (for negative, intermediate, and positive grades, with accuracy in the three-class classification as 62. 8% and 64. 3% on two datasets, respectively), but also in prediction of whole-brain region-level Aβ-PET standard uptake value ratios (SUVRs) (with the mean square errors as 0. 039 and 0. 074 for two datasets, respectively). Model interpretability examination also revealed the contributive role of the limbic network. This study demonstrated high feasibility and reproducibility of using low-cost, more accessible magnetic resonance imaging (MRI) to approximate PET-based diagnosis.

Details DOI

IJCAI Conference 2023 Conference Paper

Multi-objective Search via Lazy and Efficient Dominance Checks

  • Carlos Hernández
  • William Yeoh
  • Jorge A. Baier
  • Ariel Felner
  • Oren Salzman
  • Han Zhang
  • Shao-Hung Chan
  • Sven Koenig

Multi-objective search can be used to model many real-world problems that require finding Pareto optimal paths from a specified start state to a specified goal state, while considering different costmetrics such as distance, time, and fuel. The performance of multi-objective search can be improved by making dominance checking—an operation necessary to determine whether or not a path dominates another—more efficient. This was shown in practice by BOA*, a state-of-the-art bi-objective search algorithm, which outperforms previously existing bi-objective search algorithms in part because it adopts a lazy approach towards dominance checking. EMOA*, a recent multi-objective search algorithm, generalizes BOA* to more-than-two objectives using AVL trees for dominance checking. In this paper, we first propose Linear-Time Multi-Objective A* (LTMOA*), an multi-objective search algorithm that implements a more efficient dominance checking than EMOA* using simple data structures like arrays. We then propose an even lazier approach towards dominance checking, and the resulting algorithm, LazyLTMOA*, distinguishes from EMOA* and LTMOA* by removing the dominance checking during node generation. Our experimental results show that LazyLTMOA* outperforms EMOA* by up to an order of magnitude in terms of runtime.

PDF Details DOI

JBHI Journal 2023 Journal Article

Outcome Prediction of Unconscious Patients Based on Weighted Sparse Brain Network Construction

  • Renping Yu
  • Han Zhang
  • Xuehai Wu
  • Xuan Fei
  • Qing Yang
  • Zhiwei Ma
  • Zengxin Qi
  • Di Zang

It is quite challenging to establish a prompt and reliable prognosis assessment for acquired brain injury (ABI) patients with persistent severe disorders of consciousness (DOC) like unconscious comatose and unresponsive wakefulness syndrome (a. k. a. , vegetative state). Recent advances in brain functional imaging and functional net-work analysis have demonstrated its potential in determining the consciousness level and prognostic outcome for ABI patients with DOC. However, the diagnostic and prognostic usefulness of the whole-brain functional connectome based on advanced machine learning techniques has not been fully evaluated. The first aim of this study is to predict the outcome of individual unconscious ABI patients during a three-month follow-up. The second aim is to conduct precise individualized differentiation among different consciousness levels for exploring the neurobiological mechanisms underlying DOC. Based on resting-state fMRI, we construct large-scale functional networks by using a weighted sparse model, which ensures sparsity and interpretability by preserving strong functional connections. The functional connection strengths are exploited as features for outcome prediction and consciousness level differentiation. We achieve significantly improved consciousness level classification (accuracy: 84. 78%) and recovery outcome prediction (accuracy: 89. 74%) compared to other network construction methods. More importantly, we reveal the contributive connections across the entire brain in both tasks. These connections could serve as the potential biomarkers for better understanding of consciousness and further provide new insight into the development of diagnostic, prognostic, and effective therapeutic guidelines for ABI patients with DOC.

Details DOI

AIJ Journal 2023 Journal Article

Simple and efficient bi-objective search algorithms via fast dominance checks

  • Carlos Hernández
  • William Yeoh
  • Jorge A. Baier
  • Han Zhang
  • Luis Suazo
  • Sven Koenig
  • Oren Salzman

Many interesting search problems can be formulated as bi-objective search problems, that is, search problems where two kinds of costs have to be minimized, for example, travel distance and time for transportation problems. Instead of looking for a single optimal path, we compute a Pareto-optimal frontier in bi-objective search, which is a set of paths in which no two paths dominate each other. Bi-objective search algorithms perform dominance checks each time a new path is discovered. Thus, the efficiency of these checks is key to performance. In this article, we propose algorithms for two kinds of bi-objective search problems. First, we consider the problem of computing the Pareto-optimal frontier of the paths that connect a given start state with a given goal state. We propose Bi-Objective A* (BOA*), a heuristic search algorithm based on A*, for this problem. Second, we consider the problem of computing one Pareto-optimal frontier for each state s of the search graph, which contains the paths that connect a given start state with s. We propose Bi-Objective Dijkstra (BOD), which is based on BOA*, for this problem. A common feature of BOA* and BOD is that all dominance checks are performed in constant time, unlike the dominance checks of previous algorithms. We show in our experimental evaluation that both BOA* and BOD are substantially faster than state-of-the-art bi-objective search algorithms.

Details DOI

NeurIPS Conference 2023 Conference Paper

StoryBench: A Multifaceted Benchmark for Continuous Story Visualization

  • Emanuele Bugliarello
  • H. Hernan Moraldo
  • Ruben Villegas
  • Mohammad Babaeizadeh
  • Mohammad Taghi Saffar
  • Han Zhang
  • Dumitru Erhan
  • Vittorio Ferrari

Generating video stories from text prompts is a complex task. In addition to having high visual quality, videos need to realistically adhere to a sequence of text prompts whilst being consistent throughout the frames. Creating a benchmark for video generation requires data annotated over time, which contrasts with the single caption used often in video datasets. To fill this gap, we collect comprehensive human annotations on three existing datasets, and introduce StoryBench: a new, challenging multi-task benchmark to reliably evaluate forthcoming text-to-video models. Our benchmark includes three video generation tasks of increasing difficulty: action execution, where the next action must be generated starting from a conditioning video; story continuation, where a sequence of actions must be executed starting from a conditioning video; and story generation, where a video must be generated from only text prompts. We evaluate small yet strong text-to-video baselines, and show the benefits of training on story-like data algorithmically generated from existing video captions. Finally, we establish guidelines for human evaluation of video stories, and reaffirm the need of better automatic metrics for video generation. StoryBench aims at encouraging future research efforts in this exciting new area.

PDF Details

ICLR Conference 2023 Conference Paper

Using Language to Extend to Unseen Domains

  • Lisa Dunlap
  • Clara Mohri
  • Devin Guillory
  • Han Zhang
  • Trevor Darrell
  • Joseph E. Gonzalez
  • Aditi Raghunathan
  • Anna Rohrbach

It is expensive to collect training data for every possible domain that a vision model may encounter when deployed. We instead consider how simply $\textit{verbalizing}$ the training domain (e.g.``photos of birds'') as well as domains we want to extend to but do not have data for (e.g.``paintings of birds'') can improve robustness. Using a multimodal model with a joint image and language embedding space, our method $\textit{LADS}$ learns a transformation of the image embeddings from the source domain to each target domain, while preserving task relevant information. Without using any images from the target domain, we show that over the $\textit{extended}$ domain containing both source and target, $\textit{LADS}$ outperforms standard fine-tuning and ensemble approaches over a suite of 4 benchmarks targeting domain adaptation and dataset bias.

Details

YNICL Journal 2022 Journal Article

Alterations of dynamic redundancy of functional brain subnetworks in Alzheimer’s disease and major depression disorders

  • Maryam Ghanbari
  • Mayssa Soussia
  • Weixiong Jiang
  • Dongming Wei
  • Pew-Thian Yap
  • Dinggang Shen
  • Han Zhang

The human brain is not only efficiently but also "redundantly" connected. The redundancy design could help the brain maintain resilience to disease attacks. This paper explores subnetwork-level redundancy dynamics and the potential of such metrics in disease studies. As such, we looked into specific functional subnetworks, including those associated with high-level functions. We investigated how the subnetwork redundancy dynamics change along with Alzheimer's disease (AD) progression and with major depressive disorder (MDD), two major disorders that could share similar subnetwork alterations. We found an increased dynamic redundancy of the subcortical-cerebellum subnetwork and its connections to other high-order subnetworks in the mild cognitive impairment (MCI) and AD compared to the normal control (NC). With gained spatial specificity, we found such a redundancy index was sensitive to disease symptoms and could act as a protective mechanism to prevent the collapse of the brain network and functions. The dynamic redundancy of the medial frontal subnetwork and its connections to the frontoparietal subnetwork was also found decreased in MDD compared to NC. The spatial specificity of the redundancy dynamics changes may provide essential knowledge for a better understanding of shared neural substrates in AD and MDD.

Details DOI

YNIMG Journal 2022 Journal Article

Deep attentive spatio-temporal feature learning for automatic resting-state fMRI denoising

  • Keun-Soo Heo
  • Dong-Hee Shin
  • Sheng-Che Hung
  • Weili Lin
  • Han Zhang
  • Dinggang Shen
  • Tae-Eui Kam

Resting-state functional magnetic resonance imaging (rs-fMRI) is a non-invasive functional neuroimaging modality that has been widely used to investigate functional connectomes in the brain. Since noise and artifacts generated by non-neuronal physiological activities are predominant in raw rs-fMRI data, effective noise removal is one of the most important preprocessing steps prior to any subsequent analysis. For rs-fMRI denoising, a common trend is to decompose rs-fMRI data into multiple components and then regress out noise-related components. Therefore, various machine learning techniques have been used in such analyses with predefined procedures and manually engineered features. However, the lack of a universal definition of a noise-related source or artifact complicates manual feature engineering. Manual feature selection can result in the failure to capture unknown types of noise. Furthermore, the possibility that the hand-crafted features will only work for the broader population (e.g., healthy adults) but not for "outliers" (e.g., infants or subjects that belong to a disease cohort) is quite high. In practice, we have limited knowledge of which features should be extracted; thus, multi-classifier assembly must be implemented to improve performance, although this process is quite time-consuming. However, in real rs-fMRI applications, fast and accurate automatic identification of noise-related components on different datasets is critical. To solve this problem, we propose a novel, automatic, and end-to-end deep learning framework dedicated to noise-related component identification via a faster and more effective multi-layer feature extraction strategy that learns deeply embedded spatio-temporal features of the components. In this study, we achieved remarkable performance on various rs-fMRI datasets, including multiple adult rs-fMRI datasets from different rs-fMRI studies and an infant rs-fMRI dataset, which is quite heterogeneous and differs from that of adults. Our proposed framework also dramatically increases the noise detection speed owing to its inherent ability for deep learning (< 1s for single-component classification). It can be easily integrated into any preprocessing pipeline, even those that do not use standard procedures but depend on alternative toolboxes.

Details DOI

JBHI Journal 2022 Journal Article

Divergent and Convergent Imaging Markers Between Bipolar and Unipolar Depression Based on Machine Learning

  • Huifeng Zhang
  • Zhen Zhou
  • Lei Ding
  • Chuangxin Wu
  • Meihui Qiu
  • Yueqi Huang
  • Feng Jin
  • Ting Shen

Distinguishing bipolar depression (BD) from unipolar depression (UD) based on symptoms only is challenging. Brain functional connectivity (FC), especially dynamic FC, has emerged as a promising approach to identify possible imaging markers for differentiating BD from UD. However, most of such studies utilized conventional FC and group-level statistical comparisons, which may not be sensitive enough to quantify subtle changes in the FC dynamics between BD and UD. In this paper, we present a more effective individualized differentiation model based on machine learning and the whole-brain “high-order functional connectivity (HOFC)” network. The HOFC, capturing temporal synchronization among the dynamic FC time series, a more complex “chronnectome” metric compared to the conventional FC, was used to classify 52 BD, 73 UD, and 76 healthycontrols (HC). We achieved a satisfactory accuracy (70. 40%) in BD vs. UD differentiation. The resultant contributing features revealed the involvement of the coordinated flexible interactions among sensory (e. g. , olfaction, vision, and audition), motor, and cognitive systems. Despite sharing common chronnectome of cognitive and affective impairments, BD and UD also demonstrated unique dynamic FC synchronization patterns. UD is more associated with abnormal visual-somatomotor inter-network connections, while BD is more related to impaired ventral attention-frontoparietal inter-network connections. Moreover, we found that the illness duration modulated the BD vs. UD separation, with the differentiation performance hampered by the secondary disease effects. Our findings suggest that BD and UD may have divergent and convergent neural substrates, which further expand our knowledge of the two different mental disorders.

Details DOI

NeurIPS Conference 2022 Conference Paper

GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization

  • Xuhai Xu
  • Han Zhang
  • Yasaman Sefidgar
  • Yiyi Ren
  • Xin Liu
  • Woosuk Seo
  • Jennifer Brown
  • Kevin Kuehn

Recent research has demonstrated the capability of behavior signals captured by smartphones and wearables for longitudinal behavior modeling. However, there is a lack of a comprehensive public dataset that serves as an open testbed for fair comparison among algorithms. Moreover, prior studies mainly evaluate algorithms using data from a single population within a short period, without measuring the cross-dataset generalizability of these algorithms. We present the first multi-year passive sensing datasets, containing over 700 user-years and 497 unique users’ data collected from mobile and wearable sensors, together with a wide range of well-being metrics. Our datasets can support multiple cross-dataset evaluations of behavior modeling algorithms’ generalizability across different users and years. As a starting point, we provide the benchmark results of 18 algorithms on the task of depression detection. Our results indicate that both prior depression detection algorithms and domain generalization techniques show potential but need further research to achieve adequate cross-dataset generalizability. We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.

PDF Details

AAMAS Conference 2022 Conference Paper

Multi-Agent Path Finding for Precedence-Constrained Goal Sequences

  • Han Zhang
  • Jingkai Chen
  • Jiaoyang Li
  • Brian C. Williams
  • Sven Koenig

With the rising demand for deploying robot teams in autonomous warehouses and factories, the Multi-Agent Path Finding (MAPF) problem has drawn more and more attention. The classical MAPF problem and most of its variants focus on navigating agent teams to goal locations while avoiding collisions. However, they do not take into account any precedence constraints that agents should respect when reaching their goal locations. Planning with precedence constraints is important for real-world multi-agent systems. For example, a mobile robot can only pick up a package at a station after it has been delivered by another robot. In this paper, we study the Multi-Agent Path Finding with Precedence Constraints (MAPF-PC) problem, in which agents need to visit sequences of goal locations while satisfying precedence constraints between the goal locations. We propose two algorithms for solving this problem systematically: Conflict-Based Search with Precedence Constraints (CBS-PC) is complete and optimal, and Priority-Based Search with Precedence Constraints (PBS-PC) is incomplete but more efficient in finding near-optimal solutions in practice. Our experimental results show that CBS-PC scales to dozens of agents and hundreds of goal locations and precedence constraints, and PBS-PC scales to hundreds of agents, around one thousand goal locations, and hundreds of precedence constraints.

PDF

AIJ Journal 2022 Journal Article

Multi-agent path finding with mutex propagation

  • Han Zhang
  • Jiaoyang Li
  • Pavel Surynek
  • T.K. Satish Kumar
  • Sven Koenig

Mutex propagation is a form of efficient constraint propagation popularly used in AI planning to tightly approximate the reachable states from a given state. We utilize this idea in the context of Multi-Agent Path Finding (MAPF). When adapted to MAPF, mutex propagation provides stronger constraints for conflict resolution in CBS, a popular optimal search-based MAPF algorithm, as well as in MDD-SAT, an optimal satisfiability-based MAPF algorithm. Mutex propagation provides CBS with the ability to break symmetries in MAPF and provides MDD-SAT with the ability to make stronger inferences than unit propagation. While existing work identifies a limited form of symmetries and requires the manual design of symmetry-breaking constraints, mutex propagation is more general and allows for the automated design of symmetry-breaking constraints. Our experimental results show that CBS with mutex propagation is capable of outperforming CBSH-RCT, a state-of-the-art variant of CBS, with respect to the success rate. We also show that MDD-SAT with mutex propagation often performs better than MDD-SAT with respect to the success rate.

Details DOI

AAAI Conference 2022 Conference Paper

Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding

  • Zizhao Zhang
  • Han Zhang
  • Long Zhao
  • Ting Chen
  • Sercan Ö. Arik
  • Tomas Pfister

Hierarchical structures are popular in recent vision transformers, however, they require sophisticated designs and massive datasets to work well. In this paper, we explore the idea of nesting basic local transformers on non-overlapping image blocks and aggregating them in a hierarchical way. We find that the block aggregation function plays a critical role in enabling cross-block non-local information communication. This observation leads us to design a simplified architecture that requires minor code changes upon the original vision transformer. The benefits of the proposed judiciouslyselected design are threefold: (1) NesT converges faster and requires much less training data to achieve good generalization on both ImageNet and small datasets like CIFAR; (2) when extending our key ideas to image generation, NesT leads to a strong decoder that is 8 times faster than previous transformer-based generators; and (3) we show that decoupling the feature learning and abstraction processes via this nested hierarchy in our design enables constructing a novel method (named GradCAT) for visually interpreting the learned model. Source code is available https: //github. com/ google-research/nested-transformer.

PDF Details

AAAI Conference 2022 Conference Paper

Powering Finetuning in Few-Shot Learning: Domain-Agnostic Bias Reduction with Selected Sampling

  • Ran Tao
  • Han Zhang
  • Yutong Zheng
  • Marios Savvides

In recent works, utilizing a deep network trained on metatraining set serves as a strong baseline in few-shot learning. In this paper, we move forward to refine novel-class features by finetuning a trained deep network. Finetuning is designed to focus on reducing biases in novel-class feature distributions, which we define as two aspects: class-agnostic and class-specific biases. Class-agnostic bias is defined as the distribution shifting introduced by domain difference, which we propose Distribution Calibration Module(DCM) to reduce. DCM owes good property of eliminating domain difference and fast feature adaptation during optimization. Classspecific bias is defined as the biased estimation using a few samples in novel classes, which we propose Selected Sampling(SS) to reduce. Without inferring the actual class distribution, SS is designed by running sampling using proposal distributions around support-set samples. By powering finetuning with DCM and SS, we achieve state-of-the-art results on Meta-Dataset with consistent performance boosts over ten datasets from different domains. We believe our simple yet effective method demonstrates its possibility to be applied on practical few-shot applications.

PDF Details

TMLR Journal 2022 Journal Article

Scaling Autoregressive Models for Content-Rich Text-to-Image Generation

  • Jiahui Yu
  • Yuanzhong Xu
  • Jing Yu Koh
  • Thang Luong
  • Gunjan Baid
  • Zirui Wang
  • Vijay Vasudevan
  • Alexander Ku

We present the Pathways Autoregressive Text-to-Image (Parti) model, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge. Parti treats text-to-image generation as a sequence-to-sequence modeling problem, akin to machine translation, with sequences of image tokens as the target outputs rather than text tokens in another language. This strategy can naturally tap into the rich body of prior work on large language models, which have seen continued advances in capabilities and performance through scaling data and model sizes. Our approach is simple: First, Parti uses a Transformer-based image tokenizer, ViT-VQGAN, to encode images as sequences of discrete tokens. Second, we achieve consistent quality improvements by scaling the encoder-decoder Transformer model up to 20B parameters, with a new state-of-the-art zero-shot FID score of 7.23 and finetuned FID score of 3.22 on MS-COCO. Our detailed analysis on Localized Narratives as well as PartiPrompts (P2), a new holistic benchmark of over 1600 English prompts, demonstrate the effectiveness of Parti across a wide variety of categories and difficulty aspects. We also explore and highlight limitations of our models in order to define and exemplify key areas of focus for further improvements.

PDF Details

JBHI Journal 2021 Journal Article

Altered Time-Frequency Feature in Default Mode Network of Autism Based on Improved Hilbert-Huang Transform

  • Han Zhang
  • Rui Li
  • Xiaotong Wen
  • Qing Li
  • Xia Wu

Autism spectrum disorder (ASD) is a pervasive neurodevelopmental disorder characterized by restricted interests and repetitive behaviors. Non-invasive measurements of brain activity with functional magnetic resonance imaging (fMRI) have demonstrated that the abnormality in the default mode network (DMN) is a crucial neural basis of ASD, but the time-frequency feature of the DMN has not yet been revealed. Hilbert-Huang transform (HHT) is conducive to feature extraction of biomedical signals and has recently been suggested as an effective way to explore the time-frequency feature of the brain mechanism. In this study, the resting-state fMRI dataset of 105 subjects including 59 ASD participants and 46 healthy control (HC) participants were involved in the time-frequency clustering analysis based on improved HHT and modified k-means clustering with label-replacement. Compared with HC, ASD selectively showed enhanced Hilbert weight frequency (HWF) in high frequency bands in crucial regions of the DMN, including the medial prefrontal cortex (MPFC), posterior cingulate cortex (PCC) and anterior cingulate cortex (ACC). Time-frequency clustering analysis revealed altered DMN organization in ASD. In the posterior DMN, the PCC and bilateral precuneus were separated for HC but clustered for ASD; in the anterior DMN, the clusters of ACC, dorsal MPFC, and ventral MPFC were relatively scattered for ASD. This study paves a promising way to uncover the alteration in the DMN and identifies a potential neuroimaging biomarker of diagnostic reference for ASD.

Details DOI

YNIMG Journal 2021 Journal Article

Common functional brain networks between attention deficit and disruptive behaviors in youth

  • Ting Yat Wong
  • Han Zhang
  • Tonya White
  • Liyuan Xu
  • Anqi Qiu

Attention deficits (AD) and disruptive behavior (DB) are highly comorbid youth externalizing behaviors. This study aimed to study reliable functional brain networks shared by AD and DB in youth aged from 8 to 21 years from the Philadelphia Neurodevelopmental Cohort (PNC). The PNC study assessed AD and DB behaviors via Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS). This study employed sparse canonical correlation analysis (SCCA) to examine the correlation of AD and DB behaviors with resting-state functional connectivity maps of the brain regions identified via activation likelihood estimation (ALE) meta-analyses on attention deficit/hyperactivity disorder (ADHD) and DB disorder (DBD). Our meta-analyses identified that the middle cingulate cortex, pre-supplementary motor area (pre-SMA), and striatum had a great consensus in existing ADHD studies and the amygdala and inferior parietal lobule were consistently found in existing DBD studies. Our SCCA analysis revealed that the AD and DB behavioral items relevant to inattention and delinquency were correlated with the functional connectivity of the pre-SMA with the ventral attentional and frontoparietal networks (FPN), and the striatum with the default mode (DMN) and dorsal attentional networks. The AD and DB behavioral items relevant to inattention and irritability were associated with the functional connectivity between the amygdala and the DMN and FPN. Our findings suggest that the functional organization of the ADHD- and DBD-related brain regions provides insights on the shared neural basis in AD and DB.

Details DOI

AAAI Conference 2021 Conference Paper

Improved Consistency Regularization for GANs

  • Zhengli Zhao
  • Sameer Singh
  • Honglak Lee
  • Zizhao Zhang
  • Augustus Odena
  • Han Zhang

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then propose several modifications to the consistency regularization procedure designed to improve its performance. We carry out extensive experiments quantifying the benefit of our improvements. For unconditional image synthesis on CIFAR-10 and CelebA, our modifications yield the best known FID scores on various GAN architectures. For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11. 48 to 9. 21. Finally, on ImageNet-2012, we apply our technique to the original Big- GAN model and improve the FID from 6. 66 to 5. 38, which is the best score at that model size.

PDF Details

NeurIPS Conference 2021 Conference Paper

Improved Transformer for High-Resolution GANs

  • Long Zhao
  • Zizhao Zhang
  • Ting Chen
  • Dimitris Metaxas
  • Han Zhang

Attention-based models, exemplified by the Transformer, can effectively model long range dependency, but suffer from the quadratic complexity of self-attention operation, making them difficult to be adopted for high-resolution image generation based on Generative Adversarial Networks (GANs). In this paper, we introduce two key ingredients to Transformer to address this challenge. First, in low-resolution stages of the generative process, standard global self-attention is replaced with the proposed multi-axis blocked self-attention which allows efficient mixing of local and global attention. Second, in high-resolution stages, we drop self-attention while only keeping multi-layer perceptrons reminiscent of the implicit neural function. To further improve the performance, we introduce an additional self-modulation component based on cross-attention. The resulting model, denoted as HiT, has a nearly linear computational complexity with respect to the image size and thus directly scales to synthesizing high definition images. We show in the experiments that the proposed HiT achieves state-of-the-art FID scores of 30. 83 and 2. 95 on unconditional ImageNet $128 \times 128$ and FFHQ $256 \times 256$, respectively, with a reasonable throughput. We believe the proposed HiT is an important milestone for generators in GANs which are completely free of convolutions. Our code is made publicly available at https: //github. com/google-research/hit-gan.

PDF Details

YNIMG Journal 2021 Journal Article

Integrated structural and functional atlases of Asian children from infancy to childhood

  • Jingwen Zhu
  • Han Zhang
  • Yap-Seng Chong
  • Lynette P. Shek
  • Peter D. Gluckman
  • Michael J. Meaney
  • Marielle V. Fortier
  • Anqi Qiu

The developing brain grows exponentially in the first few years of life. There is a need to have age-appropriate brain atlases that coherently characterize the geometry of the cerebral cortex, white matter tracts, and functional organization. This study employed multi-modal brain images of an Asian cohort and constructed brain structural and functional atlases for 6-month-old infants, 4.5-, 6-, and 7.5-year-old children. We exploited large deformation diffeomorphic metric mapping and probabilistic atlas generation approaches to integrate structural MRI and diffusion weighted images (DWIs) and to create the atlas where white matter tracts well fit into the cortical folding pattern. Based on this structural atlas, we then employed spectral clustering to parcellate the brain into functional networks from resting-state fMRI (rs-fMRI). Our results provided the atlas that characterizes the cortical folding geometry, subcortical regions, deep white matter tracts, as well as functional networks in a stereotaxic coordinate space for the four different age groups. The functional networks consisting of the primary cortex were well established in infancy and remained stable to childhood, while specific higher-order functional networks showed specific patterns of hemispherical, subcortical-cerebellar, and cortical-cortical integration and segregation from infancy to childhood. Our multi-modal fusion analysis demonstrated the use of the integrated structural and functional atlas for understanding coherent patterns of brain anatomical and functional development during childhood. Hence, our atlases can be potentially used to study coherent patterns of brain anatomical and functional development.

Details DOI

YNICL Journal 2021 Journal Article

Multiscale neural modeling of resting-state fMRI reveals executive-limbic malfunction as a core mechanism in major depressive disorder

  • Guoshi Li
  • Yujie Liu
  • Yanting Zheng
  • Ye Wu
  • Danian Li
  • Xinyu Liang
  • Yaoping Chen
  • Ying Cui

Major depressive disorder (MDD) represents a grand challenge to human health and society, but the underlying pathophysiological mechanisms remain elusive. Previous neuroimaging studies have suggested that MDD is associated with abnormal interactions and dynamics in two major neural systems including the default mode - salience (DMN-SAL) network and the executive - limbic (EXE-LIM) network, but it is not clear which network plays a central role and which network plays a subordinate role in MDD pathophysiology. To address this question, we refined a newly developed Multiscale Neural Model Inversion (MNMI) framework and applied it to test whether MDD is more affected by impaired circuit interactions in the DMN-SAL network or the EXE-LIM network. The model estimates the directed connection strengths between different neural populations both within and between brain regions based on resting-state fMRI data collected from normal healthy subjects and patients with MDD. Results show that MDD is primarily characterized by abnormal circuit interactions in the EXE-LIM network rather than the DMN-SAL network. Specifically, we observe reduced frontoparietal effective connectivity that potentially contributes to hypoactivity in the dorsolateral prefrontal cortex (dlPFC), and decreased intrinsic inhibition combined with increased excitation from the superior parietal cortex (SPC) that potentially lead to amygdala hyperactivity, together resulting in activation imbalance in the PFC-amygdala circuit that pervades in MDD. Moreover, the model reveals reduced PFC-to-hippocampus excitation but decreased SPC-to-thalamus inhibition in MDD population that potentially lead to hypoactivity in the hippocampus and hyperactivity in the thalamus, consistent with previous experimental data. Overall, our findings provide strong support for the long-standing limbic-cortical dysregulation model in major depression but also offer novel insights into the multiscale pathophysiology of this debilitating disease.

Details DOI

YNIMG Journal 2021 Journal Article

Spatio-temporal correlates of gene expression and cortical morphology across lifespan and aging

  • Anqi Qiu
  • Han Zhang
  • Brian K. Kennedy
  • Annie Lee

Evidence from neuroimaging and genetic studies supports the concept that brain aging mirrors development. However, it is unclear whether mechanisms linking brain development and aging provide new insights to delay aging and potentially reverse it. This study determined biological mechanisms and phenotypic traits underpinning brain alterations across the lifespan and in aging by examining spatio-temporal correlations between gene expression and cortical volumes using datasets d with the age range from 2 to 82 years. We revealed that a large proportion of genes whose expression was associated with cortical volumes across the lifespan were in astrocytes. These genes, which showed up-regulation during development and down-regulation during aging, contributed to fundamental homeostatic functions of astrocytes. Included among these genes were those encoding components of cAMP, Ras, and retrograde endocannabinoid signaling pathways. Genes associated with cortical volumes in the same data aged above 55 years were also enriched for the sphingolipid, renin-angiotensin system (RAS), proteasome, and TGF-β signaling pathway, which is linked to senescence-associated secretory phenotypes. Neuroticism, drinking, and smoking were the common phenotypic traits in the lifespan and aging, while memory was the unique phenotype associated with aging. These findings provide biological mechanisms mirroring development and aging as well as unique to aging.

Details DOI

ICML Conference 2020 Conference Paper

Approximation Capabilities of Neural ODEs and Invertible Residual Networks

  • Han Zhang
  • Xi Gao
  • Jacob Unterman
  • Tom Arodz

Recent interest in invertible models and normalizing flows has resulted in new architectures that ensure invertibility of the network model. Neural ODEs and i-ResNets are two recent techniques for constructing models that are invertible, but it is unclear if they can be used to approximate any continuous invertible mapping. Here, we show that out of the box, both of these architectures are limited in their approximation capabilities. We then show how to overcome this limitation: we prove that any homeomorphism on a $p$-dimensional Euclidean space can be approximated by a Neural ODE or an i-ResNet operating on a $2p$-dimensional Euclidean space. We conclude by showing that capping a Neural ODE or an i-ResNet with a single linear layer is sufficient to turn the model into a universal approximator for non-invertible continuous functions.

Details

AAAI Conference 2020 Conference Paper

Auto-GAN: Self-Supervised Collaborative Learning for Medical Image Synthesis

  • Bing Cao
  • Han Zhang
  • Nannan Wang
  • Xinbo Gao
  • Dinggang Shen

In various clinical scenarios, medical image is crucial in disease diagnosis and treatment. Different modalities of medical images provide complementary information and jointly helps doctors to make accurate clinical decision. However, due to clinical and practical restrictions, certain imaging modalities may be unavailable nor complete. To impute missing data with adequate clinical accuracy, here we propose a framework called self-supervised collaborative learning to synthesize missing modality for medical images. The proposed method comprehensively utilize all available information correlated to the target modality from multi-source-modality images to generate any missing modality in a single model. Different from the existing methods, we introduce an autoencoder network as a novel, self-supervised constraint, which provides target-modality-specific information to guide generator training. In addition, we design a modality mask vector as the target modality label. With experiments on multiple medical image databases, we demonstrate a great generalization ability as well as specialty of our method compared with other state-of-the-arts.

PDF Details

IJCAI Conference 2020 Conference Paper

ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation

  • Dongling Xiao
  • Han Zhang
  • Yukun Li
  • Yu Sun
  • Hao Tian
  • Hua Wu
  • Haifeng Wang

Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks. To address this issue, we propose an enhanced multi-flow sequence to sequence pre-training and fine-tuning framework named ERNIE-GEN, which bridges the discrepancy between training and inference with an infilling generation mechanism and a noise-aware generation method. To make generation closer to human writing patterns, this framework introduces a span-by-span generation flow that trains the model to predict semantically-complete spans consecutively rather than predicting word by word. Unlike existing pre-training methods, ERNIE-GEN incorporates multi-granularity target sampling to construct pre-training data, which enhances the correlation between encoder and decoder. Experimental results demonstrate that ERNIE-GEN achieves state-of-the-art results with a much smaller amount of pre-training data and parameters on a range of language generation tasks, including abstractive summarization (Gigaword and CNN/DailyMail), question generation (SQuAD), dialogue generation (Persona-Chat) and generative question answering (CoQA). The source codes and pre-trained models have been released at https: //github. com/PaddlePaddle/ERNIE/ernie-gen.

PDF Details DOI

NeurIPS Conference 2020 Conference Paper

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

  • Kihyuk Sohn
  • David Berthelot
  • Nicholas Carlini
  • Zizhao Zhang
  • Han Zhang
  • Colin A. Raffel
  • Ekin Dogus Cubuk
  • Alexey Kurakin

Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model’s performance. This domain has seen fast progress recently, at the cost of requiring more complex methods. In this paper we propose FixMatch, an algorithm that is a significant simplification of existing SSL methods. FixMatch first generates pseudo-labels using the model’s predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94. 93% accuracy on CIFAR-10 with 250 labels and 88. 61% accuracy with 40 – just 4 labels per class. We carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch’s success. The code is available at https: //github. com/google-research/fixmatch.

PDF Details

IJCAI Conference 2020 Conference Paper

Semi-supervised Clustering via Pairwise Constrained Optimal Graph

  • Feiping Nie
  • Han Zhang
  • Rong Wang
  • Xuelong Li

In this paper, we present a technique of definitely addressing the pairwise constraints in the semi-supervised clustering. Our method contributes to formulating the cannot-link relations and propagating them over the affinity graph flexibly. The pairwise constrained instances are provably guaranteed to be in the same or different connected components of the graph. Combined with the Laplacian rank constraint, the proposed model learns a Pairwise Constrained structured Optimal Graph (PCOG), from which the specified c clusters supporting the known pairwise constraints are directly obtained. An efficient algorithm invoked by the label propagation is designed to solve the formulation. Additionally, we also provide a compact criterion to acquire the key pairwise constraints for prompting the semi-supervised graph clustering. Substantial experimental results show that the proposed method achieves the significant improvements by using a few prior pairwise constraints.

PDF Details DOI

IJCAI Conference 2019 Conference Paper

Decoding EEG by Visual-guided Deep Neural Networks

  • Zhicheng Jiao
  • Haoxuan You
  • Fan Yang
  • Xin Li
  • Han Zhang
  • Dinggang Shen

Decoding visual stimuli from brain activities is an interdisciplinary study of neuroscience and computer vision. With the emerging of Human-AI Collaboration, Human-Computer Interaction, and the development of advanced machine learning models, brain decoding based on deep learning attracts more attention. Electroencephalogram (EEG) is a widely used neurophysiology tool. Inspired by the success of deep learning on image representation and neural decoding, we proposed a visual-guided EEG decoding method that contains a decoding stage and a generation stage. In the classification stage, we designed a visual-guided convolutional neural network (CNN) to obtain more discriminative representations from EEG, which are applied to achieve the classification results. In the generation stage, the visual-guided EEG features are input to our improved deep generative model with a visual consistence module to generate corresponding visual stimuli. With the help of our visual-guided strategies, the proposed method outperforms traditional machine learning methods and deep learning models in the EEG decoding task.

PDF Details

YNIMG Journal 2019 Journal Article

First-year development of modules and hubs in infant brain functional networks

  • Xuyun Wen
  • Han Zhang
  • Gang Li
  • Mingxia Liu
  • Weiyan Yin
  • Weili Lin
  • Jun Zhang
  • Dinggang Shen

The human brain develops rapidly in the first postnatal year, in which rewired functional brain networks could shape later behavioral and cognitive performance. Resting-state functional magnetic resonances imaging (rs-fMRI) and complex network analysis have been widely used for characterizing the developmental brain functional connectome. Yet, such studies focusing on the first year of postnatal life are still very limited. Leveraging normally developing longitudinal infant rs-fMRI scans from neonate to one year of age, we investigated how brain functional networks develop at a fine temporal scale (every 3 months). Considering challenges in the infant fMRI-based network analysis, we developed a novel algorithm to construct the robust, temporally consistent and modular structure augmented group-level network based on which functional modules were detected at each age. Our study reveals that the brain functional network is gradually subdivided into an increasing number of functional modules accompanied by the strengthened intra- and inter-modular connectivities. Based on the developing modules, we found connector hubs (the high-centrality regions connecting different modules) emerging and increasing, while provincial hubs (the high-centrality regions connecting regions in the same module) diminishing. Further region-wise longitudinal analysis validates that different hubs have distinct developmental trajectories of the intra- and inter-modular connections suggesting different types of role transitions in network, such as non-hubs to hubs or provincial hubs to connector hubs et al. All findings indicate that functional segregation and integration are both increased in the first year of postnatal life. The module reorganization and hub transition lead to more efficient brain networks, featuring increasingly segregated modular structure and more connector hubs. This study provides the first comprehensive report of the development of functional brain networks at a 3-month interval throughout the first postnatal year of life, which provides essential information to the future neurodevelopmental and developmental disorder studies.

Details DOI

YNIMG Journal 2019 Journal Article

Resting-state functional MRI studies on infant brains: A decade of gap-filling efforts

  • Han Zhang
  • Dinggang Shen
  • Weili Lin

Resting-state functional MRI (rs-fMRI) is one of the most prevalent brain functional imaging modalities. Previous rs-fMRI studies have mainly focused on adults and elderly subjects. Recently, infant rs-fMRI studies have become an area of active research. After a decade of gap filling studies, many facets of the brain functional development from early infancy to toddler has been uncovered. However, infant rs-fMRI is still in its infancy. The image analysis tools for neonates and young infants can be quite different from those for adults. From data analysis to result interpretation, more questions and issues have been raised, and new hypotheses have been formed. With the anticipated availability of unprecedented high-resolution rs-fMRI and dedicated analysis pipelines from the Baby Connectome Project (BCP), it is important now to revisit previous findings and hypotheses, discuss and comment existing issues and problems, and make a “to-do-list” for the future studies. This review article aims to comprehensively review a decade of the findings, unveiling hidden jewels of the fields of developmental neuroscience and neuroimage computing. Emphases will be given to early infancy, particularly the first few years of life. In this review, an end-to-end summary, from infant rs-fMRI experimental design to data processing, and from the development of individual functional systems to large-scale brain functional networks, is provided. A comprehensive summary of the rs-fMRI findings in developmental patterns is highlighted. Furthermore, an extensive summary of the neurodevelopmental disorders and the effects of other hazardous factors is provided. Finally, future research trends focusing on emerging dynamic functional connectivity and state-of-the-art functional connectome analysis are summarized. In next decade, early infant rs-fMRI and developmental connectome study could be one of the shining research topics.

Details DOI

YNIMG Journal 2017 Journal Article

The neural circuits for arithmetic principles

  • Jie Liu
  • Han Zhang
  • Chuansheng Chen
  • Hui Chen
  • Jiaxin Cui
  • Xinlin Zhou

Arithmetic principles are the regularities underlying arithmetic computation. Little is known about how the brain supports the processing of arithmetic principles. The current fMRI study examined neural activation and functional connectivity during the processing of verbalized arithmetic principles, as compared to numerical computation and general language processing. As expected, arithmetic principles elicited stronger activation in bilateral horizontal intraparietal sulcus and right supramarginal gyrus than did language processing, and stronger activation in left middle temporal lobe and left orbital part of inferior frontal gyrus than did computation. In contrast, computation elicited greater activation in bilateral horizontal intraparietal sulcus (extending to posterior superior parietal lobule) than did either arithmetic principles or language processing. Functional connectivity analysis with the psychophysiological interaction approach (PPI) showed that left temporal-parietal (MTG-HIPS) connectivity was stronger during the processing of arithmetic principle and language than during computation, whereas parietal-occipital connectivities were stronger during computation than during the processing of arithmetic principles and language. Additionally, the left fronto-parietal (orbital IFG-HIPS) connectivity was stronger during the processing of arithmetic principles than during computation. The results suggest that verbalized arithmetic principles engage a neural network that overlaps but is distinct from the networks for computation and language processing.

Details DOI

YNIMG Journal 2016 Journal Article

Estimating functional brain networks by incorporating a modularity prior

  • Lishan Qiao
  • Han Zhang
  • Minjeong Kim
  • Shenghua Teng
  • Limei Zhang
  • Dinggang Shen

Functional brain network analysis has become one principled way of revealing informative organization architectures in healthy brains, and providing sensitive biomarkers for diagnosis of neurological disorders. Prior to any post hoc analysis, however, a natural issue is how to construct “ideal” brain networks given, for example, a set of functional magnetic resonance imaging (fMRI) time series associated with different brain regions. Although many methods have been developed, it is currently still an open field to estimate biologically meaningful and statistically robust brain networks due to our limited understanding of the human brain as well as complex noises in the observed data. Motivated by the fact that the brain is organized with modular structures, in this paper, we propose a novel functional brain network modeling scheme by encoding a modularity prior under a matrix-regularized network learning framework, and further formulate it as a sparse low-rank graph learning problem, which can be solved by an efficient optimization algorithm. Then, we apply the learned brain networks to identify patients with mild cognitive impairment (MCI) from normal controls. We achieved 89. 01% classification accuracy even with a simple feature selection and classification pipeline, which significantly outperforms the conventional brain network construction methods. Moreover, we further explore brain network features that contributed to MCI identification, and discovered potential biomarkers for personalized diagnosis.

Details DOI

YNICL Journal 2013 Journal Article

“Awake” intraoperative functional MRI (ai-fMRI) for mapping the eloquent cortex: Is it possible in awake craniotomy?

  • Jun-Feng Lu
  • Han Zhang
  • Jin-Song Wu
  • Cheng-Jun Yao
  • Dong-Xiao Zhuang
  • Tian-Ming Qiu
  • Wen-Bin Jia
  • Ying Mao

As a promising noninvasive imaging technique, functional MRI (fMRI) has been extensively adopted as a functional localization procedure for surgical planning. However, the information provided by preoperative fMRI (pre-fMRI) is hampered by the brain deformation that is secondary to surgical procedures. Therefore, intraoperative fMRI (i-fMRI) becomes a potential alternative that can compensate for brain shifts by updating the functional localization information during craniotomy. However, previous i-fMRI studies required that patients be under general anesthesia, preventing the wider application of such a technique as the patients cannot perform tasks unless they are awake. In this study, we propose a new technique that combines awake surgery and i-fMRI, named "awake" i-fMRI (ai-fMRI). We introduced ai-fMRI to the real-time localization of sensorimotor areas during awake craniotomy in seven patients. The results showed that ai-fMRI could successfully detect activations in the bilateral primary sensorimotor areas and supplementary motor areas for all patients, indicating the feasibility of this technique in eloquent area localization. The reliability of ai-fMRI was further validated using intraoperative stimulation mapping (ISM) in two of the seven patients. Comparisons between the pre-fMRI-derived localization result and the ai-fMRI derived result showed that the former was subject to a heavy brain shift and led to incorrect localization, while the latter solved that problem. Additionally, the approaches for the acquisition and processing of the ai-fMRI data were fully illustrated and described. Some practical issues on employing ai-fMRI in awake craniotomy were systemically discussed, and guidelines were provided.

Details DOI

YNIMG Journal 2012 Journal Article

Neural correlates of numbers and mathematical terms

  • Han Zhang
  • Chuansheng Chen
  • Xinlin Zhou

Numerical processing has been demonstrated to be subserved typically by the brain regions around the bilateral intraparietal sulcus (IPS). The goal of the current study was to investigate whether the processing of mathematical terms shared the same brain regions with numerical processing. Healthy adult participants performed semantic distance judgment tasks on five types of materials, including geometric terms, algebraic terms, linguistic terms, words for tools and other common objects, and Arabic numbers. Brain activation was measured with functional magnetic resonance imaging (fMRI). The results showed that geometric terms had greater activation than algebraic terms, linguistic terms and tool words in the horizontal IPS, but algebraic terms did not have greater activation than linguistic terms and tool words in this region. Arabic numbers showed greater activation than non-number materials (including geometric terms, algebraic terms, linguistic terms and tool words) in the bilateral IPS, right inferior frontal gyrus and bilateral middle frontal gyrus, but the non-number materials showed stronger activation in the left inferior frontal gyrus and left middle temporal gyrus. These results suggest that the brain area for the processing of numbers (the left IPS) seems to be involved in semantic processing of geometric terms, but not that of other mathematical terms such as algebraic terms. Both algebraic and geometric terms share similar brain organization with basic semantic processing in the left temporal and frontal regions.

Details DOI

YNIMG Journal 2011 Journal Article

Test–retest assessment of independent component analysis-derived resting-state functional connectivity based on functional near-infrared spectroscopy

  • Han Zhang
  • Lian Duan
  • Yu-Jin Zhang
  • Chun-Ming Lu
  • Hanli Liu
  • Chao-Zhe Zhu

Recent studies of resting-state functional near-infrared spectroscopy (fNIRS) have emerged as a hot topic and revealed that resting-state functional connectivity (RSFC) is an inherent characteristic of the resting brain. However, it is currently unclear if fNIRS-based RSFC is test–retest reliable. In this study, we utilized independent component analysis (ICA) as an effective RSFC detection tool to address the reliability question. Sixteen subjects participated in two resting-state fNIRS recording sessions held 1week (6. 88±1. 09days) apart. Then, RSFC in the sensorimotor regions was extracted using ICA. Test–retest reliability was assessed for intra- and inter-sessions, at both individual and group levels, and for different hemoglobin concentration signals. Our results clearly demonstrated that map-wise reliability was excellent at the group level (with Pearson's r coefficients up to 0. 88) and generally fair at the individual level. Cluster-wise reliability was better at the group level (having reproducibility indices of up to 0. 97 for the size and up to 0. 80 for the location of the detected RSFC) and was weaker but still fair at the individual level (0. 56 and 0. 46 for intra- and inter-session reliabilities, respectively). Cluster-wise intra-class correlation coefficients (ICCs) also exhibited fair-to-good reliability (with single-measure ICC up to 0. 56), while channel-wise single-measure ICCs indicated lower reliability. We conclude that fNIRS-based, ICA-derived RSFC is an essential and reliable biomarker at the individual and group levels if interpreted in map- and cluster-wise manners. Our results also suggested that channel-wise individual-level RSFC results should be interpreted with caution if no optode co-registration procedure had been conducted and indicated that “cluster” should be treated as a minimal analytical unit in further RSFC studies using fNIRS.

Details DOI

YNIMG Journal 2010 Journal Article

Functional connectivity as revealed by independent component analysis of resting-state fNIRS measurements

  • Han Zhang
  • Yu-Jin Zhang
  • Chun-Ming Lu
  • Shuang-Ye Ma
  • Yu-Feng Zang
  • Chao-Zhe Zhu

As a promising non-invasive imaging technique, functional near infrared spectroscopy (fNIRS) has recently earned increasing attention in resting-state functional connectivity (RSFC) studies. Preliminary fNIRS-based RSFC studies adopted a seed correlation approach and yielded interesting results. However, the seed correlation approach has several inherent problems, such as neglecting of interactions among multiple regions and a dependence on seed region selection. Moreover, ineffectively reduced noise and artifacts in fNIRS measurements also negatively affect RSFC results. In this study, independent component analysis (ICA) was introduced to meet these challenges in RSFC detection based on resting-state fNIRS measurements. The results of ICA on data from the sensorimotor and the visual systems both showed functional system-specific RSFC maps. Results from comparison between ICA and the conventional seed correlation approach demonstrated, both qualitatively and quantitatively, the superior performance of ICA with higher sensitivity and specificity, especially in the case of higher noise level. The capability of ICA to separate noise and artifacts from resting-state fNIRS data was also demonstrated, and the extracted noise and artifacts were illustrated. Finally, some practical issues on performing ICA on resting-state fNIRS data were discussed.

Details DOI

YNIMG Journal 2010 Journal Article

Subject order-independent group ICA (SOI-GICA) for functional MRI data analysis

  • Han Zhang
  • Xi-Nian Zuo
  • Shuang-Ye Ma
  • Yu-Feng Zang
  • Michael P. Milham
  • Chao-Zhe Zhu

Independent component analysis (ICA) is a data-driven approach to study functional magnetic resonance imaging (fMRI) data. Particularly, for group analysis on multiple subjects, temporally concatenation group ICA (TC-GICA) is intensively used. However, due to the usually limited computational capability, data reduction with principal component analysis (PCA: a standard preprocessing step of ICA decomposition) is difficult to achieve for a large dataset. To overcome this, TC-GICA employs multiple-stage PCA data reduction. Such multiple-stage PCA data reduction, however, leads to variable outputs due to different subject concatenation orders. Consequently, the ICA algorithm uses the variable multiple-stage PCA outputs and generates variable decompositions. In this study, a rigorous theoretical analysis was conducted to prove the existence of such variability. Simulated and real fMRI experiments were used to demonstrate the subject-order-induced variability of TC-GICA results using multiple PCA data reductions. To solve this problem, we propose a new subject order-independent group ICA (SOI-GICA). Both simulated and real fMRI data experiments demonstrated the high robustness and accuracy of the SOI-GICA results compared to those of traditional TC-GICA. Accordingly, we recommend SOI-GICA for group ICA-based fMRI studies, especially those with large data sets.

Details DOI

ICRA Conference 2004 Conference Paper

Shape Memory Alloy Microgripper for Robotic Microassembly of Tissue Engineering Scaffolds

  • Han Zhang
  • Yves Bellouard
  • Etienne Burdet
  • Reymond Clavel
  • Aun Neow Poo
  • Dietmar Werner Hutmacher

This paper presents a monolithic shape memory alloy (SMA) microgripper, which was developed to assemble microscopic building blocks of width 60 /spl mu/m into tissue engineering scaffolds. It consists of two small fingers for grasping, an actuator which changes its shape upon heating by Joule effect and a parallel elastic structure to provide a pullback force on cooling as well as to guide the finger movement. All the elements are laser-cut from the same Ni-Ti-Cu sheet but have different mechanical properties and are used for different functions. Using local laser annealing, a local shape memory effect is introduced on the actuator while leaving other areas in a cold-worked state, i. e. no shape memory effect occurs. The material has nonlinear mechanics and the actuator undergoes large deflections. A numerical method is introduced to compute the nonlinear dynamics of the coupled system formed by the actuator and pullback spring. The predicted deflection and force are in good agreement with the measured ones, and this model can be used to optimize the design.

Details

ICRA Conference 2002 Conference Paper

Robotic Micro-Assembly of Scaffold/Cell Constructs with a Shape Memory Alloy Gripper

  • Han Zhang
  • Etienne Burdet
  • Dietmar Werner Hutmacher
  • Aun Neow Poo
  • Yves Bellouard
  • Reymond Clavel
  • Thomas Sidler

Describes an integrated approach to design and fabricate scaffold/cell constructs for tissue engineering. With this approach it becomes possible to produce scaffolds with controlled distribution of living cells and growth factors, a critical condition for successful grafting. Our idea consists of building a scaffold/cell construct by robotic micro-assembly of microscopic polymer building blocks. The paper introduces the rationale and concept of this interdisciplinary project and presents some realized steps. A 3D contact FEM simulation has been carried out to study the forces involved on the scaffold elements and micro-gripper during assembly. An error analysis has been performed to evaluate the accumulated error when building a scaffold/cell construct. A dedicated monolithic shape memory alloy micro-gripper has been realized and tested, which is able to handle parts in the range of 50-100 microns.

Details
v2026.09.13