Arrow Research search

Author name cluster

Qi Zhu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

37 papers
2 author rows

Possible papers

37

TAAS Journal 2026 Journal Article

A Unified Framework for Noisy Image Super-Resolution

  • Ziang Wu
  • Qi Zhang
  • Xiaoli Sun
  • Yonglin Tian
  • Qi Zhu
  • Chunwei Tian

High-quality images are essential for human-computer interaction in industrial systems; however, captured images are often degraded by device vibrations and moving objects. The single-image super-resolution (SISR) task aims to reconstruct high-quality images from low-quality inputs, where deep networks have demonstrated significant success. Nevertheless, noisy image super-resolution remains challenging due to the difficulty of separating noise from genuine structural details. Unlike traditional methods that struggle to distinguish noise from actual image details, the proposed unified framework for noisy image super-resolution (UFNet) combines discriminative learning with a degradation model to enhance both noise suppression and detail recovery. UFNet employs two parallel networks to extract informative features for reconstructing high-quality images. The upper branch utilizes a discriminative learning strategy to remove noise, while the lower branch applies the concept of a degradation model to recover structural details. To restore lost details while maintaining naturalness and structural consistency, a Feature Distillation and Refinement Block (FDRB) is embedded in the lower network. Furthermore, a refinement network is employed to eliminate redundant information introduced during the fusion operation, thereby unifying the results of the two branches and enhancing the final super-resolution outcomes. Extensive experiments demonstrate that the proposed UFNet achieves excellent performance in noisy image super-resolution. The source code of UFNet is available at https://github.com/WuZiang73/UFNet.

Details DOI

AAAI Conference 2026 Conference Paper

DeepPhy: Benchmarking Agentic VLMs on Physical Reasoning

  • Xinrun Xu
  • Pi Bu
  • Ye Wang
  • Börje F. Karlsson
  • Ziming Wang
  • Tengtao Song
  • Qi Zhu
  • Jun Song

Although Vision Language Models (VLMs) exhibit strong perceptual abilities and impressive visual reasoning, they struggle with attention to detail and precise action planning in complex, dynamic environments, leading to subpar performance. Real-world tasks typically require complex interactions, advanced spatial reasoning, long-term planning, and continuous strategy refinement, usually necessitating understanding the physics rules of the target scenario. However, evaluating these capabilities in real-world scenarios is often prohibitively expensive. To bridge this gap, we introduce DeepPHY, a novel benchmark framework designed to systematically evaluate VLMs' understanding and reasoning about fundamental physical principles through a series of challenging simulated environments. DeepPHY integrates multiple physical reasoning environments of varying difficulty levels and incorporates fine-grained evaluation metrics. Our evaluation finds that even state-of-the-art VLMs struggle to translate descriptive physical knowledge into precise, predictive control.

PDF Details DOI

JBHI Journal 2026 Journal Article

Dynamic Theta-Alpha Inter-Brain Model during Mother-Preschooler Cooperation

  • Jiayang Xu
  • Yamin Li
  • Ruxin Su
  • Saishuang Wu
  • Chengcheng Wu
  • Haiwa Wang
  • Qi Zhu
  • Yue Fang

The interaction between mothers and young children is a highly dynamic process neurally characterized by inter-brain synchrony (IBS) at θ and/or α rhythms. However, their establishment, dynamic changes, and roles in mother-child interactions remain unknown. In this study, through a simultaneous dynamic analysis of inter-brain EEG synchrony, intra-brain EEG power, and interactive behaviors from 40 mother-preschooler dyads during turn-taking cooperation, we constructed a dynamic inter-brain model that θ-IBS and α-IBS alternated with interactive behaviors, with EEG frequency-shift as a prerequisite for IBS transitions. When mothers attempt to track their children’s attention and/or predict their intentions, they will adjust their EEG frequencies to align with their children’s θ oscillations, leading to a higher occurrence of the θ-IBS state. Conversely, the α-IBS state, accompanied by the EEG frequency-shift to the α range, is more prominent during mother-led interactions. Further exploratory analysis reveals greater presence and stability of the θ-IBS state during cooperative than non-cooperative conditions, particularly in dyads with stronger emotional attachments and more frequent interactions in their daily lives. Our findings shed light on the neural oscillatory substrates underlying the IBS dynamics during mother-preschooler interactions.

Details DOI

AAAI Conference 2026 Conference Paper

PulseMind: A Multi-Modal Medical Model for Real-World Clinical Diagnosis

  • Jiao Xu
  • Junwei Liu
  • Jiangwei Lao
  • Qi Zhu
  • Yunpeng Zhao
  • Congyun Jin
  • Shinan Liu
  • Zhihong Lu

Recent advances in medical multi-modal models focus on specialized image analysis like dermatology, pathology, or radiology. However, they do not fully capture the complexity of real-world clinical diagnostics, which involve heterogeneous inputs and require ongoing contextual understanding during patient-physician interactions. To bridge this gap, we introduce PulseMind, a new family of multi-modal diagnostic models that integrates a systematically curated dataset, a comprehensive evaluation benchmark, and a tailored training framework. Specifically, we first construct a diagnostic dataset, MediScope, which comprises 98,000 real-world multi-turn consultations and 601,500 medical images, spanning over 10 major clinical departments and more than 200 sub-specialties. Then, to better reflect the requirements of real-world clinical diagnosis, we develop the PulseMind Benchmark, a multi-turn diagnostic consultation benchmark with a four-dimensional evaluation protocol comprising proactiveness, accuracy, usefulness, and language quality. Finally, we design a training framework tailored for multi-modal clinical diagnostics, centered around a core component named Comparison-based Reinforcement Policy Optimization (CRPO). Compared to absolute score rewards, CRPO uses relative preference signals from multi-dimensional comparisons to provide stable and human-aligned training guidance. Extensive experiments demonstrate that PulseMind achieves competitive performance on both the diagnostic consultation benchmark and public medical benchmarks.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

AdaptPFL: Unlocking Cross-Device Palmprint Recognition via Adaptive Personalized Federated Learning with Feature Decoupling

  • Zirui Zhang
  • Donghai Guan
  • Çetin Kaya Koç
  • Jie Wen
  • Qi Zhu

Contactless palmprint recognition has recently emerged as a promising biometric technology. However, traditional methods that require sharing user data introduce substantial security risks. While federated learning offers privacy-preserving solutions, it often compromises recognition accuracy due to feature distribution drift caused by external factors such as lighting and devices. To address this issue, we propose an adaptive personalized federated learning framework (AdaptPFL). The central innovation lies in decomposing palmprint features into identity-related and contextual-related components using a feature decoupling mechanism. This design isolates the influence of external environmental factors on identity recognition through de-entanglement. Furthermore, two adaptive aggregation strategies are introduced to correct client drift: (1) Intra-Local Adaptive Aggregation (ILAA), which addresses intra-client drift by adaptively combining the two decoupled feature types; (2) Global-Local Adaptive Aggregation (GLAA), which corrects inter-client drift by adaptively aggregating model parameters. Experimental results demonstrate that AdaptPFL achieves superior performance compared to existing state-of-the-art methods.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization

  • Yichen Yan
  • Ming Zhong
  • Qi Zhu
  • Xiaoling Gu
  • Jinpeng Chen
  • Huan Li

Multimodal large language models (MLLMs) rely heavily on instruction tuning to align vision and language capabilities, yet the computational cost of training on large-scale datasets remains a major bottleneck. Existing data selection methods aim to mitigate this by selecting important and diverse subsets, but they often suffer from two critical drawbacks: high computational overhead from processing the entire dataset and suboptimal data selection due to separate treatment of importance and diversity. We introduce CoIDO, a novel dual-objective framework that jointly optimizes data importance and diversity to overcome these challenges. Unlike existing approaches that require costly evaluations across the whole dataset, CoIDO employs a lightweight plug-in scorer. This scorer is trained on just a small random sample of data to learn the distribution of the candidate set, drastically reducing computational demands. By leveraging a homoscedastic uncertainty-based formulation, CoIDO effectively balances importance and diversity during training, enabling the scorer to assign CoIDO scores to all data points. This unified scoring approach allows for direct ranking and selection of the most valuable subsets, completely bypassing the need for specialized algorithms. In our experiments, we trained the CoIDO Scorer using only 20% of randomly sampled data. Once trained, CoIDO was applied to the entire dataset to select a 20% subset for instruction tuning. On the widely used LLaVA-1. 5-7B model across ten downstream tasks, this selected subset achieved an impressive 98. 2% of the performance of full-data fine-tuning, on average. Moreover, CoIDO outperforms all competitors in terms of both efficiency (lowest training FLOPs) and aggregated accuracy. Our code is available at: https: //github. com/SuDIS-ZJU/CoIDO

PDF Details

NeurIPS Conference 2025 Conference Paper

MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series

  • Payal Mohapatra
  • Yueyuan Sui
  • Akash Pandey
  • Stephen Xia
  • Qi Zhu

From clinical healthcare to daily living, continuous sensor monitoring across multiple modalities has shown great promise for real-world intelligent decision-making but also faces various challenges. In this work, we argue for modeling such heterogeneous data sources under the multimodal paradigm and introduce a new framework, MAESTRO. We introduce MAESTRO, a novel framework that overcomes key limitations of existing multimodal learning approaches: (1) reliance on a single primary modality for alignment, (2) pairwise modeling of modalities, and (3) assumption of complete modality observations. These limitations hinder the applicability of these approaches in real-world multimodal time-series settings, where primary modality priors are often unclear, the number of modalities can be large (making pairwise modeling impractical), and sensor failures often result in arbitrary missing observations. At its core, MAESTRO facilitates dynamic intra- and cross-modal interactions based on task relevance, and leverages symbolic tokenization and adaptive attention budgeting to construct long multimodal sequences, which are processed via sparse cross-modal attention. The resulting cross-modal tokens are routed through a sparse Mixture-of-Experts (MoE) mechanism, enabling black-box specialization under varying modality combinations. We evaluate MAESTRO against 10 baselines on four diverse datasets spanning three applications, and observe average relative improvements of 4% and 8% over the best existing multimodal and multivariate approaches, respectively, under complete observations. Under partial observations—with up to 40% of missing modalities—MAESTRO achieves an average 9% improvement. Further analysis also demonstrates the robustness and efficiency of MAESTRO's sparse, modality-aware design for learning from dynamic time series.

PDF Details

NeurIPS Conference 2025 Conference Paper

MAPLE: Multi-scale Attribute-enhanced Prompt Learning for Few-shot Whole Slide Image Classification

  • Junjie Zhou
  • WEI SHAO
  • Yagao Yue
  • Wei Mu
  • Peng Wan
  • Qi Zhu
  • Daoqiang Zhang

Prompt learning has emerged as a promising paradigm for adapting pre-trained vision-language models (VLMs) to few-shot whole slide image (WSI) classification by aligning visual features with textual representations, thereby reducing annotation cost and enhancing model generalization. Nevertheless, existing methods typically rely on slide-level prompts and fail to capture the subtype-specific phenotypic variations of histological entities (e. g. , nuclei, glands) that are critical for cancer diagnosis. To address this gap, we propose Multi-scale Attribute-enhanced Prompt Learning (MAPLE), a hierarchical framework for few-shot WSI classification that jointly integrates multi-scale visual semantics and performs prediction at both the entity and slide levels. Specifically, we first leverage large language models (LLMs) to generate entity-level prompts that can help identify multi-scale histological entities and their phenotypic attributes, as well as slide-level prompts to capture global visual descriptions. Then, an entity-guided cross-attention module is proposed to generate entity-level features, followed by aligning with their corresponding subtype-specific attributes for fine-grained entity-level prediction. To enrich entity representations, we further develop a cross-scale entity graph learning module that can update these representations by capturing their semantic correlations within and across scales. The refined representations are then aggregated into a slide-level representation and aligned with the corresponding prompts for slide-level prediction. Finally, we combine both entity-level and slide-level outputs to produce the final prediction results. Results on three cancer cohorts confirm the effectiveness of our approach in addressing few-shot pathology diagnosis tasks.

PDF Details

YNIMG Journal 2025 Journal Article

Multi-channel spatio-temporal graph attention contrastive network for brain disease diagnosis

  • Chaojun Li
  • Kai Ma
  • Shengrong Li
  • Xiangshui Meng
  • Ran Wang
  • Daoqiang Zhang
  • Qi Zhu

Dynamic brain networks (DBNs) can capture the intricate connections and temporal evolution among brain regions, becoming increasingly crucial in the diagnosis of neurological disorders. However, most existing researches tend to focus on isolated brain network sequence segmented by sliding windows, and they are difficult to effectively uncover the higher-order spatio-temporal topological pattern in DBNs. Meantime, it remains a challenge to utilize the structure connectivity prior in the DBNs analysis. To address these problems, we propose a multi-channel spatio-temporal graph attention contrastive network for DBNs analysis. Specifically, we first construct dynamic brain functional networks from fMRI data with sliding windows, and embed the structural connectivity derived from diffusion tensor imaging (DTI) to the dynamic functional connectivity graph representation to construct multi-modal brain network. Second, we develop a multi-channel spatial attention contrastive network to extract topological features from the brain network within each time window. This network incorporates an intra-window graph contrastive constraint to enhance the discriminative ability of the extracted features. Moreover, temporal dependencies across windows are captured by integrating feature embeddings through a self-attention mechanism, and the inter-window recurrent contrastive constraint is devised to extract higher-order spatio-temporal topological features. Finally, a multi-layer perceptron (MLP) is used to classify the brain networks. Experiments on epilepsy and ADNI datasets show that our method outperforms several state-of-the-art approaches in diagnosing performance, and it provides discriminative graph features for related brain diseases.

Details DOI

NeurIPS Conference 2025 Conference Paper

NeuroH-TGL: Neuro-Heterogeneity Guided Temporal Graph Learning Strategy for Brain Disease Diagnosis

  • Shengrong Li
  • Qi Zhu
  • Chunwei Tian
  • Xinyang Zhang
  • WEI SHAO
  • Jie Wen
  • Daoqiang Zhang

Dynamic functional brain networks (DFBNs) are powerful tools in neuroscience research. Recent studies reveal that DFBNs contain heterogeneous neural nodes with more extensive connections and more drastic temporal changes, which play pivotal roles in coordinating the reorganization of the brain. Moreover, the spatio-temporal patterns of these nodes are modulated by the brain's historical states. However, existing methods not only ignore the spatio-temporal heterogeneity of neural nodes, but also fail to effectively encode the temporal propagation mechanism of heterogeneous activities. These limitations hinder the deep exploration of spatio-temporal relationships within DFBNs, preventing the capture of abnormal neural heterogeneity caused by brain diseases. To address these challenges, this paper propose a neuro-heterogeneity guided temporal graph learning strategy (NeuroH-TGL). Specifically, we first develop a spatio-temporal pattern decoupling module to disentangle DFBNs into topological consistency networks and temporal trend networks that align with the brain's operational mechanisms. Then, we introduce a heterogeneity mining module to identify pivotal heterogeneity nodes that drive brain reorganization from the two decoupled networks. Finally, we design temporal propagation graph convolution to simulate the influence of the historical states of heterogeneity nodes on the current topology, thereby flexibly extracting heterogeneous spatio-temporal information from the brain. Experiments show that our method surpasses several state-of-the-art methods, and can identify abnormal heterogeneous nodes caused by brain diseases.

PDF Details

TMLR Journal 2025 Journal Article

Phase-driven Generalizable Representation Learning for Nonstationary Time Series Classification

  • Payal Mohapatra
  • Lixu Wang
  • Qi Zhu

Pattern recognition is a fundamental task in continuous sensing applications, but real-world scenarios often experience distribution shifts that necessitate learning generalizable representations for such tasks. This challenge is exacerbated with time-series data, which also exhibit inherent nonstationarity—variations in statistical and spectral properties over time. In this work, we offer a fresh perspective on learning generalizable representations for time-series classification by considering the phase information of a signal as an approximate proxy for nonstationarity and propose a phase-driven generalizable representation learning framework for time-series classification, PhASER. It consists of three key elements: 1) Hilbert transform-based augmentation, which diversifies nonstationarity while preserving task-specific discriminatory semantics, 2) separate magnitude-phase encoding, viewing time-varying magnitude and phase as independent modalities, and 3) phase-residual feature broadcasting, integrating 2D phase features with a residual connection to the 1D signal representation, providing inherent regularization to improve distribution-invariant learning. Extensive evaluations on five datasets from sleep-stage classification, human activity recognition, and gesture recognition against 13 state-of-the-art baseline methods demonstrate that PhASER consistently outperforms the best baselines by an average of 5% and up to 11% in some cases. Additionally, the principles of PhASER can be broadly applied to enhance the generalizability of existing time-series representation learning models.

PDF Details

AAAI Conference 2024 Conference Paper

Fine-Tuning Graph Neural Networks by Preserving Graph Generative Patterns

  • Yifei Sun
  • Qi Zhu
  • Yang Yang
  • Chunping Wang
  • Tianyu Fan
  • Jiajun Zhu
  • Lei Chen

Recently, the paradigm of pre-training and fine-tuning graph neural networks has been intensively studied and applied in a wide range of graph mining tasks. Its success is generally attributed to the structural consistency between pre-training and downstream datasets, which, however, does not hold in many real-world scenarios. Existing works have shown that the structural divergence between pre-training and downstream graphs significantly limits the transferability when using the vanilla fine-tuning strategy. This divergence leads to model overfitting on pre-training graphs and causes difficulties in capturing the structural properties of the downstream graphs. In this paper, we identify the fundamental cause of structural divergence as the discrepancy of generative patterns between the pre-training and downstream graphs. Furthermore, we propose G-Tuning to preserve the generative patterns of downstream graphs. Given a downstream graph G, the core idea is to tune the pre-trained GNN so that it can reconstruct the generative patterns of G, the graphon W. However, the exact reconstruction of a graphon is known to be computationally expensive. To overcome this challenge, we provide a theoretical analysis that establishes the existence of a set of alternative graphons called graphon bases for any given graphon. By utilizing a linear combination of these graphon bases, we can efficiently approximate W. This theoretical finding forms the basis of our model, as it enables effective learning of the graphon bases and their associated coefficients. Compared with existing algorithms, G-Tuning demonstrates consistent performance improvement in 7 in-domain and 7 out-of-domain transfer learning experiments.

PDF Details DOI

IROS Conference 2024 Conference Paper

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

  • Anthony Goeckner
  • Yueyuan Sui
  • Nicolas Martinet
  • Xinliang Li
  • Qi Zhu

Existing multi-agent coordination techniques are often fragile and vulnerable to anomalies such as agent attrition and communication disturbances, which are quite common in the real-world deployment of systems like field robotics. To better prepare these systems for the real world, we present a graph neural network (GNN)-based multi-agent reinforcement learning (MARL) method for resilient distributed coordination of a multi-robot system. Our method, Multi-Agent Graph Embedding-based Coordination (MAGEC), is trained using multi-agent proximal policy optimization (PPO) and enables distributed coordination around global objectives under agent attrition, partial observability, and limited or disturbed communications. We use a multi-robot patrolling scenario to demonstrate our MAGEC method in a ROS 2-based simulator and then compare its performance with prior coordination approaches. Results demonstrate that MAGEC outperforms existing methods in several experiments involving agent attrition and communication disturbance, and provides competitive results in scenarios without such anomalies.

Details

YNICL Journal 2024 Journal Article

Joint multi-site domain adaptation and multi-modality feature selection for the diagnosis of psychiatric disorders

  • Yixin Ji
  • Rogers F. Silva
  • Tülay Adali
  • Xuyun Wen
  • Qi Zhu
  • Rongtao Jiang
  • Daoqiang Zhang
  • Shile Qi

Identifying biomarkers for computer-aided diagnosis (CAD) is crucial for early intervention of psychiatric disorders. Multi-site data have been utilized to increase the sample size and improve statistical power, while multi-modality classification offers significant advantages over traditional single-modality based approaches for diagnosing psychiatric disorders. However, inter-site heterogeneity and intra-modality heterogeneity present challenges to multi-site and multi-modality based classification. In this paper, brain functional and structural networks (BFNs/BSNs) from multiple sites were constructed to establish a joint multi-site multi-modality framework for psychiatric diagnosis. To do this we developed a hypergraph based multi-source domain adaptation (HMSDA) which allowed us to transform source domain subjects into a target domain. A local ordinal structure based multi-task feature selection (LOSMFS) approach was developed by integrating the transformed functional and structural connections (FCs/SCs). The effectiveness of our method was validated by evaluating diagnosis of both schizophrenia (SZ) and autism spectrum disorder (ASD). The proposed method obtained accuracies of 92.2 %±2.22 % and 84.8 %±2.68 % for the diagnosis of SZ and ASD, respectively. We also compared with 6 DA, 10 multi-modality feature selection, and 8 multi-site and multi-modality methods. Results showed the proposed HMSDA+LOSMFS effectively integrated multi-site and multi-modality data to enhance psychiatric diagnosis and identify disorder-specific diagnostic brain connections.

Details DOI

ICLR Conference 2024 Conference Paper

Parallelizing non-linear sequential models over the sequence length

  • Yi Heng Lim
  • Qi Zhu
  • Joshua Selfridge
  • Muhammad Firmansyah Kasim

Sequential models, such as Recurrent Neural Networks and Neural Ordinary Differential Equations, have long suffered from slow training due to their inherent sequential nature. For many years this bottleneck has persisted, as many thought sequential models could not be parallelized. We challenge this long-held belief with our parallel algorithm that accelerates GPU evaluation of sequential models by up to 3 orders of magnitude faster without compromising output accuracy. The algorithm does not need any special structure in the sequential models' architecture, making it applicable to a wide range of architectures. Using our method, training sequential models can be more than 10 times faster than the common sequential method without any meaningful difference in the training results. Leveraging this accelerated training, we discovered the efficacy of the Gated Recurrent Unit in a long time series classification problem with 17k time samples. By overcoming the training bottleneck, our work serves as the first step to unlock the potential of non-linear sequential models for long sequence problems.

Details

AAAI Conference 2024 Conference Paper

REGLO: Provable Neural Network Repair for Global Robustness Properties

  • Feisi Fu
  • Zhilu Wang
  • Weichao Zhou
  • Yixuan Wang
  • Jiameng Fan
  • Chao Huang
  • Qi Zhu
  • Xin Chen

We present REGLO, a novel methodology for repairing pretrained neural networks to satisfy global robustness and individual fairness properties. A neural network is said to be globally robust with respect to a given input region if and only if all the input points in the region are locally robust. This notion of global robustness also captures the notion of individual fairness as a special case. We prove that any counterexample to a global robustness property must exhibit a corresponding large gradient. For ReLU networks, this result allows us to efficiently identify the linear regions that violate a given global robustness property. By formulating and solving a suitable robust convex optimization problem, REGLO then computes a minimal weight change that will provably repair these violating linear regions.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Semantic Feature Learning for Universal Unsupervised Cross-Domain Retrieval

  • Lixu Wang
  • Xinyu Du
  • Qi Zhu

Cross-domain retrieval (CDR) is finding increasingly broad applications across various domains. However, existing efforts have several major limitations, with the most critical being their reliance on accurate supervision. Recent studies thus focus on achieving unsupervised CDR, but they typically assume that the category spaces across domains are identical, an assumption that is often unrealistic in real-world scenarios. This is because only through dedicated and comprehensive analysis can the category composition of a data domain be obtained, which contradicts the premise of unsupervised scenarios. Therefore, in this work, we introduce the problem of U niversal U nsupervised C ross- D omain R etrieval (U^2CDR) for the first time and design a two-stage semantic feature learning framework to address it. In the first stage, a cross-domain unified prototypical structure is established under the guidance of an instance-prototype-mixed contrastive loss and a semantic-enhanced loss, to counteract category space differences. In the second stage, through a modified adversarial training mechanism, we ensure minimal changes for the established prototypical structure during domain alignment, enabling more accurate nearest-neighbor searching. Extensive experiments across multiple datasets and scenarios, including close-set, partial, and open-set CDR, demonstrate that our approach significantly outperforms existing state-of-the-art CDR methods and other related methods in solving U^2CDR challenges.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Variational Delayed Policy Optimization

  • Qingyuan Wu
  • Simon S. Zhan
  • Yixuan Wang
  • Yuhui Wang
  • Chung-Wei Lin
  • Chen Lv
  • Qi Zhu
  • Chao Huang

In environments with delayed observation, state augmentation by including actions within the delay window is adopted to retrieve Markovian property to enable reinforcement learning (RL). Whereas, state-of-the-art (SOTA) RL techniques with Temporal-Difference (TD) learning frameworks commonly suffer from learning inefficiency, due to the significant expansion of the augmented state space with the delay. To improve the learning efficiency without sacrificing performance, this work novelly introduces Variational Delayed Policy Optimization (VDPO), reforming delayed RL as a variational inference problem. This problem is further modelled as a two-step iterative optimization problem, where the first step is TD learning in the delay-free environment with a small state space, and the second step is behaviour cloning which can be addressed much more efficiently than TD learning. We not only provide a theoretical analysis of VDPO in terms of sample complexity and performance, but also empirically demonstrate that VDPO can achieve consistent performance with SOTA methods, with a significant enhancement of sample efficiency (approximately 50\% less amount of samples) in the MuJoCo benchmark.

PDF Details DOI

AAMAS Conference 2023 Conference Paper

Connectivity Enhanced Safe Neural Network Planner for Lane Changing in Mixed Traffic

  • Xiangguo Liu
  • Ruochen Jiao
  • Bowen Zheng
  • Dave Liang
  • Qi Zhu

Connectivity technology has shown great potentials in improving the safety and efficiency of transportation systems by providing information beyond the perception and prediction capabilities of individual vehicles. However, it is expected that human-driven and autonomous vehicles, and connected and non-connected vehicles need to share the transportation network during the transition period to fully connected and automated transportation systems. Such mixed traffic scenarios significantly increase the complexity in analyzing system behavior for highly interactive scenarios, e. g. , lane changing. It is even harder to ensure system safety when neural network based planners are leveraged. In this work, we propose a connectivity-enhanced neural network based lane changing planner. By cooperating with surrounding connected vehicles, our proposed planner will adapt its planned trajectory according to the analysis of a safe evasion trajectory. We demonstrate the strength of our planner design in improving efficiency and ensuring safety in various mixed traffic scenarios with extensive simulations. We also analyze the system robustness when the communication or coordination is not perfect.

PDF

IJCAI Conference 2023 Conference Paper

Efficient Global Robustness Certification of Neural Networks via Interleaving Twin-Network Encoding (Extended Abstract)

  • Zhilu Wang
  • Chao Huang
  • Qi Zhu

The robustness of deep neural networks in safety-critical systems has received significant interest recently, which measures how sensitive the model output is under input perturbations. While most previous works focused on the local robustness property, the studies of the global robustness property, i. e. , the robustness in the entire input space, are still lacking. In this work, we formulate the global robustness certification problem for ReLU neural networks and present an efficient approach to address it. Our approach includes a novel interleaving twin-network encoding scheme and an over-approximation algorithm leveraging relaxation and refinement techniques. Its timing efficiency and effectiveness are evaluated and compared with other state-of-the-art global robustness certification methods, and demonstrated via case studies on practical applications.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

FouriDown: Factoring Down-Sampling into Shuffling and Superposing

  • Qi Zhu
  • Man Zhou
  • Jie Huang
  • Naishan Zheng
  • Hongzhi Gao
  • Chongyi Li
  • Yuan Xu
  • Feng Zhao

Spatial down-sampling techniques, such as strided convolution, Gaussian, and Nearest down-sampling, are essential in deep neural networks. In this study, we revisit the working mechanism of the spatial down-sampling family and analyze the biased effects caused by the static weighting strategy employed in previous approaches. To overcome this limitation, we propose a novel down-sampling paradigm in the Fourier domain, abbreviated as FouriDown, which unifies existing down-sampling techniques. Drawing inspiration from the signal sampling theorem, we parameterize the non-parameter static weighting down-sampling operator as a learnable and context-adaptive operator within a unified Fourier function. Specifically, we organize the corresponding frequency positions of the 2D plane in a physically-closed manner within a single channel dimension. We then perform point-wise channel shuffling based on an indicator that determines whether a channel's signal frequency bin is susceptible to aliasing, ensuring the consistency of the weighting parameter learning. FouriDown, as a generic operator, comprises four key components: 2D discrete Fourier transform, context shuffling rules, Fourier weighting-adaptively superposing rules, and 2D inverse Fourier transform. These components can be easily integrated into existing image restoration networks. To demonstrate the efficacy of FouriDown, we conduct extensive experiments on image de-blurring and low-light image enhancement. The results consistently show that FouriDown can provide significant performance improvements. We will make the code publicly available to facilitate further exploration and application of FouriDown.

PDF Details

AAAI Conference 2023 Conference Paper

KPT: Keyword-Guided Pre-training for Grounded Dialog Generation

  • Qi Zhu
  • Fei Mi
  • Zheng Zhang
  • Yasheng Wang
  • Yitong Li
  • Xin Jiang
  • Qun Liu
  • Xiaoyan Zhu

Incorporating external knowledge into the response generation process is essential to building more helpful and reliable dialog agents. However, collecting knowledge-grounded conversations is often costly, calling for a better pre-trained model for grounded dialog generation that generalizes well w.r.t. different types of knowledge. In this work, we propose KPT (Keyword-guided Pre-Training), a novel self-supervised pre-training method for grounded dialog generation without relying on extra knowledge annotation. Specifically, we use a pre-trained language model to extract the most uncertain tokens in the dialog as keywords. With these keywords, we construct two kinds of knowledge and pre-train a knowledge-grounded response generation model, aiming at handling two different scenarios: (1) the knowledge should be faithfully grounded; (2) it can be selectively used. For the former, the grounding knowledge consists of keywords extracted from the response. For the latter, the grounding knowledge is additionally augmented with keywords extracted from other utterances in the same dialog. Since the knowledge is extracted from the dialog itself, KPT can be easily performed on a large volume and variety of dialogue data. We considered three data sources (open-domain, task-oriented, conversational QA) with a total of 2.5M dialogues. We conduct extensive experiments on various few-shot knowledge-grounded generation tasks, including grounding on dialog acts, knowledge graphs, persona descriptions, and Wikipedia passages. Our comprehensive experiments and analyses demonstrate that KPT consistently outperforms state-of-the-art methods on these tasks with diverse grounding knowledge.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Learning Semantic Degradation-Aware Guidance for Recognition-Driven Unsupervised Low-Light Image Enhancement

  • Naishan Zheng
  • Jie Huang
  • Man Zhou
  • Zizheng Yang
  • Qi Zhu
  • Feng Zhao

Low-light images suffer severe degradation of low lightness and noise corruption, causing unsatisfactory visual quality and visual recognition performance. To solve this problem while meeting the unavailability of paired datasets in wide-range scenarios, unsupervised low-light image enhancement (ULLIE) techniques have been developed. However, these methods are primarily guided to alleviate the degradation effect on visual quality rather than semantic levels, hence limiting their performance in visual recognition tasks. To this end, we propose to learn a Semantic Degradation-Aware Guidance (SDAG) that perceives the low-light degradation effect on semantic levels in a self-supervised manner, which is further utilized to guide the ULLIE methods. The proposed SDAG utilizes the low-light degradation factors as augmented signals to degrade the low-light images, and then capture their degradation effect on semantic levels. Specifically, our SDAG employs the subsequent pre-trained recognition model extractor to extract semantic representations, and then learns to self-reconstruct the enhanced low-light image and its augmented degraded images. By constraining the relative reconstruction effect between the original enhanced image and the augmented formats, our SDAG learns to be aware of the degradation effect on semantic levels in a relative comparison manner. Moreover, our SDAG is general and can be plugged into the training paradigm of the existing ULLIE methods. Extensive experiments demonstrate its effectiveness for improving the ULLIE approaches on the downstream recognition tasks while maintaining a competitive visual quality. Code will be available at https://github.com/zheng980629/SDAG.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

May the Force be with You: Unified Force-Centric Pre-Training for 3D Molecular Conformations

  • Rui Feng
  • Qi Zhu
  • Huan Tran
  • Binghong Chen
  • Aubrey Toland
  • Rampi Ramprasad
  • Chao Zhang

Recent works have shown the promise of learning pre-trained models for 3D molecular representation. However, existing pre-training models focus predominantly on equilibrium data and largely overlook off-equilibrium conformations. It is challenging to extend these methods to off-equilibrium data because their training objective relies on assumptions ofconformations being the local energy minima. We address this gap by proposing a force-centric pretraining model for 3D molecular conformations covering both equilibrium and off-equilibrium data. For off-equilibrium data, our model learns directly from their atomic forces. For equilibrium data, we introduce zero-force regularization and forced-based denoising techniques to approximate near-equilibrium forces. We obtain a unified pre-trained model for 3D molecular representation with over 15 million diverse conformations. Experiments show that, with our pre-training objective, we increase forces accuracy by around 3 times compared to the un-pre-trained Equivariant Transformer model. By incorporating regularizations on equilibrium data, we solved the problem of unstable MD simulations in vanilla Equivariant Transformers, achieving state-of-the-art simulation performance with 2. 45 times faster inference time than NequIP. As a powerful molecular encoder, our pre-trained model achieves on-par performance with state-of-the-art property prediction tasks.

PDF Details

TMLR Journal 2023 Journal Article

Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph

  • Haonan Wang
  • Jieyu Zhang
  • Qi Zhu
  • Wei Huang
  • Kenji Kawaguchi
  • Xiaokui Xiao

Existing graph contrastive learning (GCL) techniques typically require two forward passes for a single instance to construct the contrastive loss, which is effective for capturing the low-frequency signals of node features. Such a dual-pass design has shown empirical success on homophilic graphs, but its effectiveness on heterophilic graphs, where directly connected nodes typically have different labels, is unknown. In addition, existing GCL approaches fail to provide strong performance guarantees. Coupled with the unpredictability of GCL approaches on heterophilic graphs, their applicability in real-world contexts is limited. Then, a natural question arises: Can we design a GCL method that works for both homophilic and heterophilic graphs with a performance guarantee? To answer this question, we theoretically study the concentration property of features obtained by neighborhood aggregation on homophilic and heterophilic graphs, introduce the single-pass graph contrastive learning loss based on the property, and provide performance guarantees for the minimizer of the loss on downstream tasks. As a direct consequence of our analysis, we implement the Single-Pass Graph Contrastive Learning method (SP-GCL). Empirically, on 14 benchmark datasets with varying degrees of homophily, the features learned by the SP-GCL can match or outperform existing strong baselines with significantly less computational overhead, which demonstrates the usefulness of our findings in real-world cases.

PDF Details

AAAI Conference 2021 Conference Paper

Addressing Class Imbalance in Federated Learning

  • Lixu Wang
  • Shichao Xu
  • Xiao Wang
  • Qi Zhu

Federated learning (FL) is a promising approach for training decentralized data located on local client devices while improving efficiency and privacy. However, the distribution and quantity of the training data on the clients’ side may lead to significant challenges such as class imbalance and non- IID (non-independent and identically distributed) data, which could greatly impact the performance of the common model. While much effort has been devoted to helping FL models converge when encountering non-IID data, the imbalance issue has not been sufficiently addressed. In particular, as FL training is executed by exchanging gradients in an encrypted form, the training data is not completely observable to either clients or server, and previous methods for class imbalance do not perform well for FL. Therefore, it is crucial to design new methods for detecting class imbalance in FL and mitigating its impact. In this work, we propose a monitoring scheme that can infer the composition of training data for each FL round, and design a new loss function — Ratio Loss to mitigate the impact of the imbalance. Our experiments demonstrate the importance of acknowledging class imbalance and taking measures as early as possible in FL training, and the effectiveness of our method in mitigating the impact. Our method is shown to significantly outperform previous methods, while maintaining client privacy.

PDF Details

IJCAI Conference 2021 Conference Paper

Chop Chop BERT: Visual Question Answering by Chopping VisualBERT’s Heads

  • Chenyu Gao
  • Qi Zhu
  • Peng Wang
  • Qi Wu

Vision-and-Language (VL) pre-training has shown great potential on many related downstream tasks, such as Visual Question Answering (VQA), one of the most popular problems in the VL field. All of these pre-trained models (such as VisualBERT, ViLBERT, LXMERT and UNITER) are built with Transformer, which extends the classical attention mechanism to multiple layers and heads. To investigate why and how these models work on VQA so well, in this paper we explore the roles of individual heads and layers in Transformer models when handling 12 different types of questions. Specifically, we manually remove (chop) heads (or layers) from a pre-trained VisualBERT model at a time, and test it on different levels of questions to record its performance. As shown in the interesting echelon shape of the result matrices, experiments reveal different heads and layers are responsible for different question types, with higher-level layers activated by higher-level visual reasoning questions. Based on this observation, we design a dynamic chopping module that can automatically remove heads and layers of the VisualBERT at an instance level when dealing with different questions. Our dynamic chopping module can effectively reduce the parameters of the original model by 50%, while only damaging the accuracy by less than 1% on the VQA task.

PDF Details DOI

YNIMG Journal 2021 Journal Article

Common functional localizers to enhance NHP & cross-species neuroscience imaging research

  • Brian E Russ
  • Christopher I Petkov
  • Sze Chai Kwok
  • Qi Zhu
  • Pascal Belin
  • Wim Vanduffel
  • Suliann Ben Hamed

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Details DOI

YNIMG Journal 2021 Journal Article

Minimal specifications for non-human primate MRI: Challenges in standardizing and harmonizing data collection

  • Joonas A. Autio
  • Qi Zhu
  • Xiaolian Li
  • Matthew F. Glasser
  • Caspar M. Schwiedrzik
  • Damien A. Fair
  • Jan Zimmermann
  • Essa Yacoub

Recent methodological advances in MRI have enabled substantial growth in neuroimaging studies of non-human primates (NHPs), while open data-sharing through the PRIME-DE initiative has increased the availability of NHP MRI data and the need for robust multi-subject multi-center analyses. Streamlined acquisition and analysis protocols would accelerate and improve these efforts. However, consensus on minimal standards for data acquisition protocols and analysis pipelines for NHP imaging remains to be established, particularly for multi-center studies. Here, we draw parallels between NHP and human neuroimaging and provide minimal guidelines for harmonizing and standardizing data acquisition. We advocate robust translation of widely used open-access toolkits that are well established for analyzing human data. We also encourage the use of validated, automated pre-processing tools for analyzing NHP data sets. These guidelines aim to refine methodological and analytical strategies for small and large-scale NHP neuroimaging data. This will improve reproducibility of results, and accelerate the convergence between NHP and human neuroimaging strategies which will ultimately benefit fundamental and translational brain science.

Details DOI

NeurIPS Conference 2021 Conference Paper

Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data

  • Qi Zhu
  • Natalia Ponomareva
  • Jiawei Han
  • Bryan Perozzi

There has been a recent surge of interest in designing Graph Neural Networks (GNNs) for semi-supervised learning tasks. Unfortunately this work has assumed that the nodes labeled for use in training were selected uniformly at random (i. e. are an IID sample). However in many real world scenarios gathering labels for graph nodes is both expensive and inherently biased -- so this assumption can not be met. GNNs can suffer poor generalization when this occurs, by overfitting to superfluous regularities present in the training data. In this work we present a method, Shift-Robust GNN (SR-GNN), designed to account for distributional differences between biased training data and the graph's true inference distribution. SR-GNN adapts GNN models for the presence of distributional shifts between the nodes which have had labels provided for training and the rest of the dataset. We illustrate the effectiveness of SR-GNN in a variety of experiments with biased training datasets on common GNN benchmark datasets for semi-supervised learning, where we see that SR-GNN outperforms other GNN baselines by accuracy, eliminating at least (~40%) of the negative effects introduced by biased training data. On the largest dataset we consider, ogb-arxiv, we observe an 2% absolute improvement over the baseline and reduce 30% of the negative effects.

PDF Details

AAAI Conference 2021 Conference Paper

Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps

  • Qi Zhu
  • Chenyu Gao
  • Peng Wang
  • Qi Wu

Texts appearing in daily scenes that can be recognized by OCR (Optical Character Recognition) tools contain significant information, such as street name, product brand and prices. Two tasks – text-based visual question answering and text-based image captioning, with a text extension from existing vision-language applications, are catching on rapidly. To address these problems, many sophisticated multi-modality encoding frameworks (such as heterogeneous graph structure) are being used. In this paper, we argue that a simple attention mechanism can do the same or even better job without any bells and whistles. Under this mechanism, we simply split OCR token features into separate visual- and linguisticattention branches, and send them to a popular Transformer decoder to generate answers or captions. Surprisingly, we find this simple baseline model is rather strong – it consistently outperforms state-of-the-art (SOTA) models on two popular benchmarks, TextVQA and all three tasks of ST-VQA, although these SOTA models use far more complex encoding mechanisms. Transferring it to text-based image captioning, we also surpass the TextCaps Challenge 2020 winner. We wish this work to set the new baseline for these two OCR text related applications and to inspire new thinking of multi-modality encoder design. Code is available at https: //github. com/ZephyrZhuQi/ssbaseline

PDF Details

NeurIPS Conference 2021 Conference Paper

Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization

  • Qi Zhu
  • Carl Yang
  • Yidan Xu
  • Haonan Wang
  • Chao Zhang
  • Jiawei Han

Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards their transferability. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of EGI (Ego-Graph Information maximization) to analytically achieve this goal. Secondly, when node features are structure-relevant, we conduct an analysis of EGI transferability regarding the difference between the local graph Laplacians of the source and target graphs. We conduct controlled synthetic experiments to directly justify our theoretical conclusions. Comprehensive experiments on two real-world network datasets show consistent results in the analyzed setting of direct-transfering, while those on large-scale knowledge graphs show promising results in the more practical setting of transfering with fine-tuning.

PDF Details

JBHI Journal 2020 Journal Article

Coherent Pattern in Multi-Layer Brain Networks: Application to Epilepsy Identification

  • Jiashuang Huang
  • Qi Zhu
  • Mingliang Wang
  • Luping Zhou
  • Zhiqiang Zhang
  • Daoqiang Zhang

Currently, how to conjointly fuse structural connectivity (SC) and functional connectivity (FC) for identifying brain diseases is a hot topic in the area of brain network analysis. Most of the existing works combine two types of connectivity in decision level, thus ignoring the underlying relationship between SC and FC. To solve this problem, in this paper, we model the brain network as the multi-layer network formed by the SC and FC, and then propose a coherent pattern to represent structural information of the multi-layer network for the brain disease identification. The proposed coherent pattern consists of a paired-subgraph extracted from the FC and SC within the same node-set. Compared with the previous methods, this coherent pattern not only describes the connectivity information of both SC and FC by subgraphs at each layer, but also reflects their intrinsic relationship by the co-occurrence pattern of the paired-subgraph. Based on this coherent pattern, we further develop a framework for identifying brain diseases. Specifically, we first construct multi-layer networks by using SC and FC for each subject and then mine coherent patterns that frequently appear in each group. Next, we select the discriminative coherent pattern from these frequent coherent patterns according to their frequency of occurrence. Finally, we construct a feature matrix for each subject based on the binary indicator vector and then use the support vector machine (SVM) as its classifier. Experimental results on real epilepsy datasets demonstrate that our method outperforms several state-of-the-art approaches in the tasks of brain disease classification.

Details DOI

JBHI Journal 2019 Journal Article

Identifying Resting-State Multifrequency Biomarkers via Tree-Guided Group Sparse Learning for Schizophrenia Classification

  • Jiashuang Huang
  • Qi Zhu
  • Xiaoke Hao
  • XiaoMeng Shi
  • Shuzhan Gao
  • Xijia Xu
  • Daoqiang Zhang

The fractional amplitude of low-frequency fluctuations (fALFF) has been widely used as potential clinical biomarkers for resting-state functional-magnetic-resonance-imaging-based schizophrenia diagnosis. How-ever, previous studies usually measure the fALFF with specific bands from 0. 01 to 0. 08 Hz, which cannot fully delineate the complex variations of spontaneous fluctuations in the resting-state brain. In addition, fALFF data are intrinsically constrained by the brain structure, but most of the traditional methods have not consider it in feature selection. For addressing these problems, we propose a model to classify schizophrenia in multifrequency bands with tree-guided group sparse learning. In detail, we first acquire the fALFF data in multifrequency bands (i. e. , slow-5: 0. 01-0. 027 Hz, slow-4: 0. 027-0. 073 Hz, slow-3: 0. 073-0. 198 Hz, and slow-2: 0. 198-0. 25 Hz). Then, we divide the whole brain into different candidate patches and select those significant patches related to schizophrenia using random forest-based important score. Moreover, we use tree-structured sparse learning method for feature selection with the above patch spatial constraint. Finally, considering biomarkers from multifrequency bands can reflect complementary information among multiple-frequency bands, we adopt the multikernel learning method to combine features of multifrequency bands for classification. Our experimental results show that these biomarkers from multifrequency bands can achieve a classification accuracy of 91. 1% on 17 schizophrenia patients and 17 healthy controls, further demonstrating that the multifrequency bands analysis can better account for classification of schizophrenia.

Details DOI

YNIMG Journal 2015 Journal Article

Lateralization for dynamic facial expressions in human superior temporal sulcus

  • François-Laurent De Winter
  • Qi Zhu
  • Jan Van den Stock
  • Koen Nelissen
  • Ronald Peeters
  • Beatrice de Gelder
  • Wim Vanduffel
  • Mathieu Vandenbulcke

Most face processing studies in humans show stronger activation in the right compared to the left hemisphere. Evidence is largely based on studies with static stimuli focusing on the fusiform face area (FFA). Hence, the pattern of lateralization for dynamic faces is less clear. Furthermore, it is unclear whether this property is common to human and non-human primates due to predisposing processing strategies in the right hemisphere or that alternatively left sided specialization for language in humans could be the driving force behind this phenomenon. We aimed to address both issues by studying lateralization for dynamic facial expressions in monkeys and humans. Therefore, we conducted an event-related fMRI experiment in three macaques and twenty right handed humans. We presented human and monkey dynamic facial expressions (chewing and fear) as well as scrambled versions to both species. We studied lateralization in independently defined face-responsive and face-selective regions by calculating a weighted lateralization index (LIwm) using a bootstrapping method. In order to examine if lateralization in humans is related to language, we performed a separate fMRI experiment in ten human volunteers including a ‘speech’ expression (one syllable non-word) and its scrambled version. Both within face-responsive and selective regions, we found consistent lateralization for dynamic faces (chewing and fear) versus scrambled versions in the right human posterior superior temporal sulcus (pSTS), but not in FFA nor in ventral temporal cortex. Conversely, in monkeys no consistent pattern of lateralization for dynamic facial expressions was observed. Finally, LIwms based on the contrast between different types of dynamic facial expressions (relative to scrambled versions) revealed left-sided lateralization in human pSTS for speech-related expressions compared to chewing and emotional expressions. To conclude, we found consistent laterality effects in human posterior STS but not in visual cortex of monkeys. Based on our results, it is tempting to speculate that lateralization for dynamic face processing in humans may be driven by left-hemispheric language specialization which may not have been present yet in the common ancestor of human and macaque monkeys.

Details DOI

YNIMG Journal 2013 Journal Article

Dissimilar processing of emotional facial expressions in human and monkey temporal cortex

  • Qi Zhu
  • Koen Nelissen
  • Jan Van den Stock
  • François-Laurent De Winter
  • Karl Pauwels
  • Beatrice de Gelder
  • Wim Vanduffel
  • Mathieu Vandenbulcke

Emotional facial expressions play an important role in social communication across primates. Despite major progress made in our understanding of categorical information processing such as for objects and faces, little is known, however, about how the primate brain evolved to process emotional cues. In this study, we used functional magnetic resonance imaging (fMRI) to compare the processing of emotional facial expressions between monkeys and humans. We used a 2×2×2 factorial design with species (human and monkey), expression (fear and chewing) and configuration (intact versus scrambled) as factors. At the whole brain level, neural responses to conspecific emotional expressions were anatomically confined to the superior temporal sulcus (STS) in humans. Within the human STS, we found functional subdivisions with a face-selective right posterior STS area that also responded to emotional expressions of other species and a more anterior area in the right middle STS that responded specifically to human emotions. Hence, we argue that the latter region does not show a mere emotion-dependent modulation of activity but is primarily driven by human emotional facial expressions. Conversely, in monkeys, emotional responses appeared in earlier visual cortex and outside face-selective regions in inferior temporal cortex that responded also to multiple visual categories. Within monkey IT, we also found areas that were more responsive to conspecific than to non-conspecific emotional expressions but these responses were not as specific as in human middle STS. Overall, our results indicate that human STS may have developed unique properties to deal with social cues such as emotional expressions.

Details DOI

IROS Conference 2004 Conference Paper

Max-plus algebra model for on-line task scheduling of a reconfigurable manufacturing work-cell

  • Qi Zhu
  • Weihua Sheng
  • Ning Xi 0001

The timed Petri net model introduces the possibility of applying a set of mathematical results, mainly based on the use of max-plus algebra, for performance analysis. This paper aims to build a new scheduling method for a flexible manufacturing work-cell by merging the timed Petri net model and max-plus algebra. The results can be computed as functions of a certain set of decision parameters. These functions can be used to schedule, plan and control the flexible manufacturing work-cell, so that it can real-timely adapt itself to machine faults.

Details
v2026.09.13