Author name cluster

Hong Zhou

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

2 author rows

JBHI Journal 2025 Journal Article

MACTFusion: Lightweight Cross Transformer for Adaptive Multimodal Medical Image Fusion

Xinyu Xie
Xiaozhi Zhang
Xinglong Tang
Jiaxi Zhao
Dongping Xiong
Lijun Ouyang
Bin Yang
Hong Zhou

Multimodal medical image fusion aims to integrate complementary information from different modalities of medical images. Deep learning methods, especially recent vision Transformers, have effectively improved image fusion performance. However, there are limitations for Transformers in image fusion, such as lacks of local feature extraction and cross-modal feature interaction, resulting in insufficient multimodal feature extraction and integration. In addition, the computational cost of Transformers is higher. To address these challenges, in this work, we develop an adaptive cross-modal fusion strategy for unsupervised multimodal medical image fusion. Specifically, we propose a novel lightweight cross Transformer based on cross multi-axis attention mechanism. It includes cross-window attention and cross-grid attention to mine and integrate both local and global interactions of multimodal features. The cross Transformer is further guided by a spatial adaptation fusion module, which allows the model to focus on the most relevant information. Moreover, we design a special feature extraction module that combines multiple gradient residual dense convolutional and Transformer layers to obtain local features from coarse to fine and capture global features. The proposed strategy significantly boosts the fusion performance while minimizing computational costs. Extensive experiments, including clinical brain tumor image fusion, have shown that our model can achieve clearer texture details and better visual quality than other state-of-the-art fusion methods.

Details DOI

JBHI Journal 2025 Journal Article

PubLabeler: Enhancing Automatic Classification of Publications in UniProtKB Using Protein Textual Description and PubMedBERT

Shaojun Wang
Junyi Bian
Xiaodi Huang
Hong Zhou
Shanfeng Zhu

In UniProtKB, each protein is linked to numerous publications covering topics such as sequence, function, and structure, which are annotated manually or through automated methods. Given the vast number of proteins and literature, manual annotation is time-consuming and labour-intensive. Although UniProtKB offers automated annotations, their quality often falls short. Therefore, developing an accurate automated classifier to identify the topics of publications associated with each protein is imperative for advancing biomedical knowledge discovery. Classifying publications in UniProtKB involves protein-publication pairs characterized by multi-label, label co-occurrence, and class imbalance, which increases complexity. This paper proposes a novel method called PubLabeler, which simultaneously considers protein description and scientific literature texts as input. PubLabeler employs the PubMedBERT model to encode input texts and integrates label co-occurrence information into the model parameters. Additionally, it uses focal loss to update parameters, allowing the model to focus more on classes with a few instances. Using newly annotated literature from Swiss-Prot in 2023 as a test set, PubLabeler achieved superior results in both micro and macro metrics, showing a 28. 5% improvement in macro-F1 compared to UniProtKB's automated annotation method, UPCLASS. Furthermore, we validated PubLabeler's effectiveness in TrEMBL annotation, showcasing its comprehensive prediction results compared to TrEMBL's automated annotations. These findings highlight PubLabeler's reliability and potential to advance protein-related information extraction and knowledge discovery.

Details DOI

JBHI Journal 2025 Journal Article

SpectFusion: Cross-modal Spectrum-aware Attention Network for Unsupervised Multimodal Medical Image Fusion

Lamei Wang
Xinyu Xie
Youxi Yang
Dongping Xiong
Hong Zhou
Bin Yang
Kok Lay Teo
Bingo Wing-Kuen Ling

Medical image fusion aims to synthesize relevant and complementary information from different modalities, thereby enhancing clinical diagnosis. Current deep learning-based fusion approaches, particularly Transformer-based architectures, have achieved remarkable results due to their strong capacity for modeling long-range dependencies. However, there are still limitations in capturing sufficient global information because of the window-based local attention mechanism. Moreover, existing fusion schemes predominantly focus on spatial features while rarely considering spectral features, thus affecting the fusion performance. To address these challenges, we propose a new unsupervised cross-modal spectrum-aware fusion framework, named SpectFusion, for medical image fusion. Specifically, we devise a spatial-spectrum hybrid block, which effectively extracts fine-grained local features via a gradient retention strategy in the spatial domain, and captures global features with an image-wide receptive field through Fourier convolution in the frequency domain. Furthermore, we develop a novel cross-modal spectrum-aware attention to facilitate spatial-spectrum information interactions during fusion. It dynamically guides the retention of relevant spectral components while integrating multimodal spatial features. Additionally, to achieve more precise alignment image pairs, we incorporate a refined registration module to correct minor local deviations. We also define corresponding frequency and spatial domain losses to jointly constrain the proposed SpectFusion. By leveraging spatial-spectrum information interactions, fine-grained fusion can be adaptively realized. Extensive experiments, including clinical brain tumor image fusion, demonstrate that SpectFusion outperforms other state-of-the-art methods both qualitatively and quantitatively. We show that SpectFusion can boost performance in downstream tasks such as multimodal medical image segmentation. The code is available at https://github.com/PlumW/SpectFusion.

Details DOI

ICML Conference 2025 Conference Paper

ZipAR: Parallel Autoregressive Image Generation through Spatial Locality

Yefei He
Feng Chen
Yuanyu He
Shaoxuan He
Hong Zhou
Kaipeng Zhang
Bohan Zhuang

In this paper, we propose ZipAR, a training-free, plug-and-play parallel decoding framework for accelerating autoregressive (AR) visual generation. The motivation stems from the observation that images exhibit local structures, and spatially distant regions tend to have minimal interdependence. Given a partially decoded set of visual tokens, in addition to the original next-token prediction scheme in the row dimension, the tokens corresponding to spatially adjacent regions in the column dimension can be decoded in parallel. To ensure alignment with the contextual requirements of each token, we employ an adaptive local window assignment scheme with rejection sampling analogous to speculative decoding. By decoding multiple tokens in a single forward pass, the number of forward passes required to generate an image is significantly reduced, resulting in a substantial improvement in generation efficiency. Experiments demonstrate that ZipAR can reduce the number of model forward passes by up to 91% on the Emu3-Gen model without requiring any additional retraining.

Details

ICLR Conference 2024 Conference Paper

EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models

Yefei He
Jing Liu 0048
Weijia Wu 0001
Hong Zhou
Bohan Zhuang

Diffusion models have demonstrated remarkable capabilities in image synthesis and related generative tasks. Nevertheless, their practicality for low-latency real-world applications is constrained by substantial computational costs and latency issues. Quantization is a dominant way to compress and accelerate diffusion models, where post-training quantization (PTQ) and quantization-aware training (QAT) are two main approaches, each bearing its own properties. While PTQ exhibits efficiency in terms of both time and data usage, it may lead to diminished performance in low bit-width settings. On the other hand, QAT can help alleviate performance degradation but comes with substantial demands on computational and data resources. To capitalize on the advantages while avoiding their respective drawbacks, we introduce a data-free, quantization-aware and parameter-efficient fine-tuning framework for low-bit diffusion models, dubbed EfficientDM, to achieve QAT-level performance with PTQ-like efficiency. Specifically, we propose a quantization-aware variant of the low-rank adapter (QALoRA) that can be merged with model weights and jointly quantized to low bit-width. The fine-tuning process distills the denoising capabilities of the full-precision model into its quantized counterpart, eliminating the requirement for training data. To further enhance performance, we introduce scale-aware optimization to address ineffective learning of QALoRA due to variations in weight quantization scales across different layers. We also employ temporal learned step-size quantization to handle notable variations in activation distributions across denoising steps. Extensive experimental results demonstrate that our method significantly outperforms previous PTQ-based diffusion models while maintaining similar time and data efficiency. Specifically, there is only a marginal $0.05$ sFID increase when quantizing both weights and activations of LDM-4 to 4-bit on ImageNet $256\times256$. Compared to QAT-based methods, our EfficientDM also boasts a $16.2\times$ faster quantization speed with comparable generation quality, rendering it a compelling choice for practical applications.

Details

NeurIPS Conference 2024 Conference Paper

Harmonizing Stochasticity and Determinism: Scene-responsive Diverse Human Motion Prediction

Tuo Wang
Zhenbo Song
Zhenyu Lou
Qiongjie Cui
Luoming Zhang
Cheng Cheng
Haofan Wang
Xu Tang

Diverse human motion prediction (HMP) is a fundamental application in computer vision that has recently attracted considerable interest. Prior methods primarily focus on the stochastic nature of human motion, while neglecting the specific impact of external environment, leading to the pronounced artifacts in prediction when applied to real-world scenarios. To fill this gap, this work introduces a novel task: predicting diverse human motion within real-world 3D scenes. In contrast to prior works, it requires harmonizing the deterministic constraints imposed by the surrounding 3D scenes with the stochastic aspect of human motion. For this purpose, we propose DiMoP3D, a diverse motion prediction framework with 3D scene awareness, which leverages the 3D point cloud and observed sequence to generate diverse and high-fidelity predictions. DiMoP3D is able to comprehend the 3D scene, and determines the probable target objects and their desired interactive pose based on the historical motion. Then, it plans the obstacle-free trajectory towards these interested objects, and generates diverse and physically-consistent future motions. On top of that, DiMoP3D identifies deterministic factors in the scene and integrates them into the stochastic modeling, making the diverse HMP in realistic scenes become a controllable stochastic generation process. On two real-captured benchmarks, DiMoP3D has demonstrated significant improvements over state-of-the-art methods, showcasing its effectiveness in generating diverse and physically-consistent motion predictions within real-world 3D environments.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification

Yefei He
Luoming Zhang
Weijia Wu
Jing Liu
Hong Zhou
Bohan Zhuang

KV cache stores key and value states from previous tokens to avoid re-computation, yet it demands substantial storage space, especially for long sequences. Adaptive KV cache compression seeks to discern the saliency of tokens, preserving vital information while aggressively compressing those of less importance. However, previous methods of this approach exhibit significant performance degradation at high compression ratios due to inaccuracies in identifying salient tokens. Additionally, the compression process introduces excessive overhead, substantially increasing memory burdens and the generation latency. In this paper, we present ZipCache, an accurate and efficient KV cache quantization method for large language models (LLMs). First, we construct a strong baseline for quantizing KV cache. Through the proposed channel-separable tokenwise quantization scheme, the memory overhead of quantization parameters are substantially reduced compared to fine-grained groupwise quantization. To enhance the compression ratio, we propose normalized attention score as an effective metric for identifying salient tokens by considering the lower triangle characteristics of the attention matrix. The quantization bit-width for each token is then adaptively assigned based on their saliency. Moreover, we develop an efficient approximation method that decouples the saliency metric from full attention scores, enabling compatibility with fast attention implementations like FlashAttention. Extensive experiments demonstrate that ZipCache achieves superior compression ratios, fast generation speed and minimal performance losses compared with previous KV cache compression methods. For instance, when evaluating Mistral-7B model on GSM8k dataset, ZipCache is capable of compressing the KV cache by $4. 98\times$, with only a 0. 38% drop in accuracy. In terms of efficiency, ZipCache also showcases a 37. 3% reduction in prefill-phase latency, a 56. 9% reduction in decoding-phase latency, and a 19. 8% reduction in GPU memory usage when evaluating LLaMA3-8B model with a input length of 4096. Code is available at https: //github. com/ThisisBillhe/ZipCache/.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models

Weijia Wu
Yuzhong Zhao
Hao Chen
Yuchao Gu
Rui Zhao
Yefei He
Hong Zhou
Mike Zheng Shou

Current deep networks are very data-hungry and benefit from training on large-scale datasets, which are often time-consuming to collect and annotate. By contrast, synthetic data can be generated infinitely using generative models such as DALL-E and diffusion models, with minimal effort and cost. In this paper, we present DatasetDM, a generic dataset generation model that can produce diverse syntheticimages and the corresponding high-quality perception annotations (e. g. , segmentation masks, and depth). Our method builds upon the pre-trained diffusion model and extends text-guided image synthesis to perception data generation. We show that the rich latent code of the diffusion model can be effectively decoded as accurate perception annotations using a decoder module. Training the decoder only needs less than 1% (around 100 images) of manually labeled images, enabling the generation of an infinitely large annotated dataset. Then these synthetic data can be used for training various perception models on downstream tasks. To showcase the power of the proposed approach, we generate datasets with rich dense pixel-wise labels for a wide range of downstream tasks, including semantic15segmentation, instance segmentation, and depth estimation. Notably, it achieves 1) state-of-the-art results on semantic segmentation and instance segmentation; 2) significantly more efficient and robust in domain generalization than the real data; 3) state-of-the-art results in zero-shot segmentation setting; and 4) flexibility for efficient application and novel task composition (e. g. , image editing)

PDF Details

NeurIPS Conference 2023 Conference Paper

PTQD: Accurate Post-Training Quantization for Diffusion Models

Yefei He
Luping Liu
Jing Liu
Weijia Wu
Hong Zhou
Bohan Zhuang

Diffusion models have recently dominated image synthesis and other related generative tasks. However, the iterative denoising process is expensive in computations at inference time, making diffusion models less practical for low-latency and scalable real-world applications. Post-training quantization of diffusion models can significantly reduce the model size and accelerate the sampling process without requiring any re-training. Nonetheless, applying existing post-training quantization methods directly to low-bit diffusion models can significantly impair the quality of generated samples. Specifically, for each denoising step, quantization noise leads to deviations in the estimated mean and mismatches with the predetermined variance schedule. Moreover, as the sampling process proceeds, the quantization noise may accumulate, resulting in a low signal-to-noise ratio (SNR) during the later denoising steps. To address these challenges, we propose a unified formulation for the quantization noise and diffusion perturbed noise in the quantized denoising process. Specifically, we first disentangle the quantization noise into its correlated and residual uncorrelated parts regarding its full-precision counterpart. The correlated part can be easily corrected by estimating the correlation coefficient. For the uncorrelated part, we subtract the bias from the quantized results to correct the mean deviation and calibrate the denoising variance schedule to absorb the excess variance resulting from quantization. Moreover, we introduce a mixed-precision scheme for selecting the optimal bitwidth for each denoising step, which prioritizes lower bitwidths to expedite early denoising steps, while ensuring that higher bitwidths maintain a high signal-to-noise ratio (SNR) in the later steps. Extensive experiments demonstrate that our method outperforms previous post-training quantized diffusion models in generating high-quality samples, with only a $0. 06$ increase in FID score compared to full-precision LDM-4 on ImageNet $256\times256$, while saving $19. 9\times$ bit operations. Code is available at [https: //github. com/ziplab/PTQD](https: //github. com/ziplab/PTQD).

PDF Details

NeurIPS Conference 2021 Conference Paper

A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer

威佳吴
Debing Zhang
Yuanqiang Cai
Sibo Wang
Jiahong Li
Zhuang Li
Yejun Tang
Hong Zhou

Most existing video text spotting benchmarks focus on evaluating a single language and scenario with limited data. In this work, we introduce a large-scale, Bilingual, Open World Video text benchmark dataset(BOVText). There are four features for BOVText. Firstly, we provide 1, 850+ videos with more than 1, 600, 000+ frames, 25 times larger than the existing largest dataset with incidental text in videos. Secondly, our dataset covers 30+ open categories with a wide selection of various scenarios, Life Vlog, Driving, Movie, etc. Thirdly, abundant text types annotation (i. e. , title, caption, or scene text) are provided for the different representational meanings in the video. Fourthly, the MOVText provides multilingual text annotation to promote multiple cultures' live and communication. Besides, we propose an end-to-end video text spotting framework with Transformer, termed TransVTSpotter, which solves the multi-orient text spotting in video with a simple, but efficient attention-based query-key mechanism. It applies object features from the previous frame as a tracking query for the current frame and introduces a rotation angle prediction to fit the multi-orient text instance. On ICDAR2015(video), TransVTSpotter achieves state-of-the-art performance with 44. 2% MOTA, 13 fps. The dataset and code of TransVTSpotter can be found at https: //github. com/weijiawu/BOVText-Benchmark and https: //github. com/weijiawu/TransVTSpotter, respectively.

PDF Details

TIST Journal 2012 Journal Article

Watch the Story Unfold with TextWheel

Weiwei Cui
Huamin Qu
Hong Zhou
Wenbin Zhang
Steve Skiena

Keyword-based searching and clustering of news articles have been widely used for news analysis. However, news articles usually have other attributes such as source, author, date and time, length, and sentiment which should be taken into account. In addition, news articles and keywords have complicated macro/micro relations, which include relations between news articles (i.e., macro relation), relations between keywords (i.e., micro relation), and relations between news articles and keywords (i.e., macro-micro relation). These macro/micro relations are time varying and pose special challenges for news analysis. In this article we present a visual analytics system for news streams which can bring multiple attributes of the news articles and the macro/micro relations between news streams and keywords into one coherent analytical context, all the while conveying the dynamic natures of news streams. We introduce a new visualization primitive called TextWheel which consists of one or multiple keyword wheels, a document transportation belt, and a dynamic system which connects the wheels and belt. By observing the TextWheel and its content changes, some interesting patterns can be detected. We use our system to analyze several news corpora related to some major companies and the results demonstrate the high potential of our method.

Details DOI

EAAI Journal 1993 Journal Article

Knowledge-based automation for energy conservation and indoor air quality control in HVAC processes

Hong Zhou
Ming Rao
Karl T. Chuang

Heating, ventilating, and air conditioning (HVAC) processes provide a comfortable environment, but consume a great deal of energy. Many efforts have been put into building energy conservation since the energy crisis of 1973. On the other hand, the energy conservation efforts have led to tight building envelopes and low ventilation rates, which cause poor indoor air quality (IAQ), the so called “Sick Building Syndrome”. A conflict exists between energy saving and indoor air quality improvement. In this paper, a knowledge-based automation approach is proposed to support HVAC operations, aimed at improving energy conservation and IAQ control. The construction of an Intelligent Operation Support System (IOSS) for HVAC processes is described. It consists of expert systems for operation planning, comfort setting, conflict reasoning and some other functions. The expert system for operation planning provides recommendations of energy saving operation modes for HVAC processes. The comfort indoor setting system sets indoor temperature by a comfort strategy with the advantages of energy saving, thermal comfort and better indoor air quality, instead of thermostats. The conflict reasoning system obtains a conflict resolution for energy saving and IAQ control. An integrated distributed intelligent system framework is introduced to integrate these systems. IOSS provides a real-time integrated operation planning method and can be used to assist or train operators to achieve better operation in HVAC systems.

Details DOI