Author name cluster

Zhihai He

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

JBHI Journal 2026 Journal Article

Audio-Driven Multi-Modal Unobtrusive Health Monitoring and Inference for Smart Eldercare at Home

Xinhua Fan
Youming Li
Zhongchao Huang
Zhihai He

As the aging population grows and more elderly individuals live independently, the demand for reliable, unobtrusive home health monitoring becomes increasingly important. Existing in-home health monitoring systems often face limitations such as privacy concerns, dependence on unreliable wearable devices, degraded accuracy in complex environments, and lack of continuous monitoring capability. To address these challenges, we propose a long-term home health monitoring system that primarily relies on audio sensing, supplemented by other noninvasive modalities. Our approach is able to accurately detect and recognize overlapping acoustic events with fine-grained temporal resolution, surpassing conventional audio-based methods for activity recognition. The system incorporates a transformer-based time-frequency fusion module and a category dynamic threshold strategy to improve detection performance under semi supervised conditions. Experiments on real-world dataset demonstrate that our method outperforms existing baselines, achieving PSDS $_{1}$, PSDS $_{2}$, and EB-F1 scores of 0. 581, 0. 930 and 55. 1%, with improvements of 0. 054, 0. 019, and 2. 3%, respectively. In addition, a 30 day field deployment involving 10 elderly participants confirms the robustness and practicality of the system for real-world applications. By allowing continuous passive monitoring of daily activities and abnormal acoustic events, our system has significant potentials for early detection of health risks, behavioral anomalies, and long-term wellness tracking in aging in place scenarios.

Details DOI

AAAI Conference 2025 Conference Paper

Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression

Siqi Wu
Yinda Chen
Dong Liu
Zhihai He

In this paper, we study how to synthesize a dynamic reference from an external dictionary to perform conditional coding of the input image in the latent domain and how to learn the conditional latent synthesis and coding modules in an end-to-end manner. Our approach begins by constructing a universal image feature dictionary using a multi-stage approach involving modified spatial pyramid pooling, dimension reduction, and multi-scale feature clustering. For each input image, we learn to synthesize a conditioning latent by selecting and synthesizing relevant features from the dictionary, which significantly enhances the model's capability in capturing and exploring image source correlation. This conditional latent synthesis involves a correlation-based feature matching and alignment strategy, comprising a Conditional Latent Matching (CLM) module and a Conditional Latent Synthesis (CLS) module. The synthesized latent is then used to guide the encoding process, allowing for more efficient compression by exploiting the correlation between the input image and the reference dictionary. According to our theoretical analysis, the proposed conditional latent coding (CLC) method is robust to perturbations in the external dictionary samples and the selected conditioning latent, with an error bound that scales logarithmically with the dictionary size, ensuring stability even with large and diverse dictionaries. Experimental results on benchmark datasets show that our new method improves the coding performance by a large margin (up to 1.2 dB) with a very small overhead of approximately 0.5% bits per pixel.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations

Yi Zhang
Chun-Wun Cheng
Junyi He
Zhihai He
Carola-Bibiane Schönlieb
Yuyan Chen
Angelica I Aviles-Rivero

We introduce SONO, a novel method leveraging Second-Order Neural Ordinary Differential Equations (Second-Order NODEs) to enhance cross-modal few-shot learning. By employing a simple yet effective architecture consisting of a Second-Order NODEs model paired with a cross-modal classifier, SONO addresses the significant challenge of overfitting, which is common in few-shot scenarios due to limited training examples. Our second-order approach can approximate a broader class of functions, enhancing the model's expressive power and feature generalization capabilities. We initialize our cross-modal classifier with text embeddings derived from class-relevant prompts, streamlining training efficiency by avoiding the need for frequent text encoder processing. Additionally, we utilize text-based image augmentation, exploiting CLIP’s robust image-text correlation to enrich training data significantly. Extensive experiments across multiple datasets demonstrate that SONO outperforms existing state-of-the-art methods in few-shot learning performance.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Concept-Guided Prompt Learning for Generalization in Vision-Language Models

Yi Zhang
Ce Zhang
Ke Yu
Yushun Tang
Zhihai He

Contrastive Language-Image Pretraining (CLIP) model has exhibited remarkable efficacy in establishing cross-modal connections between texts and images, yielding impressive performance across a broad spectrum of downstream applications through fine-tuning. However, for generalization tasks, the current fine-tuning methods for CLIP, such as CoOp and CoCoOp, demonstrate relatively low performance on some fine-grained datasets. We recognize the underlying reason is that these previous methods only projected global features into the prompt, neglecting the various visual concepts, such as colors, shapes, and sizes, which are naturally transferable across domains and play a crucial role in generalization tasks. To address this issue, in this work, we propose Concept-Guided Prompt Learning (CPL) for vision-language models. Specifically, we leverage the well-learned knowledge of CLIP to create a visual concept cache to enable conceptguided prompting. In order to refine the text features, we further develop a projector that transforms multi-level visual features into text features. We observe that this concept-guided prompt learning approach is able to achieve enhanced consistency between visual and linguistic modalities. Extensive experimental results demonstrate that our CPL method significantly improves generalization capabilities compared to the current state-of-the-art methods.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Cross-Constrained Progressive Inference for 3D Hand Pose Estimation with Dynamic Observer-Decision-Adjuster Networks

Zhehan Kan
Xueting Hu
Zihan Liao
Ke Yu
Zhihai He

Generalization is very important for pose estimation, especially for 3D pose estimation where small changes in the 2D images could trigger structural changes in the 3D space. To achieve generalization, the system needs to have the capability of detecting estimation errors by double-checking the projection coherence between the 3D and 2D spaces and adapting its network inference process based on this feedback. Current pose estimation is one-time feed-forward and lacks the capability to gather feedback and adapt the inference outcome. To address this problem, we propose to explore the concept of progressive inference where the network learns an observer to continuously detect the prediction error based on constraints matching, as well as an adjuster to refine its inference outcome based on these constraints errors. Within the context of 3D hand pose estimation, we find that this observer-adjuster design is relatively unstable since the observer is operating in the 2D image domain while the adjuster is operating in the 3D domain. To address this issue, we propose to construct two sets of observers-adjusters with complementary constraints from different perspectives. They operate in a dynamic sequential manner controlled by a decision network to progressively improve the 3D pose estimation. We refer to this method as Cross-Constrained Progressive Inference (CCPI). Our extensive experimental results on FreiHAND and HO-3D benchmark datasets demonstrate that the proposed CCPI method is able to significantly improve the generalization capability and performance of 3D hand pose estimation.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Coded Residual Transform for Generalizable Deep Metric Learning

Shichao Kan
Yixiong Liang
Min Li
Yigang Cen
Jianxin Wang
Zhihai He

A fundamental challenge in deep metric learning is the generalization capability of the feature embedding network model since the embedding network learned on training classes need to be evaluated on new test classes. To address this challenge, in this paper, we introduce a new method called coded residual transform (CRT) for deep metric learning to significantly improve its generalization capability. Specifically, we learn a set of diversified prototype features, project the feature map onto each prototype, and then encode its features using their projection residuals weighted by their correlation coefficients with each prototype. The proposed CRT method has the following two unique characteristics. First, it represents and encodes the feature map from a set of complimentary perspectives based on projections onto diversified prototypes. Second, unlike existing transformer-based feature representation approaches which encode the original values of features based on global correlation analysis, the proposed coded residual transform encodes the relative differences between the original features and their projected prototypes. Embedding space density and spectral decay analysis show that this multi perspective projection onto diversified prototypes and coded residual representation are able to achieve significantly improved generalization capability in metric learning. Finally, to further enhance the generalization performance, we propose to enforce the consistency on their feature similarity matrices between coded residual transforms with different sizes of projection prototypes and embedding dimensions. Our extensive experimental results and ablation studies demonstrate that the proposed CRT method outperform the state-of-the-art deep metric learning methods by large margins and improving upon the current best method by up to 4. 28% on the CUB dataset.

PDF Details