Hypergraph-driven soft semantics flexible learning for visible–infrared person re-identification

Visible–infrared person re-identification (VI-ReID) aims to match the images of the person across different modalities. The main challenge in the engineering application of VI-ReID lies in the considerable modality gap between visible and infrared images. Existing methods primarily focus on learning low-order hard semantics to reduce the modality gap, such as specific body parts (e. g. , arms, legs, and hands), which are modality-specific and sensitive to cross-modality variations, leading to difficult modality alignment. However, high-order soft semantics contain more modality-invariant features that can be flexibly learned by the model to better align the cross-modality features and reduce the modality gap. To better learn soft semantics, we propose a novel, hypergraph-driven soft semantics flexible learning network (HSFLNet), which extracts hierarchical semantics and flexibly explores the relationships among the extracted soft semantics by using hypergraph neural networks (HGNNs). Specifically, first, we propose a soft semantics mining (SSM) module to capture and flexibly fuse hierarchical features, which integrates low- and high-level semantics across channel and spatial dimensions to align modalities. Second, a hypergraph-driven soft semantics flexible learning (HSFL) module is designed, which employs HGNNs to explore the relationships among soft semantics. These semantics are adaptively learned through gating mechanisms to capture modality-invariant features, thereby reducing the modality gap. We evaluate the performance of HSFLNet on SYSU-MM01, RegDB, and low-light cross-modality (LLCM) datasets, achieving a Rank-1/mean average precision of 75. 09%/72. 11%, 95. 82%/92. 42%, and 65. 42%/68. 55%, respectively. Experimental results demonstrate that HSFLNet outperforms state-of-the-art VI-ReID methods. Our code is available at https: //github. com/Jiacheng813/HSFLNet.

Authors

• Jiacheng Zhu
• Hongwei Ge
• Yuxuan Liu Gaoling School of Artificial Intelligence, Renmin University of China
• Chunguo Wu
• Jiulin Fan

Keywords

• Visible–infrared person re-identification
• Soft semantics
• Hypergraph neural networks
• Cross-modality retrieval