ECAI Conference 2025 Conference Paper
GEMPA: Graph-Enhanced Multimodal Prototype Alignment for Class Incremental Learning
- Yuling Chen
- Zhenming Zhang
- Chunxia Ren
- Zixuan Zhong
- Li Xiao 0005
Pre-trained models (PTMs), recognized for their generalization capabilities, have been extensively employed in Class Incremental Learning (CIL). However, existing PTM-based CIL methods encounter two critical limitations: 1) suboptimal extraction of discriminative prototypes for classifier construction, and 2) insufficient modeling of inter-class relationships, collectively leading to catastrophic forgetting of prior knowledge when incrementally learning new classes. To address these challenges, we propose Graph-Enhanced Multimodal Prototype Alignment (GEMPA) which synergizes PTM representations with class-specific semantic information. Our framework features two key innovations: First, an Embedding Modulation (EM) module rectifies the imbalanced value distributions (characterized by Leptokurtic distribution) in PTM-generated embeddings. Second, a GCN-based Multimodal Prototype Similarity Alignment (GPA) module establishes a Visual-Semantic Co-occurrence (VSC) space through CLIP-derived semantic embeddings. This VSC space aligns visual prototypes with their corresponding semantic counterparts to model incremental class relationships. A Weighted Hybrid Loss is designed to penalize similar prototypes while maintaining the intra-class distribution in GCN. We incorporate GEMPA with seven PTM-based CIL approaches and validate them across four benchmark datasets. The results demonstrate the improvements through semantic clarity of prototypes and forgetting mitigation. Code is available at: https: //github. com/AI4MyBUPT/GEMPA