Composition-Incremental Learning for Compositional Generalization

Compositional generalization has achieved substantial progress in computer vision on pre-collected training data. Nonetheless, real-world data continually emerges, with possible compositions being nearly infinite, long-tailed, and not entirely visible. Thus, an ideal model is supposed to gradually improve the capability of compositional generalization in an incremental manner. In this paper, we explore Composition-Incremental Learning for Compositional Generalization (CompIL) in the context of the compositional zero-shot learning (CZSL) task, where models need to continually learn new compositions, intending to improve their compositional generalization capability progressively. To quantitatively evaluate CompIL, we develop a benchmark construction pipeline leveraging existing datasets, yielding MIT-States-CompIL and C-GQA-CompIL. Furthermore, we propose a pseudo-replay framework utilizing a visual synthesizer to synthesize visual representations of learned compositions and a linguistic primitive distillation mechanism to maintain aligned primitive representations across the learning process. Extensive experiments demonstrate the effectiveness of the proposed framework.

Authors

• Zhen Li Beijing Institute of Technology
• Yuwei Wu Beijing Institute of Technology Shenzhen MSU-BIT University
• Chenchen Jing Zhejiang University
• Che Sun Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology, China
• Chuanhao Li Beijing Institute of Technology Shenzhen MSU-BIT University
• Yunde Jia Shenzhen MSU-BIT University Beijing Institute of Technology

Keywords

No keywords are indexed for this paper.