Continual Multimodal Knowledge Graph Construction

Current Multimodal Knowledge Graph Construction (MKGC) models struggle with the real-world dynamism of continuously emerging entities and relations, often succumbing to catastrophic forgetting—loss of previously acquired knowledge. This study introduces benchmarks aimed at fostering the development of the continual MKGC domain. We further introduce the MSPT framework, designed to surmount the shortcomings of existing MKGC approaches during multimedia data processing. MSPT harmonizes the retention of learned knowledge (stability) and the integration of new data (plasticity), outperforming current continual learning and multimodal methods. Our results confirm MSPT's superior performance in evolving knowledge environments, showcasing its capacity to navigate the balance between stability and plasticity.

Authors

• Xiang Chen Zhejiang University
• Jingtian Zhang
• Xiaohan Wang Stanford University
• Ningyu Zhang Zhejiang University Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph
• Tongtong Wu Southeast University School of Computer Science and Engineering
• Yuxiang Wang The University of Melbourne
• Yongheng Wang Zhejiang lab
• Huajun Chen College of Computer Science and Technology, Zhejiang University ZJU-Ant Group Joint Lab of Knowledge Graph Zhejiang Key Laboratory of Big Data Intelligent Computing

Keywords

• Data Mining: DM: Knowledge graphs and knowledge base completion
• Natural Language Processing: NLP: Information extraction
• Natural Language Processing: NLP: Named entities