A Two-Stage Information Extraction Network for Incomplete Multi-View Multi-Label Classification

Recently, multi-view multi-label classification (MvMLC) has received a significant amount of research interest and many methods have been proposed based on the assumptions of view completion and label completion. However, in real-world scenarios, multi-view multi-label data tends to be incomplete due to various uncertainties involved in data collection and manual annotation. As a result, the conventional MvMLC methods fail. In this paper, we propose a new two-stage MvMLC network to solve this incomplete MvMLC issue with partial missing views and missing labels. Different from the existing works, our method attempts to leverage the diverse information from the partially missing data based on the information theory. Specifically, our method aims to minimize task-irrelevant information while maximizing task-relevant information through the principles of information bottleneck theory and mutual information extraction. The first stage of our network involves training view-specific classifiers to concentrate the task-relevant information. Subsequently, in the second stage, the hidden states of these classifiers serve as input for an alignment model, an autoencoder-based mutual information extraction framework, and a weighted fusion classifier to make the final prediction. Extensive experiments performed on five datasets validate that our method outperforms other state-of-the-art methods. Code is available at https://github.com/KevinTan10/TSIEN.

Authors

• Xin Tan East China Normal University, Shanghai, China
• Ce Zhao Harbin Institute of Technology, Shenzhen
• Chengliang Liu Computer Science and Engineering, Hong Kong University of Science and Technology
• Jie Wen
• Zhanyan Tang Harbin Institute of Technology, Shenzhen

Keywords

• ML: Multi-class/Multi-label Learning & Extreme Classification
• ML: Multi-instance/Multi-view Learning
• ML: Multimodal Learning
• ML: Representation Learning