Multi-View Multi-Label Classification via View-Label Matching Selection

In multi-view multi-label classification (MVML), each object is described by several heterogeneous views while annotated with multiple related labels. The key to learn from such complicate data lies in how to fuse cross-view features and explore multi-label correlations, while accordingly obtain correct assignments between each object and its corresponding labels. In this paper, we proposed an advanced MVML method named VAMS, which treats each object as a bag of views and reformulates the task of MVML as a “view-label” matching selection problem. Specifically, we first construct an object graph and a label graph respectively. In the object graph, nodes represent the multi-view representation of an object, and each view node is connected to its K-nearest neighbor within its own view. In the label graph, nodes represent the semantic representation of a label. Then, we connect each view node with all labels to generate the unified “view-label” matching graph. Afterwards, a graph network block is introduced to aggregate and update all nodes and edges on the matching graph, and further generating a structural representation that fuses multi-view heterogeneity and multi-label correlations for each view and label. Finally, we derive a prediction score for each view-label matching and select the optimal matching via optimizing a weighted cross-entropy loss. Extensive results on various datasets have verified that our proposed VAMS can achieve superior or comparable performance against state-of-the-art methods.

Authors

• Hao Wei State Key Laboratory of Multimodal Artificial Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
• Yongjian Deng College of Computer Science, Beijing University of Technology
• Qiuru Hai College of Computer Science, Beijing University of Technology
• Yuena Lin College of Computer Science, Beijing University of Technology Idealism Beijing Technology Co., Ltd.
• Zhen Yang
• Gengyu Lyu College of Computer Science, Beijing University of Technology School of Computer Science and Technology, Beijing Jiaotong University Idealism Beijing Technology Co., Ltd.

Keywords

No keywords are indexed for this paper.