Multi-Prototype Networks for Unconstrained Set-based Face Recognition

In this paper, we address the challenging unconstrained set-based face recognition problem where each subject face is instantiated by a set of media (images and videos) instead of a single image. Naively aggregating information from all the media within a set would suffer from the large intra-set variance caused by heterogeneous factors (e. g. , varying media modalities, poses and illumination) and fail to learn discriminative face representations. A novel Multi-Prototype Network (MP- Net) model is thus proposed to learn multiple prototype face representations adaptively from the media sets. Each learned prototype is representative for the subject face under certain condition in terms of pose, illumination and media modality. Instead of handcrafting the set partition for prototype learn- ing, MPNet introduces a Dense SubGraph (DSG) learning sub-net that implicitly untangles inconsistent media and learns a number of representative prototypes. Qualitative and quantitative experiments clearly demonstrate the superiority of the proposed model over state-of-the-arts.

Authors

• Jian Zhao Zhongguancun Academy
• Jianshu Li National University of Singapore
• Xiaoguang Tu
• Fang Zhao Nanjing University
• Yuan Xin
• Junliang Xing Department of Computer Science and Technology, Tsinghua University
• Hengzhu Liu
• Shuicheng Yan Skywork AI, Singapore Nanyang Technological University, Singapore
• Jiashi Feng

Keywords

• Computer Vision: Biometrics, Face and Gesture Recognition
• Computer Vision: Computer Vision
• Machine Learning: Deep Learning