Can Semantic Labels Assist Self-Supervised Visual Representation Learning?

Recently, contrastive learning has largely advanced the progress of unsupervised visual representation learning. Pretrained on ImageNet, some self-supervised algorithms reported higher transfer learning performance compared to fully-supervised methods, seeming to deliver the message that human labels hardly contribute to learning transferrable visual features. In this paper, we defend the usefulness of semantic labels but point out that fully-supervised and selfsupervised methods are pursuing different kinds of features. To alleviate this issue, we present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN) that maximally prevents the semantic guidance from damaging the appearance feature embedding. In a series of downstream tasks, SCAN achieves superior performance compared to previous fully-supervised and self-supervised methods, and sometimes the gain is significant. More importantly, our study reveals that semantic labels are useful in assisting self-supervised methods, opening a new direction for the community.

Authors

• Longhui Wei Huawei Inc.
• Lingxi Xie Huawei Technologies Ltd.
• Jianzhong He
• Xiaopeng Zhang MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
• Qi Tian Huawei Inc.

Keywords

No keywords are indexed for this paper.