Patch-level Contrastive Learning via Positional Query for Visual Pre-training

Dense contrastive learning (DCL) has been recently explored for learning localized information for dense prediction tasks (e. g. , detection and segmentation). It still suffers the difficulty of mining pixels/patches correspondence between two views. A simple way is inputting the same view twice and aligning the pixel/patch representation. However, it would reduce the variance of inputs, and hurts the performance. We propose a plug-in method PQCL (Positional Query for patch-level Contrastive Learning), which allows performing patch-level contrasts between two views with exact patch correspondence. Besides, by using positional queries, PQCL increases the variance of inputs, to enhance training. We apply PQCL to popular transformer-based CL frameworks (DINO and iBOT, and evaluate them on classification, detection and segmentation tasks, where our method obtains stable improvements, especially for dense tasks. It achieves new state-of-the-art in most settings. Code is available at https: //github. com/Sherrylone/Query_Contrastive.

Authors

• Shaofeng Zhang
• Qiang Zhou 0001
• Zhibin Wang 0004
• Fan Wang 0019
• Junchi Yan

Keywords

No keywords are indexed for this paper.