Decoupling with Entropy-based Equalization for Semi-Supervised Semantic Segmentation

Semi-supervised semantic segmentation methods are the main solution to alleviate the problem of high annotation consumption in semantic segmentation. However, the class imbalance problem makes the model favor the head classes with sufficient training samples, resulting in poor performance of the tail classes. To address this issue, we propose a Decoupled Semi-Supervise Semantic Segmentation (DeS4) framework based on the teacher-student model. Specifically, we first propose a decoupling training strategy to split the training of the encoder and segmentation decoder, aiming at a balanced decoder. Then, a non-learnable prototype-based segmentation head is proposed to regularize the category representation distribution consistency and perform a better connection between the teacher model and the student model. Furthermore, a Multi-Entropy Sampling (MES) strategy is proposed to collect pixel representation for updating the shared prototype to get a class-unbiased head. We conduct extensive experiments of the proposed DeS4 on two challenging benchmarks (PASCAL VOC 2012 and Cityscapes) and achieve remarkable improvements over the previous state-of-the-art methods.

Authors

• Chuanghao Ding
• Jianrong Zhang Zhejiang University
• Henghui Ding Fudan University
• Hongwei Zhao College of Computer Science and Technology, Jilin University, China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China
• Zhihui Wang Dalian University of Technology
• Tengfei Xing
• Runbo Hu

Keywords

• Computer Vision: CV: Scene analysis and understanding
• Computer Vision: CV: Segmentation
• Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning