Revisiting Open-Set Panoptic Segmentation

In this paper, we focus on the open-set panoptic segmentation (OPS) task to circumvent the data explosion problem. Different from the close-set setting, OPS targets to detect both known and unknown categories, where the latter is not annotated during training. Different from existing work that only selects a few common categories as unknown ones, we move forward to the real-world scenario by considering the various tail categories (~1k). To this end, we first build a new dataset with long-tail distribution for the OPS task. Based on this dataset, we additionally add a new class type for unknown classes and re-define the training annotations to make the OPS definition more complete and reasonable. Moreover, we analyze the influence of several significant factors in the OPS task and explore the upper bound of performance on unknown classes with different settings. Furthermore, based on the analyses, we design an effective two-phase framework for the OPS task, including thing-agnostic map generation and unknown segment mining. We further adopt semi-supervised learning to improve the OPS performance. Experimental results on different datasets validate the effectiveness of our method.

Authors

• Yufei Yin CAS Key Laboratory of Technology in GIPAS, EEIS Department, University of Science and Technology of China
• Hao Chen
• Wengang Zhou University of Science and Technology of China
• Jiajun Deng University of Adelaide, Australian Institute for Machine Learning
• Haiming Xu Xidian University
• Houqiang Li University of Science and Technology of China

Keywords

• CV: Segmentation