EfficientDPS: Efficient and End-to-End Depth-aware Panoptic Segmentation

Depth-aware panoptic segmentation (DPS) combines image segmentation and monocular depth estimation in a single model to achieve semantic and geometry perception simultaneously. DPS task has important applications in the robot area but the previous DPS models are too heavy to be applied. Thus, we propose EfficientDPS, an efficient, end-to-end, and unified model for DPS. In our method, query features extracted with convolution networks are used to represent things/stuff. In this way, different vision tasks such as classification, segmentation, and depth estimation can be realized in a unified manner, leading to a compact and efficient model. EfficientDPS can be trained and tested in an end-to-end manner via bipartite matching and complex post-process is not needed at inference. To enhance the supervision signal, group query representation is proposed, leading to better performance without affecting the inference speed. Extensive experiments on Cityscapes-DPS and SemKITTI-DPS show that EfficientDPS can achieve the best trade-off between speed and accuracy than the state-of-the-art methods.

Authors

• Shengkai Wu
• Liangliang Ren
• Linfeng Gao
• Yupeng Li
• Wen-Yu Liu 0003

Keywords

• Performance evaluation
• Geometry
• Image segmentation
• Accuracy
• Convolution
• Semantics
• Estimation
• Panoptic Segmentation
• Convolutional Network
• Extensive Experiments
• Vision Tasks
• Depth Estimation
• Unified Manner
• Inference Speed
• Query Features
• Bipartite Matching
• Monocular Depth Estimation
• Convolutional Layers
• Sigmoid Function
• Feature Maps
• Object Detection
• Semantic Segmentation
• Depth Map
• Segmentation Task
• Inference Time
• Prediction Set
• 3D Geometry
• Dense Depth
• Instance Segmentation
• Horizontal Flip
• Semantic Map
• Pixel Information