Label-Efficient LiDAR Panoptic Segmentation

A main bottleneck of learning-based robotic scene understanding methods is the heavy reliance on extensive annotated training data, which often limits their generalization ability. In LiDAR panoptic segmentation, this challenge becomes even more pronounced due to the need to simultaneously address both semantic and instance segmentation from complex, high-dimensional point cloud data. In this work, we address the challenge of LiDAR panoptic segmentation with very few labeled samples by leveraging recent advances in label-efficient vision panoptic segmentation. To this end, we propose a novel method, Limited-Label LiDAR Panoptic Segmentation (L 3 PS), which requires only a minimal amount of labeled data. Our approach first utilizes a label-efficient 2D network to generate panoptic pseudo-labels from a small set of annotated images, which are subsequently projected onto point clouds. We then introduce a novel 3D refinement module that capitalizes on the geometric properties of point clouds. By incorporating clustering techniques, sequential scan accumulation, and ground point separation, this module significantly enhances the accuracy of the pseudo-labels, improving segmentation quality by up to +10. 6 PQ and +7. 9 mIoU. We demonstrate that these refined pseudo-labels can be used to effectively train off-the-shelf LiDAR segmentation networks. Through extensive experiments, we show that L 3 PS not only outperforms existing methods but also substantially reduces the annotation burden. We release the code of our work at https://l3ps.cs.uni-freiburg.de.

Authors

• Ahmet Selim Çanakçi
• Niclas Vödisch
• Kürsat Petek
• Wolfram Burgard
• Abhinav Valada

Keywords

• Point cloud compression
• Representation learning
• Image segmentation
• Visualization
• Laser radar
• Three-dimensional displays
• Annotations
• Semantics
• Refining
• Training data
• Panoptic Segmentation
• LiDAR Panoptic Segmentation
• Point Cloud
• Annotation Data
• Semantic Segmentation
• Point Cloud Data
• Segmentation Quality
• 3D Refinement
• Cloud Properties
• Weighting Factor
• Experimental Evaluation
• 3D Space
• 3D Point
• Labeling Efficiency
• Self-supervised Learning
• Frequent Class
• Lidar Data
• Image Annotation
• Foundation Model
• Rare Classes
• Semantic Annotation
• Annotation Efforts
• LiDAR Scans
• Point Cloud Segmentation
• LiDAR Point Clouds
• Raw Point Cloud
• Self-supervised Approach
• Ground Segment
• Subset Of Images