LiDAR Panoptic Segmentation for Autonomous Driving

Truly autonomous driving without the need for human intervention can only be attained when self-driving cars fully understand their surroundings. Most of these vehicles rely on a suite of active and passive sensors. LiDAR sensors are a cornerstone in most of these hardware stacks, and leveraging them as a complement to other passive sensors such as RGB cameras is an enticing goal. Understanding the semantic class of each point in a LiDAR sweep is important, as well as knowing to which instance of that class it belongs to. To this end, we present a novel, single-stage, and real-time capable panoptic segmentation approach using a shared encoder with a semantic and instance decoder. We leverage the geometric information of the LiDAR scan to perform a novel, distance- aware tri-linear upsampling, which allows our approach to use larger output strides than using transpose convolutions leading to substantial savings in computation time. Our experimental evaluation and ablation studies for each module show that combining our geometric and semantic embeddings with our learned, variable instance thresholds, a category-specific loss, and the novel trilinear upsampling module leads to higher panoptic quality. We will release the code of our approach in our LiDAR processing library LiDAR-Bonnetal [27].

Authors

• Andres Milioto
• Jens Behley
• Chris McCool
• Cyrill Stachniss

Keywords

• Convolutional codes
• Laser radar
• Runtime
• Semantics
• Real-time systems
• Sensor systems
• Autonomous vehicles
• Panoptic Segmentation
• LiDAR Panoptic Segmentation
• Class Instances
• Passive Sensors
• LiDAR Sensor
• Real-time Approach
• Semantic Embedding
• Convolutional Neural Network
• Intersection Over Union
• Point Cloud
• Bounding Box
• Increase In Performance
• Semantic Segmentation
• Segmentation Task
• Two-stage Approach
• Convolutional Block
• Training Error
• Instance Segmentation
• Image Coordinates
• Semantic Annotation
• Conditional Random Field
• Metric Learning
• Point Cloud Segmentation
• Ground Truth Segmentation
• Spherical Projection
• Upsampled Feature
• Random Projection
• Scene Understanding