Learning to Simulate Realistic LiDARs

Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling. To alleviate this, we introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We propose a model that learns a mapping between RGB images and corresponding LiDAR features such as raydrop or perpoint intensities directly from real datasets. We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces or high intensity returns on reflective materials. When applied to naively raycasted point clouds provided by off-the-shelf simulator software, our model enhances the data by predicting intensities and removing points based on the scene's appearance to match a real LiDAR sensor. We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly. Through a sample task of vehicle segmentation, we show that enhancing simulated point clouds with our technique improves downstream task performance.

Authors

• Benoît Guillard
• Sai Vemprala
• Jayesh K. Gupta
• Ondrej Miksik
• Vibhav Vineet
• Pascal Fua
• Ashish Kapoor

Keywords

• Point cloud compression
• Image segmentation
• Laser radar
• Computational modeling
• Pipelines
• Predictive models
• Data models
• Autonomic System
• Simulation Software
• Point Cloud
• RGB Images
• Segmentation Task
• Lidar Data
• LiDAR Sensor
• Ray Casting
• Neural Network
• Training Data
• Field Of View
• Corruption
• Convolutional Neural Network
• Laser Beam
• Random Noise
• Intersection Over Union
• Generative Adversarial Networks
• Semantic Segmentation
• Vanilla
• Synthetic Images
• LiDAR Point
• LiDAR Point Clouds
• Intensity Of Channel
• Sensor Noise
• Domain Adaptation
• Unreal Engine
• L1 Loss