Depth Estimation from Monocular Images and Sparse Radar Data

In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods developed for LiDAR data and images to the new fusion problem between Radar data and images. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions. Extensive experiments demonstrate that our method outperforms existing fusion methods. We also provide detailed ablation studies to show the effectiveness of each component in our method.

Authors

• Juan-Ting Lin
• Dengxin Dai
• Luc Van Gool

Keywords

• Laser radar
• Radar measurements
• Neural networks
• Estimation
• Radar imaging
• Task analysis
• Videos
• Sparse Data
• Radar Data
• Depth Estimation
• Monocular Images
• Sparse Radar Data
• Detailed Study
• Deep Neural Network
• RGB Images
• Fusion Method
• Lidar Data
• Accurate Depth
• Convolutional Neural Network
• Object Detection
• Point Cloud
• Convolutional Neural Network Model
• Semantic Segmentation
• Depth Map
• Visual Perspective
• Depth Values
• Late Fusion
• Noisy Measurements
• Lidar Measurements
• Dense Depth
• LiDAR Point
• Monocular Depth Estimation
• Early Fusion
• Limited Field Of View
• Inconsistent Measurement
• Stereo Camera