Structure-Aware Radar-Camera Depth Estimation

Radar has gained much attention in autonomous driving due to its accessibility and robustness. However, its standalone application for depth perception is constrained by issues of sparsity and noise. Radar-camera depth estimation offers a more promising complementary solution. Despite significant progress, current approaches fail to produce satisfactory dense depth maps, due to the unsatisfactory processing of the sparse and noisy radar data. They constrain the regions of interest for radar points in rigid rectangular regions, which may introduce unexpected errors and confusions. To address these issues, we develop a structure-aware strategy for radar depth enhancement, which provides more targeted regions of interest by leveraging the structural priors of RGB images. Furthermore, we design a Multi-Scale Structure Guided Network to enhance radar features and preserve detailed structures, achieving accurate and structure-detailed dense metric depth estimation. Building on these, we propose a structure-aware radar-camera depth estimation framework, named SA-RCD. Extensive experiments demonstrate that our SA-RCD achieves state-of-the-art performance on the nuScenes dataset. Our code will be available at https://github.com/FreyZhangYeh/SA-RCD.

Authors

• Fuyi Zhang
• Zhu Yu 0001
• Chunhao Li
• Runmin Zhang
• Xiaokai Bai
• Zili Zhou
• Si-Yuan Cao
• Fang Wang
• Hui-Liang Shen

Keywords

• Accuracy
• Codes
• Depth measurement
• Noise
• Buildings
• Radar
• Radar imaging
• Robustness
• Noise measurement
• Robotics and automation
• Depth Estimation
• Promising Solution
• RGB Images
• Depth Map
• Radar Data
• Depth Perception
• Accurate Depth
• Dense Depth
• Sparsity Issue
• Root Mean Square Error
• Structural Information
• Density Map
• Point Cloud
• Feature Fusion
• Skip Connections
• Inference Time
• Valid Values
• Depth Values
• Current Frame
• Depth Distribution
• Monocular Depth Estimation
• Confidence Map
• Adjacent Frames
• Unseen Images
• Height Dimensions
• Depth Features
• Pixel Depth