Revisiting Deformable Convolution for Depth Completion

Depth completion, which aims to generate high-quality dense depth maps from sparse depth maps, has attracted increasing attention in recent years. Previous work usually employs RGB images as guidance, and introduces iterative spatial propagation to refine estimated coarse depth maps. However, most of the propagation refinement methods require several iterations and suffer from a fixed receptive field, which may contain irrelevant and useless information with very sparse input. In this paper, we address these two challenges simultaneously by revisiting the idea of deformable convolution. We propose an effective architecture that leverages deformable kernel convolution as a single-pass refinement module, and empirically demonstrate its superiority. To better understand the function of deformable convolution and exploit it for depth completion, we further systematically investigate a variety of representative strategies. Our study reveals that, different from prior work, deformable convolution needs to be applied on an estimated depth map with a relatively high density for better performance. We evaluate our model on the large-scale KITTI dataset and achieve state-of-the-art level performance in both accuracy and inference speed. Our code is available at https://github.com/AlexSunNiklReDC.

Authors

• Xinglong Sun
• Jean Ponce
• Yu-Xiong Wang

Keywords

• Deformable models
• Codes
• Convolution
• Iterative methods
• Kernel
• Intelligent robots
• Deformable Convolution
• Depth Completion
• Performance Accuracy
• Receptive Field
• RGB Images
• Depth Map
• Inference Speed
• Convolution Function
• Sparse Map
• Useless Information
• Dense Depth
• Sparse Input
• Root Mean Square Error
• Learning Rate
• Convolutional Neural Network
• Convolutional Layers
• Feature Maps
• Mean Absolute Error
• Density Map
• Regular Grid
• High Sparsity
• Depth Values
• Square Grid
• Regular Square
• Top Left Corner
• Valid Pixels
• Depth Estimation
• Hyperparameter Settings
• Training Objective
• Single Pass