SDGE: Stereo Guided Depth Estimation for 360°Camera Sets

Depth estimation is a critical technology in autonomous driving, and multi-camera systems are often used to achieve a 360° perception. These 360° camera sets often have limited or low-quality overlap regions, making multi-view stereo methods infeasible for the entire image. Alternatively, monocular methods may not produce consistent cross-view predictions. To address these issues, we propose the Stereo Guided Depth Estimation (SGDE) method, which enhances depth estimation of the full image by explicitly utilizing multi-view stereo results on the overlap. We suggest building virtual pinhole cameras to resolve the distortion problem of fisheye cameras and unify the processing for the two types of 360° cameras. For handling the varying noise on camera poses caused by unstable movement, the approach employs a self-calibration method to obtain highly accurate relative poses of the adjacent cameras with minor overlap. These enable the use of robust stereo methods to obtain a high-quality depth prior in the overlap region. This prior serves not only as an additional input but also as pseudo-labels that enhance the accuracy of depth estimation methods and improve cross-view prediction consistency. The effectiveness of SGDE is evaluated on one fisheye camera dataset, Synthetic Urban, and two pinhole camera datasets, DDAD and nuScenes. Our experiments demonstrate that SGDE is effective for both supervised and self-supervised depth estimation, and highlight the potential of our method for advancing autonomous driving technology. Our project page is at https://github.com/JialeiXu/SGDE.

Authors

• Jialei Xu
• Wei Yin
• Dong Gong
• Junjun Jiang
• Xianming Liu 0005

Keywords

• Accuracy
• Depth measurement
• Robot vision systems
• Pipelines
• Noise
• Cameras
• Prediction algorithms
• Autonomous vehicles
• Optimization
• Intelligent robots
• Depth Estimation
• Overlap Region
• Camera Pose
• Fisheye Lens
• Relative Pose
• Pinhole Camera
• Virtual Camera
• Multi-view Stereo
• Multi-camera System
• Deep Neural Network
• Parametrized
• Depth Map
• Calibration Method
• Video Sequences
• Overlap Area
• Geometric Constraints
• Consistent Improvement
• Small Overlap
• Structure From Motion
• Camera Calibration
• Stereo Matching
• Depth Prediction
• Supervision Training
• Bundle Adjustment
• Supervision Signal
• Geometric Cues
• Extrinsic Parameters
• Monocular Depth Estimation
• Stereo Camera