Self-supervised Learning for Single View Depth and Surface Normal Estimation

In this work we present a self-supervised learning framework to simultaneously train two Convolutional Neural Networks (CNNs) to predict depth and surface normals from a single image. In contrast to most existing frameworks which represent outdoor scenes as fronto-parallel planes at piece-wise smooth depth, we propose to predict depth with surface orientation while assuming that natural scenes have piece-wise smooth normals. We show that a simple depth-normal consistency as a soft-constraint on the predictions is sufficient and effective for training both these networks simultaneously. The trained normal network provides state-of-the-art predictions while the depth network, relying on much realistic smooth normal assumption, outperforms the traditional self-supervised depth prediction network by a large margin on the KITTI benchmark.

Authors

• Huangying Zhan
• Chamara Saroj Weerasekera
• Ravi Garg
• Ian D. Reid 0001

Keywords

• Training
• Estimation
• Geometry
• Cameras
• Sensors
• Visual odometry
• Neural networks
• Normal Approximation
• Depth Estimation
• Self-supervised Learning
• Single View
• Single Depth
• Surface Normals
• Surface Normal Estimation
• Single-view Depth
• Convolutional Network
• Convolutional Neural Network
• Single Image
• Prediction Network
• Soft Constraints
• Depth Prediction
• Smoothness Assumption
• Supervised Learning
• Input Image
• Reference Image
• Depth Map
• Image Edge
• Left Image
• Ego-motion
• Relative Pose
• Rotated Component
• Stereo Pairs
• Normal Map
• Temporal Consistency
• Scale Ambiguity
• KITTI Dataset