TemporalStereo: Efficient Spatial-Temporal Stereo Matching Network

We present TemporalStereo, a coarse-to-fine stereo matching network that is highly efficient, and able to effectively exploit the past geometry and context information to boost matching accuracy. Our network leverages sparse cost volume and proves to be effective when a single stereo pair is given. However, its peculiar ability to use spatio-temporal information across stereo sequences allows TemporalStereo to alleviate problems such as occlusions and reflective regions while enjoying high efficiency also in this latter case. Notably, our model - trained once with stereo videos - can run in both single-pair and temporal modes seamlessly. Experiments show that our network relying on camera motion is robust even to dynamic objects when running on videos. We validate TemporalStereo through extensive experiments on synthetic (SceneFlow, TartanAir) and real (KITTI 2012, KITTI 2015) datasets. Our model achieves state-of-the-art performance on any of these datasets.

Authors

• Youmin Zhang 0008
• Matteo Poggi
• Stefano Mattoccia

Keywords

• Geometry
• Costs
• Dynamics
• Cameras
• Intelligent robots
• Videos
• Stereo Matching
• Stereo Matching Network
• Temporal Modulation
• Stereo Pairs
• Cost Volume
• Training Time
• Feature Maps
• Large Margin
• Local Map
• Optical Flow
• Temporal Shift
• Past Information
• Search Range
• 3D Convolution
• Camera Pose
• Pseudo Labels
• Current Ones
• Motion Field
• Cost Aggregation
• Difficult Areas
• Disparity Map
• Occluded Regions
• Ground Truth Pose
• Stereographic Projection
• Disparity Estimation
• Error Metrics
• Temporal Model
• Deep Learning