Multi-View Guided Multi-View Stereo

This paper introduces a novel deep framework for dense 3D reconstruction from multiple image frames, leveraging a sparse set of depth measurements gathered jointly with image acquisition. Given a deep multi-view stereo network, our framework uses sparse depth hints to guide the neural network by modulating the plane-sweep cost volume built during the forward step, enabling us to infer constantly much more accurate depth maps. Moreover, since multiple viewpoints can provide additional depth measurements, we propose a multi-view guidance strategy that increases the density of the sparse points used to guide the network, thus leading to even more accurate results. We evaluate our Multi-View Guided framework within a variety of state-of-the-art deep multi-view stereo networks, demonstrating its effectiveness at improving the results achieved by each of them on BlendedMVG and DTU datasets.

Authors

• Matteo Poggi
• Andrea Conti 0002
• Stefano Mattoccia

Keywords

• Point cloud compression
• Three-dimensional displays
• Costs
• Density measurement
• Neural networks
• Estimation
• Intelligent robots
• Multi-view Stereo
• Deep Network
• 3D Reconstruction
• Accurate Mapping
• Depth Map
• Depth Measurements
• Multiple Viewpoints
• Sparse Point
• Cost Volume
• Deep Learning
• Error Rate
• Convolutional Neural Network
• Computer Vision
• Long Short-term Memory
• Color Images
• Point Cloud
• Deep Architecture
• Training Protocol
• Source Images
• Optical Flow
• 3D Convolution
• Stereo Matching
• Depth Point
• Test Split
• Single Volume
• Original Network
• Strong Guidance
• Depth Estimation
• Forward Pass
• Point Cloud Reconstruction