ACP-MVS: Efficient Multi-View Stereo with Attention-based Context Perception

The core of Multi-View Stereo (MVS) is to find corresponding pixels in neighboring images. However, due to challenging regions in input images such as untextured areas, repetitive patterns, or reflective surfaces, existing methods struggle to find precise pixel correspondence therein, resulting in inferior reconstruction quality. In this paper, we present an efficient context-perception MVS network, termed ACP-MVS. The ACP-MVS constructs a context-aware cost volume that can enhance pixels containing essential context information while suppressing irrelevant or noisy information via our proposed Context-stimulated Weighting Fusion module. Furthermore, we introduce a new Context-Guided Global Aggregation module, based on the insight that similar-looking pixels tend to have similar depths, which exploits global contextual cues to implicitly guide depth detail propagation from high-confidence regions to low-confidence ones. These two modules work in synergy to substantially improve reconstruction quality of ACP-MVS without incurring significant additional computational and time cost. Extensive experiments demonstrate that our approach not only achieves state-of-the-art performance but also offers the fastest inference speed and minimal GPU memory usage, providing practical value for practitioners working with high-resolution MVS image sets. Notably, our method ranks 2nd on the challenging Tanks and Temples advanced benchmark among all published methods. Code is available at https://github.com/HaoJia-mongh/ACP-MVS.

Authors

• Hao Jia
• Gangwei Xu
• Miaojie Feng
• Xianqi Wang 0001
• Junda Cheng
• Min Lin 0006
• Xin Yang 0008

Keywords

• Surface reconstruction
• Costs
• Codes
• Attention mechanisms
• Graphics processing units
• Benchmark testing
• Noise measurement
• Image reconstruction
• Intelligent robots
• Multi-view Stereo
• High-resolution Images
• Contextual Information
• Input Image
• Surface Reflectance
• Repetitive Patterns
• Memory Usage
• Reconstruction Quality
• GPU Memory
• Similar Depth
• Noisy Information
• Aggregation Module
• Cost Volume
• Minimal Memory
• Work In Synergy
• Training Set
• Transformer
• Validation Set
• Attention Mechanism
• Point Cloud
• Depth Map
• Memory Efficiency
• Learning-based Methods
• Intermediate Subset
• Geometric Cues
• Reference Image
• Attention Map
• Time Overhead
• Number Of Hypotheses
• Attention-based Methods