Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

The recent multi-modality models have achieved great performance in many vision tasks because the extracted features contain the multi-modality knowledge. However, most of the current registration descriptors have only concentrated on local geometric structures. This paper proposes a method to boost point cloud registration accuracy by transferring the multi-modality knowledge of pre-trained multi-modality model to a new descriptor neural network. Different to the previous multi-modality methods that requires both modalities, the proposed method only requires point clouds during inference. Specifically, we propose an ensemble descriptor neural network combining pre-trained sparse convolution branch and a new point-based convolution branch. By fine-tuning on a single modality data, the proposed method achieves new state-of-the-art results on 3DMatch and competitive accuracy on 3DLoMatch and KITTI. The code and the trained model will be released at https://github.com/phdymz/DBENet.git.

Authors

• Mingzhi Yuan
• Xiaoshui Huang
• Kexue Fu 0001
• Zhihao Li
• Manning Wang

Keywords

• Point cloud compression
• Knowledge engineering
• Three-dimensional displays
• Codes
• Convolution
• Neural networks
• Feature extraction
• Point Cloud
• Point Cloud Registration
• 3D Registration
• 3D Point Cloud Registration
• Vision Tasks
• Multimodal Model
• Local Geometric Structure
• Transformer
• Knowledge Transfer
• Multiple Modalities
• Semantic Information
• Ensemble Model
• Large Margin
• Feature Matching
• Pre-trained Network
• Geometric Information
• Pre-trained Weights
• Extract High-level Features
• Input Point Cloud