Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything Model

Extrinsic calibration for LiDAR and camera is an essential prerequisite for sensor fusion. Recently, automatic and target-less extrinsic calibration has become the mainstream of academic research. However, geometric feature-based methods still have requirements on the scene. Deep learning methods, while achieving high accuracy and good adaptability, rely on large annotated dataset and need additional training. We propose a novel LiDAR-camera calibration method by using the Segment Anything Model(SAM) without additional training. With the automatically generated masks, we optimize the extrinsic parameters by maximizing the consistency score of the point attributes that fall on each mask. The point cloud attributes include intensity, normal vector and segmentation class. Experiments on different real-world dataset demonstrate the accuracy and robustness of our proposed method. The code is available at https://github.com/OpenCalib/CalibAnything.

Authors

• Zhaotong Luo
• Guohang Yan
• Xinyu Cai
• Botian Shi

Keywords

• Training
• Point cloud compression
• Solid modeling
• Adaptation models
• Accuracy
• Sensor fusion
• Vectors
• Calibration Method
• Extrinsic Calibration
• Normal Vector
• Point Cloud
• Consistency Score
• Segment Classification
• Extrinsic Parameters
• Intensity Classes
• Objective Function
• Convolutional Neural Network
• Image Segmentation
• Mutual Information
• Geometric Features
• Intensity Function
• Learning-based Methods
• Autonomous Vehicles
• Natural Scenes
• Normal Approximation
• Multiple Frames
• Foundation Model
• Local Extrema
• Plane Fitting
• Rotation Error
• Point Cloud Segmentation