Efficient Multimodal 3D Object Detector via Instance-Level Contrastive Distillation

Multimodal 3D object detectors leverage the strengths of both geometry-aware LiDAR point clouds and semantically rich RGB images to enhance detection performance. However, the inherent heterogeneity between these modalities, including unbalanced convergence and modal misalignment, poses significant challenges. Meanwhile, the large size of the detection-oriented feature also constrains existing fusion strategies to capture long-range dependencies for the 3D detection tasks. In this work, we introduce a fast yet effective multimodal 3D object detector, incorporating our proposed Instance-level Contrastive Distillation (ICD) framework and Cross Linear Attention Fusion Module (CLFM). ICD aligns instance-level image features with LiDAR representations through object-aware contrastive distillation, ensuring fine-grained cross-modal consistency. Meanwhile, CLFM presents an efficient and scalable fusion strategy that enhances cross-modal global interactions within sizable multimodal BEV features. Extensive experiments on the KITTI and nuScenes 3D object detection benchmarks demonstrate the effectiveness of our methods. Notably, our 3D object detector outperforms state-of-the-art (SOTA) methods while achieving superior efficiency. The implementation of our method has been released as open-source at: https://github.com/nubot-nudt/ICD-Fusion.

Authors

• Zhuoqun Su
• Huimin Lu 0002
• Shuaifeng Jiao
• Junhao Xiao 0001
• Yaonan Wang 0001
• Xieyuanli Chen

Keywords

• Point cloud compression
• Three-dimensional displays
• Laser radar
• Head
• Detectors
• Object detection
• Benchmark testing
• Feature extraction
• Knowledge transfer
• Intelligent robots
• 3D Object Detection
• Multimodal 3D
• Semantic
• Image Features
• Detection Performance
• Point Cloud
• RGB Images
• Linear Mode
• Fusion Strategy
• Long-range Dependencies
• Multimodal Features
• 3D Detection
• LiDAR Point Clouds
• Convolutional Layers
• Pedestrian
• Data Augmentation
• Attention Mechanism
• KITTI Dataset
• Multimodal Detection
• Ground-truth Bounding Box
• Alignment Strategy
• Image Encoder
• Detection Head
• Ground Truth Samples
• State-space Model
• Objects In The Scene
• Bounding Box