Anyview: General Indoor 3D Object Detection with Variable Frames

In this paper, we propose a novel network framework for indoor 3D object detection to handle variable input frame numbers in practical scenarios. Existing methods only consider fixed frames of input data for a single detector, such as monocular RGB-D images or point clouds reconstructed from dense multi-view RGB-D images. While in practical application scenes such as robot navigation and manipulation, the raw input to the 3D detectors is the RGB-D images with variable frame numbers instead of the reconstructed scene point cloud. However, the previous approaches can only handle fixed frame input data and have poor performance with variable frame input. In order to facilitate 3D object detection methods suitable for practical tasks, we present a novel 3D detection framework named AnyView for our practical applications, which generalizes well across different numbers of input frames with a single model. To be specific, we propose a geometric learner to mine the local geometric features of each input RGB-D image frame and implement local-global feature interaction through a designed spatial mixture module. Meanwhile, we further utilize a dynamic token strategy to adaptively adjust the number of extracted features for each frame, which ensures consistent global feature density and further enhances the generalization after fusion. Extensive experiments on the ScanNet dataset show our method achieves both great generalizability and high detection accuracy with a simple and clean architecture containing a similar amount of parameters with the baselines.

Authors

• Zhenyu Wu
• Xiuwei Xu
• Ziwei Wang 0001
• Chong Xia
• Linqing Zhao
• Jiwen Lu
• Haibin Yan

Keywords

• Point cloud compression
• Training
• Solid modeling
• Three-dimensional displays
• Accuracy
• Object detection
• Detectors
• Feature extraction
• Transformers
• Image reconstruction
• 3D Object Detection
• Local Features
• Global Features
• Point Cloud
• Geometric Features
• Practical Scenarios
• Dynamic Strategy
• Practical Tasks
• High Detection Accuracy
• Robot Navigation
• Input Frames
• 3D Detection
• Multi-view Images
• RGB-D Images
• Benchmark
• Model Performance
• Bounding Box
• Point Cloud Reconstruction
• Previous Benchmark
• Input Point Cloud
• Objects In The Scene
• Camera Coordinate
• Scene Representation
• Practical Deployment
• Input Point
• Proxy For The Number
• Depth Map