BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models

Tracking the 6D pose of objects in video sequences is important for robot manipulation. Most prior efforts, however, often assume that the target object's CAD model, at least at a category-level, is available for offline training or during online template matching. This work proposes BundleTrack, a general framework for 6D pose tracking of novel objects, which does not depend upon 3D models, either at the instance or category-level. It leverages the complementary attributes of recent advances in deep learning for segmentation and robust feature extraction, as well as memory-augmented pose graph optimization for spatiotemporal consistency. This enables long-term, low-drift tracking under various challenging scenarios, including significant occlusions and object motions. Comprehensive experiments given two public benchmarks demonstrate that the proposed approach significantly outperforms state-of-art, category-level 6D tracking or dynamic SLAM methods. When compared against state-of-art methods that rely on an object instance CAD model, comparable performance is achieved, despite the proposed method’s reduced information requirements. An efficient implementation in CUDA provides a real-time performance of 10Hz for the entire framework. Code is available at: https://github.com/wenbowen123/BundleTrack

Authors

• Bowen Wen
• Kostas E. Bekris

Keywords

• Training
• Solid modeling
• Target tracking
• Three-dimensional displays
• Simultaneous localization and mapping
• Motion segmentation
• Video sequences
• Pose Tracking
• 6D Pose
• Performance Comparison
• Target Object
• Object Tracking
• Robot Manipulator
• Advances In Deep Learning
• Challenging Scenarios
• CAD Model
• Object Instances
• Object Pose
• Public Benchmark
• Tracking Framework
• Robust Feature Extraction
• Deep Learning Segmentation
• Diagonal Matrix
• Global Model
• Qualitative Results
• Global Optimization
• Point Cloud
• Pose Estimation
• Dynamic Objects
• Iterative Closest Point
• Simultaneous Reconstruction
• Long-term Tracking
• Memory Pool
• 3D Bounding Box
• Geodesic Distance
• R-strategists
• Corresponding Points