SimTrack: A simulation-based framework for scalable real-time object pose detection and tracking

We propose a novel approach for real-time object pose detection and tracking that is highly scalable in terms of the number of objects tracked and the number of cameras observing the scene. Key to this scalability is a high degree of parallelism in the algorithms employed. The method maintains a single 3D simulated model of the scene consisting of multiple objects together with a robot operating on them. This allows for rapid synthesis of appearance, depth, and occlusion information from each camera viewpoint. This information is used both for updating the pose estimates and for extracting the low-level visual cues. The visual cues obtained from each camera are efficiently fused back into the single consistent scene representation using a constrained optimization method. The centralized scene representation, together with the reliability measures it enables, simplify the interaction between pose tracking and pose detection across multiple cameras. We demonstrate the robustness of our approach in a realistic manipulation scenario. We publicly release this work as a part of a general ROS software framework for real-time pose estimation, SimTrack, that can be integrated easily for different robotic applications.

Authors

• Karl Pauwels
• Danica Kragic

Keywords

• Cameras
• Robot vision systems
• Visualization
• Robot kinematics
• Real-time systems
• Scalable
• Object Tracking
• Object Pose
• Pose Detection
• Real-time Object Tracking
• Object Pose Detection
• Visual Cues
• Multiple Objects
• Number Of Objects
• Depth Information
• Pose Estimation
• 3D Simulation
• Multiple Cameras
• Software Framework
• Scene Representation
• Pose Tracking
• Computation Time
• Left Arm
• Monocular
• Optical Flow
• Left Camera
• Camera Frame
• Camera Motion
• Normal Equations
• Object Motion
• Camera Head
• World Coordinate
• Visual Servoing
• End-effector
• Tracking Failure