Point cloud video object segmentation using a persistent supervoxel world-model

Robust visual tracking is an essential precursor to understanding and replicating human actions in robotic systems. In order to accurately evaluate the semantic meaning of a sequence of video frames, or to replicate an action contained therein, one must be able to coherently track and segment all observed agents and objects. This work proposes a novel online point cloud based algorithm which simultaneously tracks 6DoF pose and determines spatial extent of all entities in indoor scenarios. This is accomplished using a persistent supervoxel world-model which is updated, rather than replaced, as new frames of data arrive. Maintenance of a world model enables general object permanence, permitting successful tracking through full occlusions. Object models are tracked using a bank of independent adaptive particle filters which use a supervoxel observation model to give rough estimates of object state. These are united using a novel multi-model RANSAC-like approach, which seeks to minimize a global energy function associating world-model supervoxels to predicted states. We present results on a standard robotic assembly benchmark for two application scenarios — human trajectory imitation and semantic action understanding — demonstrating the usefulness of the tracking in intelligent robotic systems.

Authors

• Jeremie Papon
• Tomas Kulvicius
• Eren Erdal Aksoy
• Florentin Wörgötter

Keywords

• Octrees
• Robots
• Visualization
• Trajectory
• Target tracking
• Three-dimensional displays
• Image segmentation
• Point Cloud
• Video Object Segmentation
• Spatial Extent
• Intelligent Systems
• Robotic System
• Global Energy
• Particle Filter
• Robust Tracking
• Intelligent Robots
• Successful Tracking
• Energy Minimization
• Feature Space
• 3D Space
• Line-of-sight
• Single Target
• Tracking Algorithm
• Object Tracking
• Human Motion
• Cartesian Space
• Trajectories In Space
• Adjacency Graph
• Imitation Learning
• Adjacency Relationship