Tracking deformable objects with point clouds

We introduce an algorithm for tracking deformable objects from a sequence of point clouds. The proposed tracking algorithm is based on a probabilistic generative model that incorporates observations of the point cloud and the physical properties of the tracked object and its environment. We propose a modified expectation maximization algorithm to perform maximum a posteriori estimation to update the state estimate at each time step. Our modification makes it practical to perform the inference through calls to a physics simulation engine. This is significant because (i) it allows for the use of highly optimized physics simulation engines for the core computations of our tracking algorithm, and (ii) it makes it possible to naturally, and efficiently, account for physical constraints imposed by collisions, grasping actions, and material properties in the observation updates. Even in the presence of the relatively large occlusions that occur during manipulation tasks, our algorithm is able to robustly track a variety of types of deformable objects, including ones that are one-dimensional, such as ropes; two-dimensional, such as cloth; and three-dimensional, such as sponges. Our implementation can track these objects in real time.

Authors

• John Schulman
• Alex X. Lee
• Jonathan Ho
• Pieter Abbeel

Keywords

• Computational modeling
• Noise
• Probabilistic logic
• Inference algorithms
• Physics
• Solid modeling
• Mathematical model
• Point Cloud
• Deformable Objects
• Time Step
• Probabilistic Model
• Expectation Maximization
• Physical Constraints
• Manipulation Tasks
• Tracking Algorithm
• Posterior Mode
• Maximum A Posteriori
• Physical Simulation
• Simulation Engine
• Probabilistic Generative Model
• Real-time Object
• Human Use
• Rigid Body
• Kullback-Leibler
• Model Inference
• Color Features
• Depth Images
• Iterative Closest Point Algorithm
• Presence Of Occlusion
• Piece Of Cloth
• Bending Energy
• Objective Conditions
• Iterative Closest Point
• Depth Camera
• Observation Points
• Mode Of Data Collection
• Inference Procedure