Real-Time Generative Grasping with Spatio-temporal Sparse Convolution

Robots performing mobile manipulation in unstructured environments must identify grasp affordances quickly and with robustness to perception noise. Yet in domains such as underwater manipulation, where perception noise is severe, computation is constrained, and the environment is dynamic, existing techniques fail. They are too computationally demanding, or too sensitive to noise to allow for closed loop grasping or dynamic replanning, or do not consider 6-DOF grasps. We present a novel grasp synthesis network, TSGrasp, that uses spatio-temporal sparse convolution to process a streaming point cloud in real time. The network generates 6-DOF grasps at greater speed and with less memory than Contact GraspNet, a state-of-the-art algorithm based on Point-Net++. By considering information from multiple successive frames of depth video, TSGrasp boosts robustness to noise or temporary self-occlusion and allows more grasps to be rapidly identified. Our grasp synthesis system was successfully demonstrated in an underwater environment with a Blueprint Labs Bravo robotic arm.

Authors

• Timothy R. Player
• Dongsik Chang
• Fuxin Li
• Geoffrey A. Hollinger

Keywords

• Point cloud compression
• Convolution
• Grasping
• Streaming media
• Robot sensing systems
• Real-time systems
• 6-DOF
• Sparse Convolution
• Point Cloud
• Robotic Arm
• Multiple Frames
• Greater Speed
• Undersea
• Unstructured Environments
• Mobile Manipulator
• Contact Point
• Precision And Recall
• Depth Images
• Inference Time
• Trajectory Length
• Camera Frame
• Input Point
• Camera Pose
• Stereo Camera
• Proportion Of Points
• Input Point Cloud
• Sparse Tensor
• Visual Servoing
• Oregon State University
• Temporal Convolution
• Unseen Objects
• Intel RealSense
• Ephemeris
• Temporal Noise
• Learning Formulation
• Homogeneous Matrix