Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction

Instance segmentation of unknown objects from images is regarded as relevant for several robot skills including grasping, tracking and object sorting. Recent results from computer vision have shown that large hand-labeled datasets enable high segmentation performance. To overcome the time-consuming process of manually labeling data for new environments, we present a transfer learning approach for robots that learn to segment objects by interacting with their environment in a self-supervised manner. Our robot pushes unknown objects on a table and uses information from optical flow to create training labels given by object masks. To achieve this, we fine-tune an existing DeepMask instance segmentation network on the self-labeled training data acquired by the robot. We evaluate our trained network (SelfDeepMask) on a set of real images showing challenging and cluttered scenes with novel objects. Here, SelfDeepMask outperforms the DeepMask network trained on the COCO dataset by 8. 6% in average precision.

Authors

• Andreas Eitel
• Nico Hauff
• Wolfram Burgard

Keywords

• Instance segmentation
• Training
• Accuracy
• Transfer learning
• Training data
• Labeling
• Robots
• Optical flow
• Videos
• Sorting
• Physical Interaction
• Self-supervised Learning
• Learning For Segmentation
• Computer Vision
• Average Precision
• Segmentation Performance
• COCO Dataset
• Transfer Learning Approach
• Self-supervised Manner
• Sorts Of Objects
• Training Set
• Intersection Over Union
• Learning-based Methods
• Aforementioned Methods
• Interactive Segmentation
• Intersection Over Union Threshold
• Non-maximum Suppression
• Final Performance
• Image Patches
• Pre-trained Network
• Object Instances
• Motion Information
• Robot Learning
• Object Motion