Deep Object-Centric Representations for Generalizable Robot Learning

Robotic manipulation in complex open-world scenarios requires both reliable physical manipulation skills and effective and generalizable perception. In this paper, we propose using an object-centric prior and a semantic feature space for the perception system of a learned policy. We devise an object-level attentional mechanism that can be used to determine relevant objects from a few trajectories or demonstrations, and then immediately incorporate those objects into a learned policy. A task-independent attention locates possible objects in the scene, and a task-specific attention identifies which objects are predictive of the trajectories. The scope of the task-specific attention is easily adjusted by showing demonstrations with distractor objects or with diverse relevant objects. Our results indicate that this approach exhibits good generalization across object instances using very few samples, and can be used to learn a variety of manipulation tasks using reinforcement learning.

Authors

• Coline Devin
• Pieter Abbeel
• Trevor Darrell
• Sergey Levine

Keywords

• Task analysis
• Visualization
• Semantics
• Trajectory
• Computer vision
• Standards
• Robot Learning
• Object-centric Representation
• Attention Mechanism
• Semantic Features
• Manipulation Tasks
• Perceptual System
• Good Generalization
• Robot Manipulator
• Policy Learning
• Objects In The Scene
• Relevant Objects
• Object Instances
• Scope Of Attention
• Distractor Objects
• Deep Learning
• Deep Neural Network
• Visual Representation
• Object Detection
• Visual Features
• Inverse Reinforcement Learning
• Attention Vector
• Region Proposal Network
• Raw Pixel
• Global Policy
• Region Proposal
• Object Proposals
• Training Policy
• Deep Reinforcement Learning
• Semantic Components