Affordance Learning from Play for Sample-Efficient Policy Learning

Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visual affordance model from human teleoperated play data and leverages it to enable efficient policy learning and motion planning. We combine model-based planning with model-free deep reinforcement learning (RL) to learn policies that favor the same object regions favored by people, while requiring minimal robot interactions with the environment. We evaluate our algorithm, Visual Affordance-guided Policy Optimization (VAPO), with both diverse simulation manipulation tasks and real world robot tidy-up experiments to demonstrate the effectiveness of our affordance-guided policies. We find that our policies train 4 × faster than the baselines and generalize better to novel objects because our visual affordance model can anticipate their affordance regions.

Authors

• Jessica Borja-Diaz
• Oier Mees
• Gabriel Kalweit
• Lukás Hermann
• Joschka Boedecker
• Wolfram Burgard

Keywords

• Visualization
• Solid modeling
• Three-dimensional displays
• Shape
• Affordances
• Robot control
• Planning
• Policy Learning
• Affordance Learning
• Path Planning
• Manipulation Tasks
• Visual Model
• Deep Reinforcement Learning
• Convolutional Neural Network
• Simulation Experiments
• Local Policy
• Camera Images
• Reward Function
• Real-world Experiments
• Markov Decision Process
• End-effector
• Objects In The Scene
• Euler Angles
• Successful Interaction
• Semantic Labels
• Reinforcement Learning Methods
• Reinforcement Learning Policy
• Model-free Reinforcement Learning
• Object Affordances
• RGB-D Images
• Training Policy
• Strong Prior
• Observation Space
• Tracking System
• Representation Learning
• 3D Position