Efficient Bimanual Manipulation Using Learned Task Schemas

We address the problem of effectively composing skills to solve sparse-reward tasks in the real world. Given a set of parameterized skills (such as exerting a force or doing a top grasp at a location), our goal is to learn policies that invoke these skills to efficiently solve such tasks. Our insight is that for many tasks, the learning process can be decomposed into learning a state-independent task schema (a sequence of skills to execute) and a policy to choose the parameterizations of the skills in a state-dependent manner. For such tasks, we show that explicitly modeling the schema’s state-independence can yield significant improvements in sample efficiency for model-free reinforcement learning algorithms. Furthermore, these schemas can be transferred to solve related tasks, by simply re-learning the parameterizations with which the skills are invoked. We find that doing so enables learning to solve sparse-reward tasks on real-world robotic systems very efficiently. We validate our approach experimentally over a suite of robotic bimanual manipulation tasks, both in simulation and on real hardware. See videos at http://tinyurl.com/chitnis-schema.

Authors

• Rohan Chitnis
• Shubham Tulsiani
• Saurabh Gupta 0001
• Abhinav Gupta 0001

Keywords

• Task analysis
• Learning (artificial intelligence)
• Neural networks
• Force
• Geometry
• End effectors
• Bimanual Manipulation
• Task Schema
• Related Tasks
• Robot Manipulator
• Policy Learning
• Reinforcement Learning Algorithm
• Real-world Tasks
• Robotic Tasks
• Model-free Reinforcement Learning
• Real Hardware
• Neural Network
• State Space
• Search Space
• Transfer Learning
• Raw Images
• Friction Coefficient
• Optimal Policy
• Deep Reinforcement Learning
• End-effector
• Hours Of Training
• Proximal Policy Optimization
• Simulated Task
• Object Pose
• Large Search Space
• Policy Gradient Method
• Hinge Joint
• Reinforcement Learning Methods
• Policy Gradient
• Problem Setup
• Continuous Parameters