Demonstration-Bootstrapped Autonomous Practicing via Multi-Task Reinforcement Learning

Reinforcement learning systems have the potential to enable continuous improvement in unstructured environments, leveraging data collected autonomously. However, in practice these systems require significant amounts of instrumentation or human intervention to learn in the real world. In this work, we propose a system for reinforcement learning that leverages multi-task reinforcement learning bootstrapped with prior data to enable continuous autonomous practicing, minimizing the number of resets needed while being able to learn temporally extended behaviors. We show how appropriately provided prior data can help bootstrap both low-level multi-task policies and strategies for sequencing these tasks one after another to enable learning with minimal resets. This mechanism enables our robotic system to practice with minimal human intervention at training time, while being able to solve long horizon tasks at test time. We show the efficacy of the proposed system on a challenging kitchen manipulation task both in simulation and the real world, demonstrating the ability to practice autonomously in order to solve temporally extended problems.

Authors

• Abhishek Gupta 0004
• Corey Lynch
• Brandon Kinman
• Garrett Peake
• Sergey Levine
• Karol Hausman

Keywords

• Training
• Sequential analysis
• Automation
• Instruments
• Reinforcement learning
• Multitasking
• Behavioral sciences
• Multi-task Reinforcement Learning
• Training Time
• Human Intervention
• Multi-task Learning
• Minimal Human Intervention
• Neural Network
• Learning Process
• Shortest Path
• National Interests
• Randomized Control
• State Machine
• Goal State
• Combination Of Elements
• Sequential Task
• End-effector
• Robot Manipulator
• Policy Learning
• Reinforcement Learning Algorithm
• Continuous Practice
• High-level Policy
• Interest Goals
• Dijkstra’s Algorithm
• Imitation Learning
• Autonomous Learning
• Wide Range Of Tasks
• Multiple Tasks
• Learning Behavior