Divide & Conquer Imitation Learning

When cast into the Deep Reinforcement Learning framework, many robotics tasks require solving a long horizon and sparse reward problem, where learning algorithms struggle. In such context, Imitation Learning (IL) can be a powerful approach to bootstrap the learning process. However, most IL methods require several expert demonstrations which can be prohibitively difficult to acquire. Only a handful of IL algorithms have shown efficiency in the context of an extreme low expert data regime where a single expert demonstration is available. In this paper, we present a novel algorithm designed to imitate complex robotic tasks from the states of an expert trajectory. Based on a sequential inductive bias, our method divides the complex task into smaller skills. The skills are learned into a goal-conditioned policy that is able to solve each skill individually and chain skills to solve the entire task. We show that our method imitates a non-holonomic navigation task and scales to a complex simulated robotic manipulation task with very high sample efficiency.

Authors

• Alexandre Chenu
• Nicolas Perrin-Gilbert
• Olivier Sigaud

Keywords

• Deep learning
• Navigation
• Reinforcement learning
• Trajectory
• Task analysis
• Intelligent robots
• Imitation Learning
• Learning Algorithms
• Sampling Efficiency
• Manipulation Tasks
• Deep Reinforcement Learning
• Robot Manipulator
• Robotic Tasks
• Deep Reinforcement Learning Framework
• Expert Demonstrations
• Value Function
• State Space
• Reward Function
• Matching Condition
• Starting State
• Definition Of Space
• Deep Reinforcement Learning Algorithm
• Physical Robot
• Wasserstein Distance