State-Only Imitation Learning for Dexterous Manipulation

Modern model-free reinforcement learning methods have recently demonstrated impressive results on a number of problems. However, complex domains like dexterous manipulation remain a challenge due to the high sample complexity. To address this, current approaches employ expert demonstrations in the form of state-action pairs, which are difficult to obtain for real-world settings such as learning from videos. In this paper, we move toward a more realistic setting and explore state-only imitation learning. To tackle this setting, we train an inverse dynamics model and use it to predict actions for state-only demonstrations. The inverse dynamics model and the policy are trained jointly. Our method performs on par with state-action approaches and considerably outperforms RL alone. By not relying on expert actions, we are able to learn from demonstrations with different dynamics, morphologies, and objects. Videos available on the ${\text{project page}}$.

Authors

• Ilija Radosavovic
• Xiaolong Wang 0004
• Lerrel Pinto
• Jitendra Malik

Keywords

• Morphology
• Reinforcement learning
• Soil
• Predictive models
• Complexity theory
• Intelligent robots
• Videos
• Imitation Learning
• Dexterous Manipulation
• Inverse Model
• Pair Formation
• Model-free Reinforcement Learning
• Expert Demonstrations
• High-dimensional
• State Space
• Simulation Method
• Manipulation Tasks
• Training Examples
• Reward Function
• Domain Adaptation
• Policy Network
• Policy Gradient
• Virtual Reality Headset
• Replay Buffer
• Inverse Reinforcement Learning
• Consecutive States
• Model-based Reinforcement Learning
• Chamfer Distance