Adversarial Imitation Learning from Video Using a State Observer

The imitation learning research community has recently made significant progress towards the goal of enabling artificial agents to imitate behaviors from video demonstrations alone. However, current state-of-the-art approaches developed for this problem exhibit high sample complexity due, in part, to the high-dimensional nature of video observations. Towards addressing this issue, we introduce here a new algorithm called Visual Generative Adversarial Imitation from Observation using a State Observer (VGAIfO-SO). At its core, VGAIfO-SO seeks to address sample inefficiency using a novel, self-supervised state observer, which provides estimates of lower-dimensional proprioceptive state representations from high-dimensional images. We show experimentally in several continuous control environments that VGAIfO-SO is more sample efficient than other IfO algorithms at learning from video-only demonstrations and can sometimes even achieve performance close to the Generative Adversarial Imitation from Observation (GAIfO) algorithm that has privileged access to the demonstrator's proprioceptive state information.

Authors

• Haresh Karnan
• Faraz Torabi
• Garrett Warnell
• Peter Stone 0001

Keywords

• Visualization
• Training data
• Reinforcement learning
• Learning (artificial intelligence)
• Observers
• Prediction algorithms
• Market research
• Generative Adversarial Networks
• State Observer
• Imitation Learning
• Adversarial Imitation Learning
• State Representation
• Sampling Efficiency
• Video Presentation
• Proprioceptive Information
• High-dimensional Image
• Convolutional Neural Network
• Prediction Error
• Multilayer Perceptron
• Visual Observation
• Performance Gap
• Reward Function
• Markov Decision Process
• Self-supervised Learning
• Discriminator Network
• Inverse Reinforcement Learning
• Model-based Reinforcement Learning
• Expert Demonstrations