Planning from Pixels using Inverse Dynamics Models

Learning dynamics models in high-dimensional observation spaces can be challenging for model-based RL agents. We propose a novel way to learn models in a latent space by learning to predict sequences of future actions conditioned on task completion. These models track task-relevant environment dynamics over a distribution of tasks, while simultaneously serving as an effective heuristic for planning with sparse rewards. We evaluate our method on challenging visual goal completion tasks and show a substantial increase in performance compared to prior model-free approaches.

Authors

• Keiran Paster
• Sheila A. McIlraith
• Jimmy Ba

Keywords

• model based reinforcement learning
• deep reinforcement learning
• multi-task learning
• deep learning
• goal-conditioned reinforcement learning