Visual Robot Task Planning

Prospection is key to solving challenging problems in new environments, but it has not been deeply explored as applied to task planning for perception-driven robotics. We propose visual robot task planning, where we take in an input image and must generate a sequence of high-level actions and associated observations that achieve some task. In this paper, we describe a neural network architecture and associated planning algorithm that (1) learns a representation of the world that can generate prospective futures, (2) uses this generative model to simulate the result of sequences of high-level actions in a variety of environments, and (3) evaluates these actions via a variant of Monte Carlo Tree Search to find a viable solution to a particular problem. Our approach allows us to visualize intermediate motion goals and learn to plan complex activity from visual information, and used this to generate and visualize task plans on held-out examples of a block-stacking simulation.

Authors

• Chris Paxton 0001
• Yotam Barnoy
• Kapil D. Katyal
• Raman Arora
• Gregory D. Hager

Keywords

• Task analysis
• Planning
• Visualization
• Predictive models
• Robots
• Transforms
• Computer architecture
• Visual Task
• Task Planning
• Neural Network
• Sequence Of Actions
• Tree Search
• Prospection
• Monte Carlo Tree
• Monte Carlo Tree Search
• Prediction Model
• Effect Of Activity
• Value Function
• Morphine
• Simple Task
• Representation Learning
• Path Planning
• Hidden State
• Transformation Function
• Skip Connections
• Scene Images
• Hidden Space
• Planning Problem
• Learning Spaces
• L1 Loss
• Instance Normalization
• Value Iteration
• Deep Generative Models