Transferable Task Execution from Pixels through Deep Planning Domain Learning

While robots can learn models to solve many manipulation tasks from raw visual input, they cannot usually use these models to solve new problems. On the other hand, symbolic planning methods such as STRIPS have long been able to solve new problems given only a domain definition and a symbolic goal, but these approaches often struggle on the real world robotic tasks due to the challenges of grounding these symbols from sensor data in a partially-observable world. We propose Deep Planning Domain Learning (DPDL), an approach that combines the strengths of both methods to learn a hierarchical model. DPDL learns a high-level model which predicts values for a large set of logical predicates consisting of the current symbolic world state, and separately learns a low-level policy which translates symbolic operators into executable actions on the robot. This allows us to perform complex, multistep tasks even when the robot has not been explicitly trained on them. We show our method on manipulation tasks in a photorealistic kitchen scenario.

Authors

• Kei Kase
• Chris Paxton 0001
• Hammad Mazhar
• Tetsuya Ogata
• Dieter Fox

Keywords

• Task analysis
• Planning
• Robot sensing systems
• Grounding
• Robustness
• Feature extraction
• Task Execution
• Planning Domain
• Sensor Data
• Global Status
• Manipulation Tasks
• Robotic Tasks
• First-order Logic
• High-level Model
• Preconditioning
• Convolutional Layers
• Simulation Environment
• Semantic Segmentation
• RGB Images
• Path Planning
• Joint Angles
• Depth Images
• Boolean Logic
• Pose Estimation
• Variational Autoencoder
• Latent Features
• Logic State
• Inverse Reinforcement Learning
• Task Planning
• High-level Policy
• Video Presentation
• Current Operation