Towards learning hierarchical skills for multi-phase manipulation tasks

Most manipulation tasks can be decomposed into a sequence of phases, where the robot's actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e. g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.

Authors

• Oliver Kroemer
• Christian G. Daniel
• Gerhard Neumann
• Herke van Hoof
• Jan Peters 0001

Keywords

• Robots
• Hidden Markov models
• Computational modeling
• Libraries
• Sequential analysis
• Shape
• Learning (artificial intelligence)
• Manipulation Tasks
• Phase Transition
• Value Function
• Probabilistic Model
• High-level Policy
• Model-based Reinforcement Learning
• Right-hand
• Learning Models
• Transition State
• State Space
• Expectation Maximization
• Multi-label
• Steps Of Algorithm
• Task Model
• Part Of The State
• Goal State
• Reward Function
• Policy Learning
• Part Of Environment
• Imitation Learning
• Multiphase Model
• Maximum Step
• Starting State
• Hidden Variables
• Reinforcement Learning Approach
• Policy Search
• Kullback-Leibler
• Position Of Box
• Joint Angles