Imitation learning with generalized task descriptions

In this paper, we present an approach that allows a robot to observe, generalize, and reproduce tasks observed from multiple demonstrations. Motion capture data is recorded in which a human instructor manipulates a set of objects. In our approach, we learn relations between body parts of the demonstrator and objects in the scene. These relations result in a generalized task description. The problem of learning and reproducing human actions is formulated using a dynamic Bayesian network (DBN). The posteriors corresponding to the nodes of the DBN are estimated by observing objects in the scene and body parts of the demonstrator. To reproduce a task, we seek for the maximum-likelihood action sequence according to the DBN. We additionally show how further constraints can be incorporated online, for example, to robustly deal with unforeseen obstacles. Experiments carried out with a real 6-DoF robotic manipulator as well as in simulation show that our approach enables a robot to reproduce a task carried out by a human demonstrator. Our approach yields a high degree of generalization illustrated by performing a pick-and-place and a whiteboard cleaning task.

Authors

• Clemens Eppner
• Jürgen Sturm
• Maren Bennewitz
• Cyrill Stachniss
• Wolfram Burgard

Keywords

• Robots
• Humans
• Layout
• Manipulators
• Cleaning
• Robotics and automation
• Bayesian methods
• Contracts
• Maximum likelihood estimation
• Robustness
• Description Task
• Imitation Learning
• Motion Capture
• Robot Manipulator
• Objects In The Scene
• Dynamic Bayesian Network
• Time Step
• Remainder Of This Paper
• Hidden Markov Model
• Additional Constraints
• Joint Angles
• Learning Phase
• Object Position
• Joint Space
• Configuration Space
• End-effector
• Reproductive Phase
• Presence Of Obstacles
• Object Pose
• Joint Configuration
• Rapidly-exploring Random Tree
• Number Of Demonstrations
• World Coordinate