Learning mobile manipulation actions from human demonstrations

Over the past years learning from demonstration has become a popular method to intuitively teach new skills to service robots without explicit programming. However, most teaching approaches in literature use kinesthetic training and do not include mobile platforms. Here, we present a novel approach to learn joint robot base and gripper action models from observing demonstrations carried out by a human teacher. To achieve this we adapt RGBD observations of the human teacher to the capabilities of the robot. We formulate a graph optimization problem that links observations with robot grasping capabilities and kinematic constraints between co-occurring base and gripper poses. In real world experiments we show that the robot is able to learn complex mobile manipulation tasks such as opening and driving through a door.

Authors

• Tim Welschehold
• Christian Dornhege
• Wolfram Burgard

Keywords

• Robots
• Trajectory
• Grippers
• Torso
• Mobile communication
• Optimization
• Mobile Manipulator
• Manipulation Actions
• Manipulation Tasks
• Real-world Experiments
• Mobile Platform
• Human Education
• Kinematic Constraints
• Service Robots
• Graph Optimization
• Robot Capabilities
• Mobility Tasks
• Robot Base
• Robotic Gripper
• Simulation Environment
• Error Function
• Graph Structure
• Depth Camera
• Fisher Information
• Object Motion
• Pose Estimation
• Human Hand
• Door Handle
• Object Trajectory
• Translational Part
• Rotational Part
• Hand Trajectory
• Handling Of Objects
• Automatic Localization
• Hand Motion
• Robot Motion