Learning motion primitive goals for robust manipulation

Applying model-free reinforcement learning to manipulation remains challenging for several reasons. First, manipulation involves physical contact, which causes discontinuous cost functions. Second, in manipulation, the end-point of the movement must be chosen carefully, as it represents a grasp which must be adapted to the pose and shape of the object. Finally, there is uncertainty in the object pose, and even the most carefully planned movement may fail if the object is not at the expected position. To address these challenges we 1) present a simplified, computationally more efficient version of our model-free reinforcement learning algorithm PI 2; 2) extend PI 2 so that it simultaneously learns shape parameters and goal parameters of motion primitives; 3) use shape and goal learning to acquire motion primitives that are robust to object pose uncertainty. We evaluate these contributions on a manipulation platform consisting of a 7-DOF arm with a 4-DOF hand.

Authors

• Freek Stulp
• Evangelos A. Theodorou
• Mrinal Kalakrishnan
• Peter Pastor
• Ludovic Righetti
• Stefan Schaal

Keywords

• Shape
• Trajectory
• Uncertainty
• Robots
• Cost function
• Learning
• Grasping
• Motion Primitives
• Shape Parameter
• Object Shape
• Object Pose
• Model-free Reinforcement Learning
• Time Step
• Parameter Space
• Average Cost
• Joint Angles
• Manipulation Tasks
• Object Position
• Distribution Of Positions
• Parameter Update
• Update Rule
• Position Uncertainty
• Reactive Control
• Learning Session
• Policy Improvement
• Uncertainty Distribution
• Movement Goals
• Movement Learning
• Reinforcement Learning Problem
• Number Of Updates