Efficient policy search with a parameterized skill memory

Motion primitives are an established paradigm to generate complex motions from simpler building blocks. A much less addressed issue is at which level to encode and organize a library of motion primitives, and how to retrieve motion primitives from a library that fit a particular task. This paper proposes a parameterized skill memory, which organizes a set of motion primitives in a low-dimensional, topology-preserving embedding space. The skill memory acts as a pivotal mechanism that links low-dimensional skill parametrizations to motion primitive parameters and complete motion trajectories. The skill memory is implemented by means of a dynamical system which features continuous generalization of motion shapes. It is shown that the low-dimensional skill parametrization is beneficial for efficient, reward-based retrieval of motion primitives and simplifies the shaping of reward functions. The excellent generalization of motion shapes by parameterized skill memories from few training examples is demonstrated in a bimanual manipulation task with the humanoid robot iCub.

Authors

• René Felix Reinhart
• Jochen J. Steil

Keywords

• Training
• Shape
• Trajectory
• Libraries
• Motion segmentation
• Humanoid robots
• Policy Search
• Skill Memory
• Parameterized Skill
• System Dynamics
• Latent Space
• Training Examples
• Reward Function
• Pivotal Mechanism
• Motion Primitives
• Left Side
• Time Constant
• Left Hand
• Parametrized
• Radial Basis Function
• Low-dimensional Space
• Level Of Representation
• Improvement In Skills
• Single Vector
• Periodic Motion
• End-effector
• Radial Basis Function Network
• Low-dimensional Embedding
• Policy Improvement
• Reward Evaluation
• Input Modalities
• Policy Parameters
• Motion Generation
• Discrete Time Steps
• Position Embedding
• End-effector Position