Policy Gradient Methods for Robotics

The acquisition and improvement of motor skills and control policies for robotics from trial and error is of essential importance if robots should ever leave precisely pre-structured environments. However, to date only few existing reinforcement learning methods have been scaled into the domains of high-dimensional robots such as manipulator, legged or humanoid robots. Policy gradient methods remain one of the few exceptions and have found a variety of applications. Nevertheless, the application of such methods is not without peril if done in an uninformed manner. In this paper, we give an overview on learning with policy gradient methods for robotics with a strong focus on recent advances in the field. We outline previous applications to robotics and show how the most recently developed methods can significantly improve learning performance. Finally, we evaluate our most promising algorithm in the application of hitting a baseball with an anthropomorphic arm

Authors

• Jan Peters 0001
• Stefan Schaal

Keywords

• Gradient methods
• Intelligent robots
• Learning
• Robot control
• Humanoid robots
• Anthropomorphism
• Orbital robotics
• Probability distribution
• Optimal control
• USA Councils
• Gradient Method
• Policy Gradient
• Policy Gradient Method
• Motor Skills
• Reinforcement Learning Methods
• Humanoid Robot
• Improvements In Motor Skills
• Application Of Such Methods
• Degrees Of Freedom
• Likelihood Ratio
• Supervised Learning
• Convergence Rate
• Finite Difference
• Implementation Of Algorithm
• Gradient Approximation
• Finite Difference Method
• Update Step
• Joint Velocity
• Real Robot
• Natural Gradient
• Likelihood Ratio Method
• Policy Parameters
• Imitation Learning
• Robot Learning
• Simulation Optimization
• Robotic Tasks
• Generalized Likelihood Ratio
• Dynamic Programming
• Learning Rate