A Simple Reinforcement Learning Algorithm for Biped Walking

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately place the swing leg. This decision is based on a learned model of the Poincare map of the periodic walking pattern. The model maps from a state at the middle of a step and foot placement to a state at next middle of a step. We also modify the desired walking cycle frequency based on online measurements. We present simulation results, and are currently implementing this approach on an actual biped robot.

Authors

• Jun Morimoto
• Gordon Cheng
• Christopher G. Atkeson
• Garth Zeglin

Keywords

• Learning
• Legged locomotion
• Frequency estimation
• Foot
• Timing
• Phase estimation
• Humanoid robots
• Leg
• Humans
• Oscillators
• Bipedal Walking
• Walking Pattern
• Present Simulation Results
• Poincaré Map
• Model-based Reinforcement Learning
• Learning Rate
• Value Function
• Knee Joint
• Parameter Vector
• Function Approximation
• Hip Joint
• Joint Angles
• Distance Metrics
• Learning Control
• Reinforcement Learning Framework
• Policy Outputs
• Simulated Robot
• Adaptive Gain
• Natural Walking
• Walking Period
• Hip Position
• Frequency Of Walking
• Negative Reward
• Phase Resetting
• Walking Stability
• Estimation Method
• Stance Leg
• Reinforcement Learning Methods
• Body Position