Learning Primitive Skills for Mobile Robots

Achieving effective task performance on real mobile robots is a great challenge when hand-coding algorithms, both due to the amount of effort involved and manually tuned parameters required for each skill. Learning algorithms instead have the potential to lighten up this challenge by using one single set of training parameters for learning different skills, but the question of the feasibility of such learning in real robots remains a research pursuit. We focus on a kind of mobile robot system - the robot soccer “small-size” domain, in which tactical and high-level team strategies build upon individual robot ball-based skills. In this paper, we present our work using a Deep Reinforcement Learning algorithm to learn three real robot primitive skills in continuous action space: go-to-ball, turn-and-shoot and shoot-goalie, for which there is a clear success metric to reach a destination or score a goal. We introduce the state and action representation, as well as the reward and network architecture. We describe our training and testing using a simulator of high physical and hardware fidelity. Then we test the policies trained from simulation on real robots. Our results show that the learned skills achieve an overall better success rate at the expense of taking 0. 29 seconds slower on average for all three skills. In the end, we show that our policies trained in simulation have good performance on real robots by directly transferring the policy.

Authors

• Yifeng Zhu
• Devin Schwab
• Manuela Veloso

Keywords

• Robot kinematics
• Sports
• Task analysis
• Training
• Legged locomotion
• Mobile Robot
• Primitive Skills
• Learning Algorithms
• Learning Skills
• Robotic System
• State Representation
• Deep Reinforcement Learning
• Real Robot
• Deep Reinforcement Learning Algorithm
• Single Set Of Parameters
• Continuous Action Space
• Robot Learning
• Rest Of The Paper
• Angular Velocity
• Angle Difference
• Evolutionary Strategy
• Reward Function
• Linear Velocity
• Markov Decision Process
• Policy Learning
• Policy Gradient Algorithm
• Replay Memory
• Training Policy
• Ball Possession
• Arbitrary Orientation
• Relational Coordination
• Deep Q-network
• Training Curves