Q-RAN: A Constructive Reinforcement Learning Approach for Robot Behavior Learning

This paper presents a learning system that uses Q-learning with a resource allocating network (RAN) for behavior learning in mobile robotics. The RAN is used as a function approximator, and Q-learning is used to learn the control policy in 'off-policy' fashion that enables learning to be bootstrapped by a prior knowledge controller, thus speeding up the reinforcement learning. Our approach is verified on a PeopleBot robot executing a visual servoing based docking behavior in which the robot is required to reach a goal pose. Further experiments show that the RAN network can also be used for supervised learning prior to reinforcement learning in a layered architecture, thus further improving the performance of the docking behavior

Authors

• Jun Li
• Achim J. Lilienthal
• Tomás Martínez-Marín
• Tom Duckett

Keywords

• Learning systems
• Radio access networks
• Intelligent robots
• State-space methods
• Backpropagation
• Neurons
• Robotics and automation
• Mobile robots
• Resource management
• Visual servoing
• Supervised Learning
• Learning System
• Function Approximation
• Mobile Robot
• Neural Network
• Time Step
• Artificial Neural Network
• State Space
• Angular Velocity
• Radial Basis Function
• Avoidance Behavior
• Imaging Center
• Goal State
• Successful Control
• Robot Control
• Object Tracking
• Linear Control
• Learning Control
• Real Robot
• Global Frame
• HSV Color
• Learning Layer
• Teaching Signal
• Radial Basis Function Network
• Goal Position
• Translational Velocity
• Learning Process
• Obstacle Avoidance
• Camera Control