Learning Sample-Efficient Target Reaching for Mobile Robots

In this paper, we propose a novel architecture and a self-supervised policy gradient algorithm, which employs unsupervised auxiliary tasks to enable a mobile robot to learn how to navigate to a given goal. The dependency on the global information is eliminated by providing only sparse range-finder measurements to the robot. The partially observable planning problem is addressed by splitting it into a hierarchical process. We use convolutional networks to plan locally, and a differentiable memory to provide information about past time steps in the trajectory. These modules, combined in our network architecture, produce globally consistent plans. The sparse reward problem is mitigated by our modified policy gradient algorithm. We model the robots uncertainty with unsupervised tasks to force exploration. The novel architecture we propose with the modified version of the policy gradient algorithm allows our robot to reach the goal in a sample efficient manner, which is orders of magnitude faster than the current state of the art policy gradient algorithm. Simulation and experimental results are provided to validate the proposed approach.

Authors

• Arbaaz Khan
• Vijay Kumar 0001
• Alejandro Ribeiro

Keywords

• Task analysis
• Robot sensing systems
• Planning
• Encoding
• Uncertainty
• Training
• Mobile Robot
• Convolutional Network
• Scan Range
• Policy Gradient
• Auxiliary Task
• Unsupervised Tasks
• Policy Gradient Algorithm
• Neural Network
• Deep Learning
• Value Function
• Supervised Learning
• Control Network
• Use Of Markers
• Optimal Policy
• Robotic Arm
• Sparse Representation
• Deep Reinforcement Learning
• Markov Decision Process
• Local Observations
• Environmental Planning
• External Memory
• Real Robot
• Value Iteration
• Unknown Environment
• Entire Architecture
• Reward Signal
• Trajectory Planning
• LiDAR Scans
• Hyperparameters
• State Prediction