Mapless Humanoid Navigation Using Learned Latent Dynamics

In this paper, we propose a novel Deep Reinforcement Learning approach to address the mapless navigation problem, in which the locomotion actions of a humanoid robot are taken online based on the knowledge encoded in learned models. Planning happens by generating open-loop trajectories in a learned latent space that captures the dynamics of the environment. Our planner considers visual (RGB images) and non-visual observations (e. g. , attitude estimations). This confers the agent upon awareness not only of the scenario, but also of its own state. In addition, we incorporate a termination likelihood predictor model as an auxiliary loss function of the control policy, which enables the agent to anticipate terminal states of success and failure. In this manner, the sample efficiency of the approach for episodic tasks is increased. Our model is evaluated on the NimbRo-OP2X humanoid robot that navigates in scenes avoiding collisions efficiently in simulation and with the real hardware.

Authors

• André Brandenburger
• Diego Rodriguez
• Sven Behnke

Keywords

• Visualization
• Navigation
• Tracking
• Humanoid robots
• Reinforcement learning
• Predictive models
• Planning
• Latent Dynamics
• Collision
• Learning Models
• Visual Observation
• Latent Space
• RGB Images
• Sampling Efficiency
• Termination Condition
• Deep Reinforcement Learning
• Humanoid Robot
• Reinforcement Learning Approach
• Real Hardware
• Deep Reinforcement Learning Approach
• Dynamic Model
• Image Segmentation
• Target Location
• Kullback-Leibler
• Semantic Segmentation
• Reward Function
• Markov Decision Process
• Real Robot
• Roll Angle
• Target Pose
• Dynamic Obstacles
• Latent State
• Goal Of The Agent
• Policy Learning
• Gait Velocity
• Failure Conditions
• None Of These Approaches