Spiking Reinforcement Learning with Memory Ability for Mapless Navigation

Our study focuses on mapless navigation in robotics, which involves navigating without an established obstacle map of the environment. Spiking Neural Networks (SNNs) have recently been applied to this task using Deep Reinforcement Learning (DRL), but face challenges in dynamic and partially observable environments, as well as inaccuracies in transmitted data. To overcome these issues, we propose a Multi-Critic DDPG with Spiking Memory (MC-DDPGSM) framework. Our approach introduces a spiking Gate Recurrent Unit layer (Spiking-GRU) to provide memory function and evaluates the state-action value with multi-critic networks. The experimental results demonstrate that our method achieves better performance (success rate, navigation distance, navigation time spent, and power consumption) in complex navigation tasks compared to the state-of-the-art approaches. Furthermore, our model can be transferred to unseen environments without the need for fine-tuning.

Authors

• Bo Yang
• Mengwen Yuan
• Chengjun Zhang
• Chaofei Hong
• Gang Pan 0001
• Huajin Tang

Keywords

• Deep learning
• Power demand
• Navigation
• Neural networks
• Reinforcement learning
• Logic gates
• Task analysis
• Mapless Navigation
• Neural Network
• Deep Reinforcement Learning
• Spiking Neural Networks
• Gated Recurrent Unit
• Navigation Task
• Robot Navigation
• Unseen Environments
• Energy Consumption
• Output Layer
• Complex Environment
• Neuroanatomical
• Dynamic Environment
• Actor Network
• Reward Function
• Markov Decision Process
• Mobile Robot
• Floating-point Operations
• Critic Network
• Dynamic Scenarios
• Dynamic Obstacles
• Gated Recurrent Unit Layer
• Deep Reinforcement Learning Framework
• Static Obstacles