Egocentric Spatial Memory

Egocentric spatial memory (ESM) defines a memory system with encoding, storing, recognizing and recalling the spatial information about the environment from an egocentric perspective. We introduce an integrated deep neural network architecture for modeling ESM. It learns to estimate the occupancy state of the world and progressively construct top-down 2D global maps from egocentric views in a spatially extended environment. During the exploration, our proposed ESM model updates belief of the global map based on local observations using a recurrent neural network. It also augments the local mapping with a novel external memory to encode and store latent representations of the visited places over longterm exploration in large environments which enables agents to perform place recognition and hence, loop closure. Our proposed ESM network contributes in the following aspects: (1) without feature engineering, our model predicts free space based on egocentric views efficiently in an end-to-end manner; (2) different from other deep learning-based mapping system, ESMN deals with continuous actions and states which is vitally important for robotic control in real applications. In the experiments, we demonstrate its accurate and robust global mapping capacities in 3D virtual mazes and realistic indoor environments by comparing with several competitive baselines.

Authors

• Mengmi Zhang
• Keng Teck Ma
• Shih-Cheng Yen
• Joo Hwee Lim
• Qi Zhao 0001
• Jiashi Feng

Keywords

• Computer architecture
• Cameras
• Navigation
• Microprocessors
• Sensors
• Task analysis
• Motion measurement
• Spatial Memory
• Neural Network
• Deep Network
• Deep Neural Network
• Spatial Information
• Free Space
• Recurrent Neural Network
• Continuous Action
• Indoor Environments
• Global Map
• Local Map
• Memory System
• Feature Engineering
• External Memory
• Loop Closure
• Place Recognition
• Egocentric Perspective
• Time Step
• Previous Step
• Long Short-term Memory
• Grid Unit
• Ego-motion
• Camera View
• Simultaneous Localization And Mapping
• Continuous Action Space
• Prediction Map
• Past Information
• Ground Truth Map
• Local Space
• Motion Detection