Learning to Map Efficiently by Active Echolocation

Using visual SLAM to map new environments requires time-consuming visits to all regions for data collection. We propose an approach to estimate maps of areas beyond the visible regions using a cheap and readily available modality of data-sound. We introduce the idea of an active audio-visual mapping agent. Besides collecting visual data, the proposed agent emits sounds during navigation, captures the echoes, and uses them to accurately map unknown areas. We propose a reinforcement learning based method that simultaneously trains models to 1) estimate a map from the visual data, 2) output navigation actions, 3) output the decision to emit a sound and 4) refine estimated mans using the cantured audio. Our agent is trained and tested on 85 real-world homes from the Matterport3D dataset using the Habitat and SoundSpaces simulators for visual and audio data. Our method, unlike visual-data reliant approaches, yields more accurate maps with broader environmental coverage. In addition, compared to an agent that continually emits sounds, we observe that intelligently choosing when to emit sounds leads to accurate maps obatined with greater efficiency.

Authors

• Xixi Hu 0001
• Senthil Purushwalkam
• David Harwath
• Kristen Grauman

Keywords

• Visualization
• Simultaneous localization and mapping
• Navigation
• Habitats
• Reinforcement learning
• Data collection
• Data models
• Intelligence
• Habitat
• Visible Light
• Data Visualization
• Accurate Mapping
• Activation Maps
• Audio Data
• Home Environment
• Ground Plane
• Visual Input
• Global Map
• Local Map
• Depth Images
• Depth Camera
• Reward Function
• Posterior Mode
• Auditory Input
• Sound Localization
• Mapping Efficiency
• Scene Structure
• Laser Ranging
• RGB-D Images
• Environment Map
• Sound Policies
• Reinforcement Learning Problem