Learning Shape-based Representation for Visual Localization in Extremely Changing Conditions

Visual localization is an important task for applications such as navigation and augmented reality, but is a challenging problem when there are changes in scene appearances through day, seasons, or environments. In this paper, we present a convolutional neural network (CNN)-based approach for visual localization across normal to drastic appearance variations such as pre- and post-disaster cases. Our approach aims to address two key challenges: (1) to reduce the biases based on scene textures as in traditional CNNs, our model learns a shape-based representation by training on stylized images; (2) to make the model robust against layout changes, our approach uses the estimated dominant planes of query images as approximate scene coordinates. Our method is evaluated on various scenes including a simulated disaster dataset to demonstrate the effectiveness of our method in significant changes of scene layout. Experimental results show that our method provides reliable camera pose predictions in various changing conditions.

Authors

• Hae-Gon Jeon
• Sunghoon Im 0001
• Jean Oh
• Martial Hebert

Keywords

• Visualization
• Shape
• Cameras
• Semantics
• Robustness
• Geometry
• Buildings
• Changes In Conditions
• Visual Localization
• Convolutional Neural Network
• Changes In Appearance
• Appearance Variations
• Camera Pose
• Query Image
• Traditional Convolutional Neural Network
• Pose Prediction
• Severe Conditions
• Single Image
• Seasonal Changes
• Color Images
• 3D Space
• Position Error
• Reference Image
• Fully-connected Layer
• Depth Map
• High Bias
• Pose Estimation
• Parameter Plane
• Scene Geometry
• Style Transfer
• 3D Information
• Changes In Geometry
• CNN-based Approaches
• Image Database
• Geometric Changes
• Scene Model
• Rotation Error