Localization and Mapping using Instance-specific Mesh Models

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.

Authors

• Qiaojun Feng
• Yue Meng
• Mo Shan
• Nikolay Atanasov 0001

Keywords

• Visualization
• Technological innovation
• Accuracy
• Shape
• Semantics
• Robot vision systems
• Buildings
• Transportation
• Cameras
• Image reconstruction
• Mesh Model
• Object Detection
• Semantic Information
• Camera Images
• Object Shape
• Object Pose
• KITTI Dataset
• Monocular Camera
• Intersection Over Union
• Bounding Box
• Semantic Segmentation
• Model Projections
• Visual Perspective
• Localization Task
• Pose Estimation
• Triangular Mesh
• Simultaneous Localization And Mapping
• Camera Pose
• Object Instances
• Inliers
• Car Model
• Mesh Vertices
• Visual-inertial Odometry
• Map Tasks
• Factor Graph
• Contextual Reasons
• Keypoint Detection
• Detection Confidence
• Object Tracking
• Multiple Frames