Detection-based object labeling in 3D scenes

We propose a view-based approach for labeling objects in 3D scenes reconstructed from RGB-D (color+depth) videos. We utilize sliding window detectors trained from object views to assign class probabilities to pixels in every RGB-D frame. These probabilities are projected into the reconstructed 3D scene and integrated using a voxel representation. We perform efficient inference on a Markov Random Field over the voxels, combining cues from view-based detection and 3D shape, to label the scene. Our detection-based approach produces accurate scene labeling on the RGB-D Scenes Dataset and improves the robustness of object detection.

Authors

• Kevin Lai 0001
• Liefeng Bo
• Xiaofeng Ren
• Dieter Fox

Keywords

• Detectors
• Labeling
• Feature extraction
• Shape
• Object detection
• Training
• Videos
• 3D Scene
• Object Labels
• Markov Random Field
• Objects In The Scene
• Accurate Labels
• Objective View
• Pixel In Frame
• Detection Training
• Local Features
• Visual Features
• 3D Reconstruction
• Probability Function
• Precision And Recall
• Point Cloud
• Bounding Box
• RGB Images
• Small Objects
• Depth Images
• 3D Point
• Histogram Of Oriented Gradients
• Surface Normals
• Pairwise Terms
• Object Boundaries
• Scene Reconstruction
• Background Class
• Score Map
• 3D Point Cloud
• RGB-D Images
• Shape Features