Mapping in Indoor Environments Including Transparent Objects Using Stereo Polarization Camera and Projector

This paper proposes a method for generating maps in indoor environments that include transparent objects by using a stereo polarization camera and projector. Conventional sensors like LiDAR and stereo cameras struggle with glass, as they rely on diffuse reflection, while glass allows light to pass through. In contrast, polarization cameras can measure light polarization and estimate surface normals, enabling depth estimation by combining polarization and RGB information. However, when measuring transparent objects, reflected and transmitted light cancel each other out, reducing polarization contrast, and the RGB information causes the depth estimation to output the depth of objects behind the glass. To address this issue, this paper proposes a novel method that (1) improves the S/N ration in polarization measument via diffuse reflection on non-glass regions and (2) masks out the RGB color from polarimetric depth estimation to not compute depth map of objects behind the glass to obtain depth images that include glass surfaces. Additionally, (3) in the mapping part, depth estimation is repeated at multiple locations, and the results are integrated using self-localization to generate a complete environmental map. Experiments in an indoor environment confirmed the effectiveness of the proposed method, enabling glass-inclusive depth estimation and successful map generation on a mobile robot.

Authors

• Yusuke Ogihara
• Hiroshi Higuchi
• Takuya Igaue
• Qi An
• Atsushi Yamashita

Keywords

• Accuracy
• Three-dimensional displays
• Depth measurement
• Robot vision systems
• Glass
• Cameras
• Reflection
• Indoor environment
• Windows
• Signal to noise ratio
• Indoor Environments
• Transparent Objects
• Polarization Camera
• Polarized Light
• Light Detection And Ranging
• Diffuse Reflectance
• Glass Surface
• Depth Map
• Depth Images
• Mobile Robot
• Depth Estimation
• RGB Color
• Environment Map
• Stereo Camera
• Surface Normals
• Object Depth
• Polarization Information
• RGB Information
• Light Source
• Incident Angle
• Left Camera
• Polarization Imaging
• RGB Images
• Degree Of Polarization
• Structure From Motion
• Point Cloud
• Selective Illumination
• Texture Of Objects
• 3D Coordinates
• Segmentation Results