Self-supervised Transparent Liquid Segmentation for Robotic Pouring

Liquid state estimation is important for robotics tasks such as pouring; however, estimating the state of transparent liquids is a challenging problem. We propose a novel segmentation pipeline that can segment transparent liquids such as water from a static, RGB image without requiring any manual annotations or heating of the liquid for training. Instead, we use a generative model that is capable of translating images of colored liquids into synthetically generated transparent liquid images, trained only on an unpaired dataset of colored and transparent liquid images. Segmentation labels of colored liquids are obtained automatically using background subtraction. Our experiments show that we are able to accurately predict a segmentation mask for transparent liquids without requiring any manual annotations. We demonstrate the utility of transparent liquid segmentation in a robotic pouring task that controls pouring by perceiving the liquid height in a transparent cup. Accompanying video and supplementary materials can be found at https://sites.google.com/view/transparentliquidpouring.

Authors

• Gautham Narayan Narasimhan
• Kai Zhang
• Ben Eisner
• Xingyu Lin
• David Held

Keywords

• Training
• Image segmentation
• Liquids
• Annotations
• Water heating
• Manuals
• Containers
• Transparent Liquid
• Background Subtraction
• RGB Images
• Manual Annotation
• Robotic Tasks
• Synthetic Generation
• Segmentation Labels
• Liquid Heat
• Liquid Height
• Diverse Backgrounds
• Input Image
• Workspace
• Intersection Over Union
• Bounding Box
• Segmentation Model
• Robotic System
• Supplementary Materials For Details
• Ground Truth Labels
• Target Domain
• Robotic Arm
• Transparent Container
• Source Domain
• Synthetic Images
• Transparent Model
• Color Jittering
• Liquid Motion
• Perspective Camera
• Translational Model
• Tedious Process
• Infrared Imaging