Generating object hypotheses in natural scenes through human-robot interaction

We propose a method for interactive modeling of objects and object relations based on real-time segmentation of video sequences. In interaction with a human, the robot can perform multi-object segmentation through principled modeling of physical constraints. The key contribution is an efficient multi-labeling framework, that allows object modeling and disambiguation in natural scenes. Object modeling and labeling is done in a real-time segmentation system, to which hypotheses and constraints denoting relations between objects can be added incrementally. Through instructions such as key presses or spoken words, a scene can be segmented in regions corresponding to multiple physical objects. The approach solves some of the difficult problems related to disambiguation of objects merged due to their direct physical contact. Results show that even a limited set of simple interactions with a human operator can substantially improve segmentation results.

Authors

• Niklas Bergström
• Mårten Björkman
• Danica Kragic

Keywords

• Image segmentation
• Robots
• Humans
• Clutter
• Three dimensional displays
• Image color analysis
• Real time systems
• Natural Scenes
• Human-robot Interaction
• Disambiguation
• Object Relations
• Human Operator
• Points In Space
• 3D Space
• Multiple Hypothesis
• Number Of Objects
• Motion Detection
• Recall Score
• Precision Score
• Object Segmentation
• Markov Random Field
• User Feedback
• Image Coordinates
• Maximum Step
• Related Constraints
• Ground Truth Segmentation
• Segmentation Framework
• Interactive Segmentation
• Disparity Information
• Belief Propagation
• Proper Direction
• Nearby Objects
• Abstract Objects
• Model Constraints