Segment Any Repeated Object

Understanding a scene in terms of objects and their properties is fundamental for various vision-based robotic applications, including item picking. To effectively clear a bin, a robot must comprehend objects as graspable entities, often without prior access to models of the target object. This study focuses on open world object segmentation with the additional requirement of assigning identical class labels for repeated instances of the same object. This capability enables item picking tasks with homogeneous bins, filtering out packaging material, and sorting tasks. We propose a novel pipeline for detecting repeated instances of identical objects, building on recent advancements in vision foundation models and exploring approaches for estimating object similarities based on feature embeddings or keypoint correspondence matching. Through a comprehensive experimental evaluation, we establish a new state-of-the-art on ARMBench repeated objects segmentation, a particularly challenging open problem in bin-picking robotics. Additionally, we demonstrate the real-world application of our method integrated into a robot picking cell to showcase its relevance to industrial use cases.

Authors

• Yushi Liu
• Christian Graf
• Markus Spies
• Margret Keuper

Keywords

• Foundation models
• Service robots
• Pipelines
• Refining
• Object segmentation
• Object detection
• Packaging
• Proposals
• Object recognition
• Sorting
• Packaging Materials
• Target Object
• Identical Objects
• Object Instances
• Foundation Model
• Open World
• Convolutional Neural Network
• Image Features
• Similarity Measure
• Local Features
• Intersection Over Union
• Target Image
• Object Features
• RGB Images
• Single Object
• Average Precision
• Maximum A Posteriori
• Objects In The Scene
• Object Proposals
• COCO Dataset
• Object Counting
• Query Image
• Specific Use Case
• Proposal Generation
• RGB-D Images
• Bounding Box
• Separate Objects
• Object Detection Methods