Instance-Aware Diffusion Implicit Process for Box-Based Instance Segmentation

The diffusion model has demonstrated impressive performance in image generation, but its potential for discriminative tasks such as instance segmentation remains unexplored. In this paper, we propose an Instance-aware Diffusion Implicit Process (IDIP) framework for instance segmentation based on boxes. During training, IDIP diffuses ground-truth boxes across various time steps, extracting corresponding Region of Interest (RoI) features. Dynamic convolution is then used to predict boxes and categories for each RoI, and the mask head generates masks from these predictions. During inference, IDIP iteratively refines randomly generated boxes with the denoising diffusion implicit model, while the mask head derives final masks from RoIs based on the refined boxes. Our method surpasses existing approaches on the COCO benchmark, requiring fewer training steps and less memory resources due to its dynamic design and instance-aware characteristic.

Authors

• Hao Ren 0002
• Xingsong Liu
• Junjian Huang
• Ru Wan
• Jian Pu
• Hong Lu 0001

Keywords

No keywords are indexed for this paper.