Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning

Segment Anything Model (SAM) has made great progress in anomaly segmentation tasks due to its impressive generalization ability. However, existing methods that directly apply SAM through prompting often overlook the domain shift issue, where SAM performs well on natural images but struggles in industrial scenarios. Parameter-Efficient Fine-Tuning (PEFT) offers a promising solution, but it may yield suboptimal performance by not adequately addressing the perception challenges during adaptation to anomaly images. In this paper, we propose a novel Self-Perception Tuning (SPT) method, aiming to enhance SAM's perception capability for anomaly segmentation. The SPT method incorporates a self-drafting tuning strategy, which generates an initial coarse draft of the anomaly mask, followed by a refinement process. Additionally, a visual-relation-aware adapter is introduced to improve the perception of discriminative relational information for mask generation. Extensive experimental results on several benchmark datasets demonstrate that our SPT method can significantly outperform baseline methods, validating its effectiveness.

Authors

• Hui-Yue Yang School of Software, Tsinghua University
• Hui Chen
• Ao Wang School of Software, Tsinghua University
• Kai Chen Shanghai Key Lab of Intell. Info. Processing, School of Computer Science, Fudan University, Shanghai Collaborative Innovation Center on Intelligent Visual Computing
• Zijia Lin School of Software, Tsinghua University
• Yongliang Tang LUSTER LightTech Co., Ltd.
• Pengcheng Gao LUSTER LightTech Co., Ltd.
• Yuming Quan LUSTER LightTech Co., Ltd.
• Jungong Han Department of Automation, Tsinghua University
• Guiguang Ding School of Software, Tsinghua University

Keywords

No keywords are indexed for this paper.