An Efficient Multi-Task Synergetic Network for Polyp Segmentation and Classification

Colonoscopy is considered the best diagnostic tool for early detection and resection of polyps, which can effectively prevent consequential colorectal cancer. In clinical practice, segmenting and classifying polyps from colonoscopic images have a great significance since they provide precious information for diagnosis and treatment. In this study, we propose an efficient multi-task synergetic network (EMTS-Net) for concurrent polyp segmentation and classification, and we introduce a polyp classification benchmark for exploring the potential correlations of the above-mentioned two tasks. This framework is composed of an enhanced multi-scale network (EMS-Net) for coarse-grained polyp segmentation, an EMTS-Net (Class) for accurate polyp classification, and an EMTS-Net (Seg) for fine-grained polyp segmentation. Specifically, we first obtain coarse segmentation masks by using EMS-Net. Then, we concatenate these rough masks with colonoscopic images to assist EMTS-Net (Class) in locating and classifying polyps precisely. To further enhance the segmentation performance of polyps, we propose a random multi-scale (RMS) training strategy to eliminate the interference caused by redundant information. In addition, we design an offline dynamic class activation mapping (OFLD CAM) generated by the combined effect of EMTS-Net (Class) and RMS strategy, which optimizes bottlenecks between multi-task networks efficiently and elegantly and helps EMTS-Net (Seg) to perform more accurate polyp segmentation. We evaluate the proposed EMTS-Net on the polyp segmentation and classification benchmarks, and it achieves an average mDice of 0. 864 in polyp segmentation and an average AUC of 0. 913 with an average accuracy of 0. 924 in polyp classification. Quantitative and qualitative evaluations on the polyp segmentation and classification benchmarks demonstrate that our EMTS-Net achieves the best performance and outperforms previous state-of-the-art methods in terms of both efficiency and generalization.

Authors

• Miao Wang School of Electronic Science and Engineering, Nanjing University
• Xingwei An
• Zhengcun Pei
• Ning Li Key Laboratory of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin 541004, China Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin 541004, China
• Li Zhang
• Gang Liu
• Dong Ming

Keywords

• Image segmentation
• Task analysis
• Training
• Deep learning
• Convolutional neural networks
• Shape
• Correlation
• Classification Network
• Network Efficiency
• Polyp Classification
• Polyp Segmentation
• Colorectal Cancer
• Training Strategy
• Colonoscopy
• Segmentation Performance
• Class Activation Maps
• Multi-scale Strategy
• Classification Benchmarks
• Polyp Detection
• Convolutional Neural Network
• Classification Performance
• Classification Task
• Feature Maps
• Learning Ability
• Segmentation Method
• Segmentation Model
• Colorectal Polyps
• Strong Generalization Ability
• Medical Image Segmentation
• Neural Architecture Search
• Segmentation Task
• Binary Cross-entropy Loss
• SOTA Methods
• Computer-aided Diagnosis
• Resolution Strategies
• Dice Loss
• multi-task synergetic network
• offline dynamic class activation mapping (OFLD CAM)
• Humans
• Benchmarking
• Image Processing, Computer-Assisted