Dynamic Depth-Aware Network for Endoscopy Super-Resolution

Endoscopy super-resolution (SR) plays an important role in improving diagnostic results and reducing the misdiagnosis rate. Even though recent studies have investigated the SR for endoscopy, these methods apply equal importance to the whole image and do not consider the relationship among pixels, especially the depth information, which can provide diagnosis-related information for clinicians. To address this problem, we propose a dynamic depth-aware network for endoscopy super-resolution, which represents the first effort to comprehensively integrate the depth information to the SR task for endoscopic images. It includes a depth-wise feature extracting branch (DW-B) and a depth-guided SR branch (DGSR-B). The DW-B aims to extract the representative feature for each depth level (i. e. depth matrix) further to provide auxiliary information and guide the super-resolution of texture under different depth levels. In DGSR-B, a depth-guided block (DGB) consisting of depth-focus normalization (DFN) is introduced to inject both the depth matrix and depth map into the LR image feature, so as to guide the image generation for each depth region. To adaptively super-resolve the regions under different depth levels, we devise a dynamic depth-aware loss to assign different trainable weights to each region for SR optimization. Extensive experiments have been conducted on two main publicly available datasets, i. e. , the Kvasir dataset and the EndoScene dataset, and the superior performance verifies the effectiveness of our method for SR task and polyp segmentation. Source code is to be released.

Authors

• Wenting Chen City University of Hong Kong
• Yifan Liu The Chinese University of Hong Kong
• Jiancong Hu
• Yixuan Yuan The Chinese University of Hong Kong

Keywords

• Feature extraction
• Endoscopes
• Cancer
• Superresolution
• Deep learning
• Task analysis
• Three-dimensional displays
• Dynamic Network
• Image Features
• Feature Representation
• Depth Map
• Depth Information
• Weight Training
• Depth Level
• Auxiliary Information
• Endoscopic Images
• Super-resolution Task
• Gastrointestinal Tract
• High-resolution Images
• Deep Neural Network
• Quantitative Results
• Scaling Factor
• Convolutional Layers
• Image Regions
• Normalization Layer
• Activation Maps
• Residual Block
• Peak Signal-to-noise Ratio
• Depth Estimation
• Super-resolution Network
• Super-resolution Performance
• Dynamic Weight
• Peak Signal-to-noise Ratio Values
• Polyp Size
• Current Frame
• Bicubic Interpolation
• Tract Organs
• Endoscopy
• super-resolution
• polyp segmentation
• Humans