Transformer-Based Selective Super-resolution for Efficient Image Refinement

Conventional super-resolution methods suffer from two drawbacks: substantial computational cost in upscaling an entire large image, and the introduction of extraneous or potentially detrimental information for downstream computer vision tasks during the refinement of the background. To solve these issues, we propose a novel transformer-based algorithm, Selective Super-Resolution (SSR), which partitions images into non-overlapping tiles, selects tiles of interest at various scales with a pyramid architecture, and exclusively reconstructs these selected tiles with deep features. Experimental results on three datasets demonstrate the efficiency and robust performance of our approach for super-resolution. Compared to the state-of-the-art methods, the FID score is reduced from 26.78 to 10.41 with 40% reduction in computation cost for the BDD100K dataset.

Authors

• Tianyi Zhang Department of Computing, the Hong Kong Polytechnic University, Hong Kong
• Kishore Kasichainula Arizona State University
• Yaoxin Zhuo Arizona State University
• Baoxin Li Arizona State University
• Jae-Sun Seo Cornell Tech
• Yu Cao Singapore Management University

Keywords

• CV: Computational Photography, Image & Video Synthesis
• CV: Other Foundations of Computer Vision
• CV: Representation Learning for Vision
• ML: Deep Generative Models & Autoencoders