Towards Illumination-Aware Restoration of Metalens-Captured Images: A New Dataset and a Strong Baseline

Metalenses offer compelling advantages such as lightweight and ultra-thin design, making them promising alternatives to conventional lenses. However, their widespread adoption is hindered by image quality degradation caused by chromatic and angular aberrations. To mitigate this, restoration processes are often necessary to recover high-quality RGB images from metalens-captured inputs. While recent deep learning-based restoration methods show promise, they typically (1) blur or distort peripheral regions, or (2) fail entirely under unseen illumination conditions. To advance metalens image restoration, we introduce IlluMeta---the first and largest real-world, illumination-aware metalens image dataset—captured across diverse lighting environments. In addition, we propose a novel end-to-end restoration framework that directs attention to challenging regions and adaptively adjusts to varying illuminations via reinforcement learning. Experiments show that our method can be applied in a plug-and-play manner to enhance existing models, significantly improving image restoration quality, especially under unseen lighting conditions, paving the way for broader real-world deployment of metalens technologies.

Authors

• Fen Fang
• Xinan Liang Institute of Materials Research and Engineering (IMRE), A*STAR, Singapore National Semiconductor Translation and Innovation Center (NSTIC), Singapore
• Muli Yang
• Jinghong Zheng
• Tobias Mass MetaOptics, Singapore
• Ying Sun Pennsylvania State University
• Xulei Yang Institute for Infocomm Research (I2R), A*STAR
• Xuewu Xu Institute of Materials Research and Engineering (IMRE), A*STAR, Singapore National Semiconductor Translation and Innovation Center (NSTIC), Singapore
• Zhengguo Li Institute for Infocomm Research (I2R), A*STAR, Singapore

Keywords

No keywords are indexed for this paper.