Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption

The standard Neural Radiance Fields (NeRF) paradigm employs a viewer-centered methodology, entangling the aspects of illumination and material reflectance into emission solely from 3D points. This simplified rendering approach presents challenges in accurately modeling images captured under adverse lighting conditions, such as low light or over-exposure. Motivated by the ancient Greek emission theory that posits visual perception as a result of rays emanating from the eyes, we slightly refine the conventional NeRF framework to train NeRF under challenging light conditions and generate normal-light condition novel views unsupervisedly. We introduce the concept of a ``Concealing Field," which assigns transmittance values to the surrounding air to account for illumination effects. In dark scenarios, we assume that object emissions maintain a standard lighting level but are attenuated as they traverse the air during the rendering process. Concealing Field thus compel NeRF to learn reasonable density and colour estimations for objects even in dimly lit situations. Similarly, the Concealing Field can mitigate over-exposed emissions during rendering stage. Furthermore, we present a comprehensive multi-view dataset captured under challenging illumination conditions for evaluation. Our code and proposed dataset are available at https://github.com/cuiziteng/Aleth-NeRF.

Authors

• Ziteng Cui The University of Tokyo Shanghai AI Laboratory
• Lin Gu RIKEN AIP, Tokyo103-0027, Japan The University of Tokyo, Japan
• Xiao Sun Hefei University of Technology
• Xianzheng Ma University of Oxford
• Yu Qiao Shanghai Artificial Intelligence Laboratory
• Tatsuya Harada The University of Tokyo RIKEN AIP

Keywords

• CV: 3D Computer Vision
• CV: Computational Photography, Image & Video Synthesis
• CV: Low Level & Physics-based Vision
• ML: Deep Learning Algorithms
• ML: Deep Learning Theory
• ML: Semi-Supervised Learning
• ML: Transfer, Domain Adaptation, Multi-Task Learning
• ML: Transparent, Interpretable, Explainable ML
• ML: Unsupervised & Self-Supervised Learning