Colorizing Monochromatic Radiance Fields

Though Neural Radiance Fields (NeRF) can produce colorful 3D representations of the world by using a set of 2D images, such ability becomes non-existent when only monochromatic images are provided. Since color is necessary in representing the world, reproducing color from monochromatic radiance fields becomes crucial. To achieve this goal, instead of manipulating the monochromatic radiance fields directly, we consider it as a representation-prediction task in the Lab color space. By first constructing the luminance and density representation using monochromatic images, our prediction stage can recreate color representation on the basis of an image colorization module. We then reproduce a colorful implicit model through the representation of luminance, density, and color. Extensive experiments have been conducted to validate the effectiveness of our approaches. Our project page: https://liquidammonia.github.io/color-nerf.

Authors

• Yean Cheng National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University National Engineering Research Center of Visual Technology, School of Computer Science, Peking University AI Innovation Center, School of Computer Science, Peking University
• Renjie Wan Hong Kong Baptist University
• Shuchen Weng National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University National Engineering Research Center of Visual Technology, School of Computer Science, Peking University
• Chengxuan Zhu Peking University
• Yakun Chang National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University National Engineering Research Center of Visual Technology, School of Computer Science, Peking University
• Boxin Shi Peking University

Keywords

• CV: Computational Photography, Image & Video Synthesis
• CV: Low Level & Physics-based Vision