Reschedule Diffusion-based Bokeh Rendering

Bokeh rendering for images shot with small apertures has drawn much attention in practice. Very recently people start to explore diffusion models for bokeh rendering, aiming to leverage the models' surging power of image generation. However, we can clearly observe two big issues with the images rendered by diffusion models: large fluctuation and severe color deviation. To address these issues, we propose in this paper a prior-aware sampling approach, which can adaptively control the noise scale through learned priors, and a prior-aware noise scheduling strategy, which can greatly reduce the number of inference steps without sacrificing performance. Extensive experiments show that our method can effectively alleviate the fluctuation problem of sampling results while ensuring similar color styles to the input image. In addition, our method outperforms state-of-the-art methods, sometimes even with only two steps of sampling. Our code is available at https: //github. com/Loeiii/Reschedule-Diffusion-based-Bokeh-Rendering.

Authors

• Shiyue Yan
• Xiaoshi Qiu
• Qingmin Liao Tsinghua University
• Jing-Hao Xue University College London, UK
• Shaojun Liu

Keywords

• Computer Vision: CV: Image and video synthesis and generation
• Humans and AI: HAI: Applications
• Machine Learning: ML: Applications