Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models

Denoising Diffusion models have exhibited remarkable capabilities in image generation. However, generating high-quality samples requires a large number of iterations. Knowledge distillation for diffusion models is an effective method to address this limitation with a shortened sampling process but causes degraded generative quality. Based on our analysis with bias-variance decomposition and experimental observations, we attribute the degradation to the spatial fitting error occurring in the training of both the teacher and student model in the distillation. Accordingly, we propose Spatial Fitting-Error Reduction Distillation model (SFERD). SFERD utilizes attention guidance from the teacher model and a designed semantic gradient predictor to reduce the student's fitting error. Empirically, our proposed model facilitates high-quality sample generation in a few function evaluations. We achieve an FID of 5.31 on CIFAR-10 and 9.39 on ImageNet 64x64 with only one step, outperforming existing diffusion methods. Our study provides a new perspective on diffusion distillation by highlighting the intrinsic denoising ability of models.

Authors

• Shengzhe Zhou Zhejiang University
• Zejian Li Zhejiang University
• Shengyuan Zhang Zhejiang University
• Lefan Hou ZheJiang University
• Changyuan Yang Alibaba Group
• Guang Yang
• Zhiyuan Yang Alibaba Group
• Lingyun Sun Zhejiang University

Keywords

• CV: Computational Photography, Image & Video Synthesis
• ML: Deep Generative Models & Autoencoders
• ML: Representation Learning