Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head Generation

Diffusion-based talking head models generate high-quality, photorealistic videos but suffer from slow inference, limiting practical applications. Existing acceleration methods for gen- eral diffusion models fail to exploit the temporal and spatial redundancies unique to talking head generation. In this paper, we propose a task-specific framework addressing these inefficiencies through two key innovations. First, we introduce Lightning-fast Caching-based Parallel denoising prediction (LightningCP), caching static features to bypass most model layers in inference time. We also enable parallel prediction using cached features and estimated noisy latents as inputs, efficiently bypassing sequential sampling. Second, we propose Decoupled Foreground Attention (DFA) to further accelerate attention computations, exploiting the spatial decoupling in talking head videos to restrict attention to dynamic foreground regions. Additionally, we remove reference features in certain layers to bring extra speedup. Extensive experiments demonstrate that our framework significantly improves inference speed while preserving video quality.

Authors

• Jianzhi Long The University of Sydney
• Wenhao Sun School of Statistics and Mathematics, Zhejiang Gongshang University, Hangzhou, China
• Rong-Cheng Tu Beijing Institute of Technology
• Dacheng Tao College of Computing and Data Science, Nanyang Technological University

Keywords

No keywords are indexed for this paper.