Text Diffusion with Reinforced Conditioning

Diffusion models have demonstrated exceptional capability in generating high-quality images, videos, and audio. Due to their adaptiveness in iterative refinement, they provide a strong potential for achieving better non-autoregressive sequence generation. However, existing text diffusion models still fall short in their performance due to a challenge in handling the discreteness of language. This paper thoroughly analyzes text diffusion models and uncovers two significant limitations: degradation of self-conditioning during training and misalignment between training and sampling. Motivated by our findings, we propose a novel Text Diffusion model called TReC, which mitigates the degradation with Reinforced Conditioning and the misalignment by Time-Aware Variance Scaling. Our extensive experiments demonstrate the competitiveness of TReC against autoregressive, non-autoregressive, and diffusion baselines. Moreover, qualitative analysis shows its advanced ability to fully utilize the diffusion process in refining samples.

Authors

• Yuxuan Liu Gaoling School of Artificial Intelligence, Renmin University of China
• Tianchi Yang Microsoft Corporation
• Shaohan Huang Microsoft Corporation
• Zihan Zhang Microsoft Corporation
• Haizhen Huang Microsoft Corporation
• Furu Wei Microsoft
• Weiwei Deng Microsoft Corporation
• Feng Sun Microsoft Corporation
• Qi Zhang School of Computer Science, Fudan University, Shanghai, China Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China

Keywords

• ML: Deep Generative Models & Autoencoders
• ML: Reinforcement Learning
• NLP: (Large) Language Models
• NLP: Generation