ShortListing Model: A Streamlined Simplex Diffusion for Discrete Variable Generation

Generative modeling of discrete variables is challenging yet crucial for applications in natural language processing and biological sequence design. We introduce the Shortlisting Model (SLM), a novel simplex-based diffusion model inspired by progressive candidate pruning. SLM operates on simplex centroids, reducing generation complexity and enhancing scalability. Additionally, SLM incorporates a flexible implementation of classifier-free guidance, enhancing unconditional generation performance. Extensive experiments on DNA promoter and enhancer design, protein design, character-level and large-vocabulary language modeling demonstrate the competitive performance and strong potential of SLM. Our code can be found at https: //github. com/GenSI-THUAIR/SLM.

Authors

• Yuxuan Song The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences The School of Artificial Intelligence, University of Chinese Academy of Sciences
• Zhe Zhang
• Yu Pei
• Jingjing Gong
• Qiying Yu Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology
• Zheng Zhang
• Mingxuan Wang Bytedance Seed, China
• Hao Zhou Naval University of Engineering
• Jingjing Liu
• Wei-Ying Ma

Keywords

No keywords are indexed for this paper.