Improved off-policy training of diffusion samplers

We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at this link as a base for future work on diffusion models for amortized inference.

Authors

• Marcin Sendera
• Minsu Kim Korea Advanced Institute of Science and Technology (KAIST)
• Sarthak Mittal
• Pablo Lemos
• Luca Scimeca
• Jarrid Rector-Brooks
• Alexandre Adam
• Yoshua Bengio Mila University of Montreal
• Nikolay Malkin

Keywords

No keywords are indexed for this paper.