A Theory of Non-acyclic Generative Flow Networks

GFlowNets is a novel flow-based method for learning a stochastic policy to generate objects via a sequence of actions and with probability proportional to a given positive reward. We contribute to relaxing hypotheses limiting the application range of GFlowNets, in particular: acyclicity (or lack thereof). To this end, we extend the theory of GFlowNets on measurable spaces which includes continuous state spaces without cycle restrictions, and provide a generalization of cycles in this generalized context. We show that losses used so far push flows to get stuck into cycles and we define a family of losses solving this issue. Experiments on graphs and continuous tasks validate those principles.

Authors

• Leo Brunswic Huawei Shanghai Shanghai Research Center Shanghai Jiaotong University
• Yinchuan Li Huawei Noah’s Ark Lab, Beijing, China
• Yushun Xu Huawei Shanghai Shanghai Research Center
• Yijun Feng Huawei Shanghai Shanghai Research Center
• Shangling Jui Huawei Shanghai Shanghai Research Center
• Lizhuang Ma Shanghai Jiao Tong University, Shanghai, China East China Normal University, Shanghai, China MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China

Keywords

• ML: Deep Generative Models & Autoencoders
• ML: Deep Learning Theory
• ML: Graph-based Machine Learning
• ML: Multimodal Learning
• ML: Online Learning & Bandits
• ML: Optimization
• ML: Reinforcement Learning
• SO: Heuristic Search
• SO: Mixed Discrete/Continuous Search
• SO: Non-convex Optimization
• SO: Other Foundations of Search & Optimization