SceneX: Procedural Controllable Large-Scale Scene Generation

Developing comprehensive explicit world models is crucial for understanding and simulating real-world scenarios. Recently, Procedural Controllable Generation (PCG) has gained significant attention in large-scale scene generation by enabling the creation of scalable, high-quality assets. However, PCG faces challenges such as limited modular diversity, high expertise requirements, and challenges in managing the diverse elements and structures in complex scenes. In this paper, we introduce a large-scale scene generation framework, SceneX, which can automatically produce high-quality procedural models according to designers' textual descriptions. Specifically, the proposed method comprises two components, PCGHub and PCGPlanner. The former encompasses an extensive collection of accessible procedural assets and thousands of hand-craft API documents to perform as a standard protocol for PCG controller. The latter aims to generate executable actions for Blender to produce controllable and precise 3D assets guided by the user's instructions. Extensive experiments demonstrated the capability of our method in controllable large-scale scene generation, including nature scenes and unbounded cities, as well as scene editing such as asset placement and season translation.

Authors

• Mengqi Zhou University of Chinese Academy of Sciences Institute of Automation, Chinese Academy of Sciences State Key Laboratory of Multimodal Artificial Intelligence Systems New Laboratory of Pattern Recognition
• Yuxi Wang Centre for Artificial Intelligence and Robotics
• Jun Hou Centre for Artificial Intelligence and Robotics
• Shougao Zhang China University of Geoscience Beijing
• Yiwei Li University of Chinese Academy of Sciences
• Chuanchen Luo Shandong University
• Junran Peng University of Science and Technology Beijing
• Zhaoxiang Zhang Center for Artificial Intelligence and Robotics, HKISI-CAS Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences

Keywords

No keywords are indexed for this paper.