STACKGEN: Generating Stable Structures from Silhouettes via Diffusion

Humans naturally obtain intuition about the interactions between and the stability of rigid objects by observing and interacting with the world. It is this intuition that governs the way in which we regularly configure objects in our environment, allowing us to build complex structures from simple, everyday objects. Robotic agents, on the other hand, traditionally require an explicit model of the world that includes the detailed geometry of each object and an analytical model of the environment dynamics, which are difficult to scale and preclude generalization. Instead, robots would benefit from an awareness of intuitive physics that enables them to similarly reason over the stable interaction of objects in their environment. Towards that goal, we propose STACKGEN—a diffusion model that generates diverse stable configurations of building blocks matching a target silhouette. To demonstrate the capability of the method, we evaluate it in a simulated environment and deploy it in the real setting using a robotic arm to assemble structures generated by the model. Our code is available at https://ripl.github.io/StackGen.

Authors

• Luzhe Sun
• Takuma Yoneda
• Samuel Wheeler
• Tianchong Jiang
• Matthew R. Walter

Keywords

• Hands
• Geometry
• Analytical models
• Visualization
• Three-dimensional displays
• Reliability engineering
• Manipulators
• Stability analysis
• Physics
• Intelligent robots
• Structural Stability
• Simulation Environment
• Diffusion Model
• Electronic Configuration
• Robotic Arm
• Training Set
• Convolutional Neural Network
• Bimodal
• Probabilistic Model
• Diffusion Process
• Intersection Over Union
• Reversible Process
• Learning-based Methods
• Binary Image
• Median Filter
• Inference Time
• Real-world Experiments
• Encoder Layer
• Forward Process
• Pre-trained Convolutional Neural Network
• Block Stacking
• Hidden Dimension
• Transformer Architecture
• Physics Engine
• Object Pose