Deep Q-Learning for Dry Stacking Irregular Objects

We propose a reinforcement learning approach for automatically building dry stacked (i. e. no mortar) structures with irregular objects. Stacking irregular objects is a challenging problem since each assembly action can be drawn from a continuous space of poses for an object, and several local geometric and physical considerations strongly affect the stability. To tackle this challenge, we concentrate on a simplified 2D version of the problem. We present a reinforcement learning algorithm based on deep Q-learning, where the learned Q-function, which maps state-action pairs into expected long-term rewards, is represented by a deep neural network. As the action space is continuous the Q-network is trained by sampling a finite number of actions that consider both geometric and physical constraints to approximate the target Q-values, Experiments show that the proposed method outperforms previous heuristics-based planning, leading to super construction with objects containing a significant amount of variations. We validate the generated stacking plans by executing them using a robot arm and manufactured, irregular objects.

Authors

• Yifang Liu
• Seyed Mahdi Shamsi
• Le Fang 0002
• Changyou Chen
• Nils Napp

Keywords

• Stacking
• Buildings
• Robots
• Planning
• Shape
• Stability analysis
• Reinforcement learning
• Deep Q-learning
• Irregular Objects
• Dry Stacking
• Dry Objects
• Learning Algorithms
• Deep Neural Network
• Robotic Arm
• Geometric Constraints
• Version Of Problem
• State-action Pair
• Heuristic
• Deep Learning
• Structural Stability
• Gaussian Noise
• Learning Framework
• Building Materials
• Building Structures
• Reward Function
• Deep Reinforcement Learning
• Markov Decision Process
• Physical Simulation
• Object Displacement
• World Space
• Continuous Action Space
• Policy Learning
• Corner Stone
• Target Shape
• Planning Methods
• Executive Order
• Directed Acyclic Graph