Structured agents for physical construction

Physical construction—the ability to compose objects, subject to physical dynamics, to serve some function—is fundamental to human intelligence. We introduce a suite of challenging physical construction tasks inspired by how children play with blocks, such as matching a target configuration, stacking blocks to connect objects together, and creating shelter-like structures over target objects. We examine how a range of deep reinforcement learning agents fare on these challenges, and introduce several new approaches which provide superior performance. Our results show that agents which use structured representations (e. g. , objects and scene graphs) and structured policies (e. g. , object-centric actions) outperform those which use less structured representations, and generalize better beyond their training when asked to reason about larger scenes. Model-based agents which use Monte-Carlo Tree Search also outperform strictly model-free agents in our most challenging construction problems. We conclude that approaches which combine structured representations and reasoning with powerful learning are a key path toward agents that possess rich intuitive physics, scene understanding, and planning.

Authors

• Victor Bapst
• Alvaro Sanchez-Gonzalez
• Carl Doersch
• Kimberly L. Stachenfeld
• Pushmeet Kohli
• Peter W. Battaglia
• Jessica B. Hamrick

Keywords

No keywords are indexed for this paper.