Object Exchangability in Reinforcement Learning

Although deep reinforcement learning has advanced significantly over the past several years, sample efficiency remains a major challenge. Careful choice of input representations can help improve efficiency depending on the structure present in the problem. In this work, we present an attention-based method to project inputs into an efficient representation space that is invariant under changes to input ordering. We show that our proposed representation results in a search space that is a factor ofm! smaller for inputs ofm objects. Our experiments demonstrate improvements in sample efficiency for policy gradient methods on a variety of tasks. We show that our representation allows us to solve problems that are otherwise intractable when using naïve approaches.

Authors

• John Mern
• Dorsa Sadigh
• Mykel Kochenderfer

Keywords

• Knowledge Representation
• Reasoning
• Reinforcement Learning