Learning to Plan with Tree Search via Deep RL

Tree search is an important component of many decision-making algorithms but often relies on an evaluation function that estimates the desirability of each node. In this paper, we propose to learn which nodes to expand based on a variety of object-level features. We introduce a reward function for this problem based on value of computation estimates with respect to improving the policy for the underlying problem. We apply deep reinforcement learning to this problem in an approach we call Reinforcement Learning for Tree Search (RLTS) and demonstrate that it can yield better performance than baselines in a procedurally generated environment.

Authors

• Dylan Cope
• Justin Svegliato
• Stuart Russell Center for Human-Compatible AI, UC Berkeley

Keywords

• Deep Reinforcement Learning
• meta-level control
• metareasoning
• planning
• tree search