A Compression-Inspired Framework for Macro Discovery

We consider the problem of how a reinforcement learning agent, tasked with solving a set of related Markov decision processes, can use knowledge acquired early on in its lifetime to improve its ability to more rapidly solve novel tasks. We propose a threestep framework that generates a diverse set of macros that lead to high rewards when solving a set of related tasks. Our experiments show that augmenting the original action-set of the agent with the identified macros allows it to more rapidly learn optimal policies in novel MDPs.

Authors

• Francisco M. Garcia University of Massachusetts, Amherst
• Bruno C. da Silva Eduardo W. Basso
• Philip S. Thomas University of Massachusetts

Keywords

• Reinforcement Learning
• Hierarchical RL
• Exploration