Automating Curriculum Learning for Reinforcement Learning using a Skill-Based Bayesian Network

A major challenge for reinforcement learning is automatically generating curricula to reduce training time or improve performance in some target task. We introduce SEBNs (Skill-Environment Bayesian Networks) which model a probabilistic relationship between a set of skills, a set of goals that relate to the reward structure, and a set of environment features to predict policy performance on (possibly unseen) tasks. We develop an algorithm that uses the inferred estimates of agent success from an SEBN to weigh the possible next tasks by expected improvement. We evaluate the benefit of the resulting curriculum on three environments: a discrete gridworld, continuous control, and simulated robotics. The results show that SEBN-based curricula frequently outperform other baselines.

Authors

• Vincent Hsiao NRC Postdoctoral Fellow, Naval Research Laboratory
• Mark Roberts
• Laura M. Hiatt Naval Research Laboratory
• George Konidaris Brown University
• Dana S. Nau

Keywords

• Bayesian Networks
• Automated Curriculum Generation
• Reinforcement Learning
• Transfer Learning