Gradient-Based Methods For Option Learning in Inverse Reinforcement Learning

In the pursuit of increasingly intelligent systems, abstraction plays a vital role in enabling sophis- ticated decisions to be made in complex environments. While good methods for learning useful abstraction exist in perceptual domains, the search for useful abstraction in control is ongoing. Here, we present two algorithms that apply gradient-descent methods and importance sampling in order to learn useful abstrac- tions of control, as well as a reward function, in the Inverse Reinforcement Learning setting. These methods can be used to learn abstract structure and provide potentially interpretable insight into human or animal behaviour, as well as formulate ”forwards” RL problems in a safe manner, when it is difficult to express a reward function directly. We also provide initial experimental results for one of these algorithms.

Authors

• Matthew Smith
• Pierre-Luc Bacon
• Joelle Pineau

Keywords

No keywords are indexed for this paper.