Unifying Multi-Step Methods through Matrix Splitting

We show that multi-step reinforcement learning methods can be analyzed through a new perspec- tive by using the notion of matrix splitting, a notion originally developed in the literature on linear iterative solvers. This new perspective allows us to better understand how seemingly different concepts, such as temporally extended actions in the options framework and TD(λ)-style bootstrapping, relate to each other. Mapping out existing algorithms on this spectrum also allows us to identify new variants and opens the door towards designing new algorithms.

Authors

• Pierre-Luc Bacon
• Doina Precup McGill University Mila

Keywords

No keywords are indexed for this paper.