Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning

Research in multi-objective reinforcement learning (MORL) has introduced the utility-based paradigm, which makes use of both environmental rewards and a function that defines the utility derived by the user from those rewards. In this paper we extend this paradigm to the context of single-objective reinforcement learning (RL), and outline multiple potential benefits including the ability to perform multi-policy learning across tasks relating to uncertain objectives, risk-aware RL, discounting, and safe RL. We also examine the algorithmic implications of adopting a utility-based approach.

Authors

• Peter Vamplew Federation University
• Cameron Foale
• Conor F. Hayes Cognizant AI Lab
• Patrick Mannion University of Galway
• Enda Howley
• Richard Dazeley Deakin University
• Scott Johnson
• Johan Källström Linköping University
• Gabriel Ramos Universidade do Vale do Rio dos Sinos
• Roxana Rădulescu Utrecht University
• Willem Röpke
• Diederik M. Roijers

Keywords

• reinforcement learning
• utility