Author name cluster

Johan Källström

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

AAMAS Conference 2024 Conference Paper

Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning

Peter Vamplew
Cameron Foale
Conor F. Hayes
Patrick Mannion
Enda Howley
Richard Dazeley
Scott Johnson
Johan Källström

Research in multi-objective reinforcement learning (MORL) has introduced the utility-based paradigm, which makes use of both environmental rewards and a function that defines the utility derived by the user from those rewards. In this paper we extend this paradigm to the context of single-objective reinforcement learning (RL), and outline multiple potential benefits including the ability to perform multi-policy learning across tasks relating to uncertain objectives, risk-aware RL, discounting, and safe RL. We also examine the algorithmic implications of adopting a utility-based approach.

PDF

AAMAS Conference 2023 Conference Paper

A Brief Guide to Multi-Objective Reinforcement Learning and Planning

Conor F. Hayes
Roxana Rădulescu
Eugenio Bargiacchi
Johan Källström
Matthew Macfarlane
Mathieu Reymond
Timothy Verstraeten
Luisa M. Zintgraf

Real-world sequential decision-making tasks are usually complex, and require trade-offs between multiple – often conflicting – objectives. However, the majority of research in reinforcement learning (RL) and decision-theoretic planning assumes a single objective, or that multiple objectives can be handled via a predefined weighted sum over the objectives. Such approaches may oversimplify the underlying problem, and produce suboptimal results. This extended abstract outlines the limitations of using a semi-blind iterative process to solve multi-objective decision making problems. Our extended paper [4], serves as a guide for the application of explicitly multi-objective methods to difficult problems.

PDF

AAMAS Conference 2023 Conference Paper

Model-Based Actor-Critic for Multi-Objective Reinforcement Learning with Dynamic Utility Functions

Johan Källström
Fredrik Heintz

Many real-world problems require a trade-off between multiple conflicting objectives. Decision-makers’ preferences over solutions to such problems are determined by their utility functions, which convert multi-objective values to scalars. In some settings, utility functions change over time, and the goal is to find methods that can efficiently adapt an agent’s policy to changes in utility. Previous work on learning with dynamic utility functions has focused on model-free methods, which often suffer from poor sample efficiency. In this work, we instead propose a model-based actor-critic, which explores with diverse utility functions through imagined rollouts within a learned world model between interactions with the real environment. An experimental evaluation on Minecart, a well-known benchmark for multi-objective reinforcement learning, shows that by learning a model of the environment the quality of the agent’s policy is improved compared to model-free algorithms.

PDF

AAMAS Conference 2023 Conference Paper

Scalar Reward is Not Enough

Peter Vamplew
Benjamin J. Smith
Johan Källström
Gabriel Ramos
Roxana Rădulescu
Diederik M. Roijers
Conor F. Hayes
Friedrik Hentz

Silver et al. [14] posit that scalar reward maximisation is sufficient to underpin all intelligence and provides a suitable basis for artificial general intelligence (AGI). This extended abstract summarises the counter-argument from our JAAMAS paper[19].

PDF