Optimizing Local Satisfaction of Long-Run Average Objectives in Markov Decision Processes

Long-run average optimization problems for Markov decision processes (MDPs) require constructing policies with optimal steady-state behavior, i.e., optimal limit frequency of visits to the states. However, such policies may suffer from local instability in the sense that the frequency of states visited in a bounded time horizon along a run differs significantly from the limit frequency. In this work, we propose an efficient algorithmic solution to this problem.

Authors

• David Klaška Masaryk University
• Antonín Kučera
• Vojtěch Kůr Masaryk University
• Vít Musil Masaryk University
• Vojtěch Řehák

Keywords

• MAS: Multiagent Planning
• PRS: Planning with Markov Models (MDPs, POMDPs)
• ROB: Motion and Path Planning