Robust Active Measuring under Model Uncertainty

Partial observability and uncertainty are common problems in sequential decision-making that particularly impede the use of formal models such as Markov decision processes (MDPs). However, in practice, agents may be able to employ costly sensors to measure their environment and resolve partial observability by gathering information. Moreover, imprecise transition functions can capture model uncertainty. We combine these concepts and extend MDPs to robust active-measuring MDPs (RAM-MDPs). We present an active-measure heuristic to solve RAM-MDPs efficiently and show that model uncertainty can, counterintuitively, let agents take fewer measurements. We propose a method to counteract this behavior while only incurring a bounded additional cost. We empirically compare our methods to several baselines and show their superior scalability and performance.

Authors

• Merlijn Krale Radboud University, Nijmegen
• Thiago D. Simão Eindhoven University of Technology, The Netherlands
• Jana Tumova KTH Royal Institute of Technology, Stockholm
• Nils Jansen Radboud University Nijmegen, The Netherlands Ruhr-University Bochum, Germany

Keywords

• General