Parameterized Exploration

We introduce Parameterized Exploration (PE), a simple family of methods for model-based tun- ing of the exploration schedule in sequential decision problems. Unlike common heuristics for exploration, our method accounts for the time horizon of the decision problem as well as the agent’s current state of knowledge of the dynamics of the decision problem. We show our method as applied to several common exploration techniques has superior performance relative to un-tuned counterparts in Gaussian multi-armed bandits, as well as a Markov decision process based on a mobile health (mHealth) study. We also examine the effects of model accuracy on the performance of PE.

Authors

• Jesse Clifton Center on Long-Term Risk
• Lili Wu
• Eric Laber

Keywords

No keywords are indexed for this paper.