A Novel Framework for Uncertainty-Driven Adaptive Exploration

Adaptive exploration methods learn complex policies via alternating between exploration and exploitation. An important question for such methods is to determine the appropriate moment to switch between exploration and exploitation and vice versa. This is critical in domains that require the learning of long and complex sequences of actions. In this work, we present a generic adaptive exploration frameworkthatemploysuncertaintytoaddressthisimportantissue in a principled manner. Our framework includes previous adaptive exploration approaches as special cases. Moreover, it can incorporate any uncertainty-measuring mechanism of choice, such as mechanisms used in intrinsic motivation, or epistemic uncertaintybased exploration methods; and is experimentally shown to give rise to adaptive exploration strategies that outperform standard ones across several environments. Moreover, we showcase its potential for utilization in safety-critical domains. The code for this work can be found at https: //github. com/leoBakop/ADEU

Authors

• Leonidas Bakopoulos Technical University of Crete
• Georgios Chalkiadakis Technical University of Crete

Keywords

• Deep Reinforcement Learning
• Adaptive Exploration
• Uncertainty- Driven Exploration