Active Exploration in Dynamic Environments

\Vhenever an agent learns to control an unknown environment, two oppos(cid: 173) ing principles have to be combined, namely: exploration (long-term opti(cid: 173) mization) and exploitation (short-term optimization). Many real-valued connectionist approaches to learning control realize exploration by ran(cid: 173) domness in action selection. This might be disadvantageous when costs are assigned to "negative experiences". The basic idea presented in this paper is to make an agent explore unknown regions in a more directed manner. This is achieved by a so-called competence map, which is trained to predict the controller's accuracy, and is used for guiding exploration. Based on this, a bistable system enables smoothly switching attention between two behaviors - exploration and exploitation - depending on ex(cid: 173) pected costs and knowledge gain. The appropriateness of this method is demonstrated by a simple robot navigation task.

Authors

• Sebastian Thrun Stanford University
• Knut Möller

Keywords

No keywords are indexed for this paper.