Daishi Harada

ICML Conference 1999 Conference Paper

Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping

• Andrew Y. Ng
• Daishi Harada
• Stuart Russell 0001

AAAI Conference 1999 Short Paper

Towards Bounded Optimal Meta-Level Control: A Case Study

• Daishi Harada
• University of California
• Berkeley

Suppose we allow the controller to perform arbitrary search, and to base its control on the backed up information. To do this, we need to make decisions about the following: the order in which search nodes are expanded, and when to stop searching and actually "commit" to a control. The approach that we take is to view these decisions as the meta-level control problem. With some care in the formulation, it can be seen that a solution to this meta-level control problem will provide us with a bounded optimal controller. We would like to solve this problem by using algorithms from reinforcement learning.

AAAI Conference 1997 Conference Paper

Reinforcement Learning with Time

• Daishi Harada

This paper steps back from the standard infinite horizon formulation of reinforcement learning problems to consider the simpler case of finite horizon problems. Although finite horizon problems may be solved using infinite horizon learning algorithms by recasting the problem as an infinite horizon problem over a state space extended to include time, we show that such an application of infinite horizon learning algorithms does not make use of what is known about the environment structure, and is therefore inefficient. Preserving a notion of time within the environment allows us to consider extending the environment model to include, for example, random action duration. Such extentions allow us to model non-Markov environments which can be learned using reinforcement learning algorithms.