Episodic Policy Gradient Training

We introduce a novel training procedure for policy gradient methods wherein episodic memory is used to optimize the hyperparameters of reinforcement learning algorithms onthe-fly. Unlike other hyperparameter searches, we formulate hyperparameter scheduling as a standard Markov Decision Process and use episodic memory to store the outcome of used hyperparameters and their training contexts. At any policy update step, the policy learner refers to the stored experiences, and adaptively reconfigures its learning algorithm with the new hyperparameters determined by the memory. This mechanism, dubbed as Episodic Policy Gradient Training (EPGT), enables an episodic learning process, and jointly learns the policy and the learning algorithm’s hyperparameters within a single run. Experimental results on both continuous and discrete environments demonstrate the advantage of using the proposed method in boosting the performance of various policy gradient algorithms.

Authors

• Hung Le Deakin University
• Majid Abdolshah
• Thommen K. George
• Kien Do Applied Artificial Intelligence Institute, Deakin University
• Dung Nguyen Deakin University
• Svetha Venkatesh Deakin University

Keywords

No keywords are indexed for this paper.