Enhancing Context-Based Meta-Reinforcement Learning Algorithms via An Efficient Task Encoder (Student Abstract)

Meta-Reinforcement Learning (meta-RL) algorithms enable agents to adapt to new tasks from small amounts of exploration, based on the experience of similar tasks. Recent studies have pointed out that a good representation of a task is key to the success of off-policy context-based meta-RL. Inspired by contrastive methods in unsupervised representation learning, we propose a new method to learn the task representation based on the mutual information between transition tuples in a trajectory and the task embedding. We also propose a new estimation for task similarity based on Q-function, which can be used to form a constraint on the distribution of the encoded task variables, making the task encoder encode the task variables more effective on new tasks. Experiments on meta-RL tasks show that the newly proposed method outperforms existing meta-RL algorithms.

Authors

• Feng Xu Nanjing University
• Shengyi Jiang
• Hao Yin Nanjing University
• Zongzhang Zhang National Key Laboratory for Novel Software Technology, Nanjing University, China School of Artificial Intelligence, Nanjing University, China
• Yang Yu School of Artificial Intelligence, Jilin University, Changchun Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University, Changchun
• Ming Li
• Dong Li University of Science and Technology of China
• Wulong Liu

Keywords

No keywords are indexed for this paper.