Contextual Markov Decision Processes using Generalized Linear Models

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes(CMDP), where the agent interacts with infinitely many tabular environments in a se- quence. In this paper, we propose a no regret online RL algorithm in the setting where the MDP parameters are obtained from the context using generalized linear models (GLM). The proposed algorithm GL-ORL is completely online and memory efficient and also improves over the known regret bounds in the linear case. In addition to an Online Newton Step based method, we also extend existing tools to show conversion from any online no-regret algorithm to confidence sets in the multinomial GLM case. A lower bound is also provided for the problem.

Authors

• Aditya Modi
• Ambuj Tewari

Keywords

No keywords are indexed for this paper.