Author name cluster

Akshay Narayan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

PRL Workshop 2020 Workshop Paper

Hierarchical Reinforcement Learning in StarCraft II with Human Expertise in Subgoals Selection

Xinyi Xu
Tiancheng Huang
Pengfei Wei
Akshay Narayan
Tze-Yun Leong

This work is inspired by recent advances in hierarchical reinforcement learning (HRL) (Barto and Mahadevan 2003; Hengst 2010), and improvements in learning efficiency from heuristic-based subgoal selection, experience replay (Lin 1993; Andrychowicz et al. 2017), and task-based curriculum learning (Bengio et al. 2009; Zaremba and Sutskever 2014). We propose a new method to integrate HRL, experience replay and effective subgoal selection through an implicit curriculum design based on human expertise to support sample-efficient learning and enhance interpretability of the agent’s behavior. Human expertise remains indispensable in many areas such as medicine (Buch, Ahmed, and Maruthappu 2018) and law (Cath 2018), where interpretability, explainability and transparency are crucial in the decision making process, for ethical and legal reasons. Our method simplifies the complex task sets for achieving the overall objectives by decomposing them into subgoals at different levels of abstraction. Incorporating relevant subjective knowledge also significantly reduces the computational resources spent in exploration for RL, especially in high speed, changing, and complex environments where the transition dynamics cannot be effectively learned and modelled in a short time. Experimental results in two StarCraft II (SC2) (Vinyals et al. 2017) minigames demonstrate that our method can achieve better sample efficiency than flat and end-to-end RL methods, and provides an effective method for explaining the agent’s performance.

PDF Details

AAMAS Conference 2019 Conference Paper

Effects of Task Similarity on Policy Transfer with Selective Exploration in Reinforcement Learning

Akshay Narayan
Tze Yun Leong

The SEAPoT algorithm [9] is a knowledge transfer mechanism in model-based reinforcement learning. By constructing subspaces around the changed regions, and selectively and efficiently exploring the target task, the transfer is most effective when the source and target tasks share similar objectives but differ in the transition dynamics. In this work, we identify the similarity between tasks using a new light-weight metric, based on the Jensen-Shannon distance, and show how the degree of similarity affects the transfer efficacy. We also empirically show that SEAPoT performs better in terms of jump starts and average rewards, as compared to the state-of-the-art policy reuse methods.

PDF

NeurIPS Conference 2019 Conference Paper

Park: An Open Platform for Learning-Augmented Computer Systems

Hongzi Mao
Parimarjan Negi
Akshay Narayan
Hanrui Wang
Jiacheng Yang
Haonan Wang
Ryan Marcus
ravichandra addanki

We present Park, a platform for researchers to experiment with Reinforcement Learning (RL) for computer systems. Using RL for improving the performance of systems has a lot of potential, but is also in many ways very different from, for example, using RL for games. Thus, in this work we first discuss the unique challenges RL for systems has, and then propose Park an open extensible platform, which makes it easier for ML researchers to work on systems problems. Currently, Park consists of 12 real world system-centric optimization problems with one common easy to use interface. Finally, we present the performance of existing RL approaches over those 12 problems and outline potential areas of future work.

PDF Details

AAAI Conference 2017 Conference Paper

An Efficient Approach to Model-Based Hierarchical Reinforcement Learning

Zhuoru Li
Akshay Narayan
Tze-Yun Leong

We propose a model-based approach to hierarchical reinforcement learning that exploits shared knowledge and selective execution at different levels of abstraction, to efﬁciently solve large, complex problems. Our framework adopts a new transition dynamics learning algorithm that identiﬁes the common action-feature combinations of the subtasks, and evaluates the subtask execution choices through simulation. The framework is sample efﬁcient, and tolerates uncertain and incomplete problem characterization of the subtasks. We test the framework on common benchmark problems and complex simulated robotic environments. It compares favorably against the stateof-the-art algorithms, and scales well in very large problems.

PDF Details

AAAI Conference 2017 Short Paper

SEAPoT-RL: Selective Exploration Algorithm for Policy Transfer in RL

Akshay Narayan
Zhuoru Li
Tze-Yun Leong

We propose a new method for transferring a policy from a source task to a target task in model-based reinforcement learning. Our work is motivated by scenarios where a robotic agent operates in similar but challenging environments, such as hospital wards, differentiated by structural arrangements or obstacles, such as furniture. We address problems that require fast responses adapted from incomplete, prior knowledge of the agent in new scenarios. We present an efﬁcient selective exploration strategy that maximally reuses the source task policy. Reuse efﬁciency is effected through identifying subspaces that are different in the target environment, thus limiting the exploration needed in the target task. We empirically show that SEAPoT performs better in terms of jump starts and cumulative average rewards, as compared to existing state-ofthe-art policy reuse methods.

PDF Details

AAMAS Conference 2016 Conference Paper

A Core Task Abstraction Approach to Hierarchical Reinforcement Learning (Extended Abstract)

Zhuoru Li
Akshay Narayan
Tze-Yun Leong

We propose a new, core task abstraction (CTA) approach to learning the relevant transition functions in model-based hierarchical reinforcement learning. CTA exploits contextual independences of the state variables conditional on the taskspecific actions; its promising performance is demonstrated through a set of benchmark problems.

PDF