Author name cluster

Vikranth R. Dwaracherla

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

AAAI Conference 2020 Conference Paper

Parameterized Indexed Value Function for Efficient Exploration in Reinforcement Learning

Tian Tan
Zhihan Xiong
Vikranth R. Dwaracherla

It is well known that quantifying uncertainty in the actionvalue estimates is crucial for efﬁcient exploration in reinforcement learning. Ensemble sampling offers a relatively computationally tractable way of doing this using randomized value functions. However, it still requires a huge amount of computational resources for complex problems. In this paper, we present an alternative, computationally efﬁcient way to induce exploration using index sampling. We use an indexed value function to represent uncertainty in our actionvalue estimates. We ﬁrst present an algorithm to learn parameterized indexed value function through a distributional version of temporal difference in a tabular setting and prove its regret bound. Then, in a computational point of view, we propose a dual-network architecture, Parameterized Indexed Networks (PINs), comprising one mean network and one uncertainty network to learn the indexed value function. Finally, we show the efﬁcacy of PINs through computational experiments.

PDF Details

JMLR Journal 2018 Journal Article

Gradient Estimation with Simultaneous Perturbation and Compressive Sensing

Vivek S. Borkar
Vikranth R. Dwaracherla
Neeraja Sahasrabudhe

We propose a scheme for finding a "good" estimator for the gradient of a function on a high-dimensional space with few function evaluations, for applications where function evaluations are expensive and the function under consideration is not sensitive in all coordinates locally, making its gradient almost sparse. Exploiting the latter aspect, our method combines ideas from Spall's Simultaneous Perturbation Stochastic Approximation with compressive sensing. We theoretically justify its computational advantages and illustrate them empirically by numerical experiments. In particular, applications to estimating gradient outer product matrix as well as standard optimization problems are illustrated via simulations. [abs] [ pdf ][ bib ] &copy JMLR 2018. ( edit, beta )

PDF Details