Pravesh Ranchod Papers

AAAI Conference 2016 Conference Paper

Reinforcement Learning with Parameterized Actions

Warwick Masson
Pravesh Ranchod
George Konidaris

We introduce a model-free algorithm for learning in Markov decision processes with parameterized actions—discrete actions with continuous parameters. At each step the agent must select both which action to use and which parameters to use with that action. We introduce the Q-PAMDP algorithm for learning in these domains, show that it converges to a local optimum, and compare it to direct policy search in the goalscoring and Platform domains.

PDF Details

IROS Conference 2015 Conference Paper

Nonparametric Bayesian reward segmentation for skill discovery using inverse reinforcement learning

Pravesh Ranchod
Benjamin Rosman
George Konidaris 0001

We present a method for segmenting a set of unstructured demonstration trajectories to discover reusable skills using inverse reinforcement learning (IRL). Each skill is characterised by a latent reward function which the demonstrator is assumed to be optimizing. The skill boundaries and the number of skills making up each demonstration are unknown. We use a Bayesian nonparametric approach to propose skill segmentations and maximum entropy inverse reinforcement learning to infer reward functions from the segments. This method produces a set of Markov Decision Processes (MDPs) that best describe the input trajectories. We evaluate this approach in a car driving domain and a simulated quadcopter obstacle course, showing that it is able to recover demonstrated skills more effectively than existing methods.

Details

Possible papers

Reinforcement Learning with Parameterized Actions

Nonparametric Bayesian reward segmentation for skill discovery using inverse reinforcement learning