Author name cluster

Drew Bagnell

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

RLDM Conference 2015 Conference Abstract

Approximate MaxEnt Inverse Optimal Control

De-An Huang
Amir-massoud Farahmand
Kris Kitani
Drew Bagnell

Maximum entropy inverse optimal control (MaxEnt IOC) is an effective means of discovering the underlying cost function of demonstrated agent’s activity. To enable inference in large state spaces, we introduce an approximate MaxEnt IOC procedure to address the fundamental computational bottleneck stemming from calculating the partition function via dynamic programming. Approximate MaxEnt IOC is based on two components: approximate dynamic programming and Monte Carlo sampling. This approach has a finite-sample error upper bound guarantee on its excess loss. We validate the proposed method in the context of analyzing dual-agent interactions from video, where we use approximate MaxEnt IOC to simulate mental images of a single agents body pose sequence (a high-dimensional image space). We experiment with sequences image data taken from RGB data and show that it is possible to learn cost functions that lead to accurate predictions in high-dimensional problems that were previously intractable.

PDF Details

RLDM Conference 2013 Conference Abstract

Rich Sutton was right: Provably good RL via online learning

Drew Bagnell

A defining distinction between learning for control and traditional supervised learning is the influence of the learner’s own predictions on the test distribution of examples. We ignore this problem at our peril: the resulting theory provides weak guarantees for RL and control; in practice, we suffer from unstable approximate policy iteration, cascades of errors in imitation learning, and catastrophic failure in system identification for model-based RL. These common problems share a common solution – more stable learning procedures. In particular, we show that every no-regret online learner can be used to provide strong statistical guarantees for each of these settings, and moreover that such interactive learning is a requirement for good performance. The results provide theoretical support to anecdotal observations and suggest a general strategy for the analysis and synthesis of algorithms for learning control.

PDF Details

NeurIPS Conference 2012 Conference Paper

Efficient high dimensional maximum entropy modeling via symmetric partition functions

Paul Vernaza
Drew Bagnell

The application of the maximum entropy principle to sequence modeling has been popularized by methods such as Conditional Random Fields (CRFs). However, these approaches are generally limited to modeling paths in discrete spaces of low dimensionality. We consider the problem of modeling distributions over paths in continuous spaces of high dimensionality---a problem for which inference is generally intractable. Our main contribution is to show that maximum entropy modeling of high-dimensional, continuous paths is tractable as long as the constrained features possess a certain kind of low dimensional structure. In this case, we show that the associated {\em partition function} is symmetric and that this symmetry can be exploited to compute the partition function efficiently in a compressed form. Empirical results are given showing an application of our method to maximum entropy modeling of high dimensional human motion capture data.

PDF Details

NeurIPS Conference 2005 Conference Paper

On Local Rewards and Scaling Distributed Reinforcement Learning

Drew Bagnell
Andrew Ng

We consider the scaling of the number of examples necessary to achieve good performance in distributed, cooperative, multi-agent reinforcement learning, as a function of the the number of agents n. We prove a worstcase lower bound showing that algorithms that rely solely on a global reward signal to learn policies confront a fundamental limit: They require a number of real-world examples that scales roughly linearly in the number of agents. For settings of interest with a very large number of agents, this is impractical. We demonstrate, however, that there is a class of algorithms that, by taking advantage of local reward signals in large distributed Markov Decision Processes, are able to ensure good performance with a number of samples that scales as O(log n). This makes them applicable even in settings with a very large number of agents n.

PDF Details