Author name cluster

William Curran

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

IROS Conference 2017 Conference Paper

Neural networks for incremental dimensionality reduced reinforcement learning

William Curran
Rey Pocius
William D. Smart

State-of-the-art personal robots must perform complex manipulation tasks to be viable in assistive scenarios. However, many of these robots, like the PR2, use manipulators with high degrees-of-freedom. The complexity of these robots lead to large dimensional state spaces, which are difficult to fully explore. Our previous work introduced the IDRRL algorithm, which compresses the learning space by transforming a high-dimensional learning space onto a lower-dimensional manifold while preserving expressivity. In this work we formally prove that IDRRL maintains PAC-MDP guarantees. We then improve upon our previous formulation of IDRRL by introducing cascading autoencoders (CAE) for dimensionality reduction, producing the new algorithm IDRRL-CAE. We demonstrate the improvement of this extension over our previous formulation, IDRRL-PCA, in the Mountain Car and Swimmers domains.

Details

AAAI Conference 2016 Conference Paper

Robust Learning from Demonstration Techniques and Tools

William Curran

Large state spaces and the curse of dimensionality contribute to the complexity of a task. Learning from demonstration techniques can be combined with reinforcement learning to narrow the exploration space of an agent, but require consistent and accurate demonstrations, as well as the state-action pairs for an entire demonstration. Individuals with severe motor disabilities are often slow and prone to human errors in demonstrations while teaching. My dissertation develops tools to allow persons with severe motor disabilities, and individuals in general, to train these systems. To handle these large state spaces as well as human error, we developed Dimensionality Reduced Reinforcement Learning. To accommodate slower feedback, we will develop a movie-reel style learning from demonstration interface.

PDF Details

ICRA Conference 2015 Conference Paper

Evaluating impact in the ROS ecosystem

William Curran
Thomas Thornton
Benjamin Arvey
William D. Smart

The ROS ecosystem is an interconnected web of packages, nodes and people with no efficient means to compare, assess or visualize them. We develop a set of tools consisting of various metrics, a data visualization web app, and an active monitoring system. With these tools, we measure the current state of the ecosystem as well as determine where the community should direct their efforts. We also encourage the community to provide input on potential applications, additional metrics, and further improvements to address the needs of the ROS ecosystem. We incentivize this input by gamifying community contributions to the infrastructure. Encouraging user-driven improvements to the ROS infrastructure through the use of a leaderboard and friendly competition will advance ROS development and community support far into the future.

Details

EWRL Workshop 2015 Workshop Paper

Using PCA to Efficiently Represent State Spaces

William Curran
Tim Brys
Matthew Taylor
William Smart

Reinforcement learning algorithms need to deal with the exponential growth of states and actions when exploring optimal control in high-dimensional spaces. This is known as the curse of dimensionality. By projecting the agent’s state onto a low-dimensional manifold, we can represent the state space in a smaller and more efficient representation. By using this representation during learning, the agent can converge to a good policy much faster. We test this approach in the Mario Benchmarking Domain. When using dimensionality reduction in Mario, learning converges much faster to a good policy. But, there is a critical convergence-performance trade-off. By projecting onto a low-dimensional manifold, we are ignoring important data. In this paper, we explore this trade-off of convergence and performance. We find that learning in as few as 4 dimensions (instead of 9), we can improve performance past learning in the full dimensional space at a faster convergence rate.

Details

AAMAS Conference 2013 Conference Paper

Addressing Hard Constraints in the Air Traffic Problem through Partitioning and Difference Rewards

William Curran
Adrian Agogino
Kagan Tumer

In the US alone, weather hazards and airport congestion cause thousands of hours of delay, costing billions of dollars annually. The task of managing delay may be modeled as a multiagent congestion problem with tightly coupled agents who collectively impact the system. Reward shaping has been effective at reducing noise caused by agent interaction and improving learning in soft constraint problems. We extend those results to hard constraints that cannot be easily learned, and must be algorithmically enforced. We present an agent partitioning algorithm in conjunction with reward shaping to simplify the learning domain. Our results show that a partitioning of the agents using system features leads to up to a 1000x speed up over the straight reward shaping approach, as well as up to a 30% improvement in performance over a greedy scheduling solution, corresponding to hundreds of hours of delay saved in a single day.

PDF