Author name cluster

Warren Powell

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

RLDM Conference 2013 Conference Abstract

A Scalable Approximate Dynamic Programming Algorithm for Control of Multidimensional Energy Storage Portfolios

Daniel Salas
Warren Powell

We present and benchmark an approximate dynamic programming algorithm that is capable of designing near-optimal control policies for time-dependent, finite-horizon energy storage problems, where wind energy supply, demand and electricity prices may evolve stochastically. In deterministic comparisons, the algorithm was able to design storage policies that are within 0. 08 % of optimal. In stochastic compar- isons, the policies are within 1. 34 % of optimal, much better than those obtained using model predictive control. We used the algorithm to analyze a dual-storage system with different capacities and losses, and found that the policy properly uses the low-loss device (which is typically much more expensive) for high- frequency variations. We also tested the algorithm on a five-device system. The algorithm easily scales to handle heterogeneous portfolios of storage devices distributed over the grid and more complex storage networks.

PDF Details

RLDM Conference 2013 Conference Abstract

An Approximate Dynamic Programming Algorithm for Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage

Daniel Jiang
Warren Powell

There is growing interest in the use of grid–level storage to smooth variations in the loads that are likely to arise with increased use of wind and solar energy. Battery arbitrage, the process of buying, storing, and selling electricity to exploit variations in electricity spot prices, is becoming an important way of paying for expensive investments into grid level storage. Independent system operators such as the NYISO (New York Independent System Operator) require that battery storage operators place bids into an hour–ahead market (although settlements may occur in increments as small as 5 minutes, which is considered near “real–time”). The operator has to place these bids without knowing the energy level in the battery at the beginning of the hour, while simultaneously accounting for the value of left–over energy at the end of the hour. The problem is formulated using the dynamic programming framework. We describe and employ a convergent approximate dynamic programming (ADP) algorithm that exploits monotonicity of the value functions to find a profitable bidding policy.

PDF Details

AAAI Conference 2012 Conference Paper

An Intelligent Battery Controller Using Bias-Corrected Q-learning

Donghun Lee
Warren Powell

The transition to renewables requires storage to help smooth short-term variations in energy from wind and solar sources, as well as to respond to spikes in electricity spot prices, which can easily exceed 20 times their average. Efficient operation of an energy storage device is a fundamental problem, yet classical algorithms such as Q-learning can diverge for millions of iterations, limiting practical applications. We have traced this behavior to the max-operator bias, which is exacerbated by high volatility in the reward function, and high discount factors due to the small time steps. We propose an elegant bias correction procedure and demonstrate its effectiveness.

PDF Details

NeurIPS Conference 2010 Conference Paper

Nonparametric Density Estimation for Stochastic Optimization with an Observable State Variable

Lauren Hannah
Warren Powell
David Blei

We study convex stochastic optimization problems where a noisy objective function value is observed after a decision is made. There are many stochastic optimization problems whose behavior depends on an exogenous state variable which affects the shape of the objective function. Currently, there is no general purpose algorithm to solve this class of problems. We use nonparametric density estimation for the joint distribution of state-outcome pairs to create weights for previous observations. The weights effectively group similar states. Those similar to the current state are used to create a convex, deterministic approximation of the objective function. We propose two solution methods that depend on the problem characteristics: function-based and gradient-based optimization. We offer two weighting schemes, kernel based weights and Dirichlet process based weights, for use with the solution methods. The weights and solution methods are tested on a synthetic multi-product newsvendor problem and the hour ahead wind commitment problem. Our results show Dirichlet process weights can offer substantial benefits over kernel based weights and, more generally, that nonparametric estimation methods provide good solutions to otherwise intractable problems.

PDF Details