Author name cluster

Warren B. Powell

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

AAMAS Conference 2019 Conference Paper

Meta-learning of Bidding Agent with Knowledge Gradient in a Fully Agent-based Sponsored Search Auction Simulator

Donghun Lee
Warren B. Powell

We take a practical approach on learning how to bid in sponsored search auctions, and model the problem of improving real world profit of advertisers in sponsored search auction as a meta-learning problem of configuring adaptive bidding agents. We construct a fully agent-based sponsored search auction simulator that 1) captures the dynamic nature of sponsored search auctions, 2) emulates the interface of Google AdWords platforms, and 3) can be customized and extended by modules. We then present Meta-LQKG algorithm, an agent-based meta-learning algorithm using knowledge gradient, and show the effect of meta-learning with Meta-LQKG on the performance of adaptive bidding agents.

PDF

ICML Conference 2016 Conference Paper

The Knowledge Gradient for Sequential Decision Making with Stochastic Binary Feedbacks

Yingfei Wang
Chu Wang
Warren B. Powell

We consider the problem of sequentially making decisions that are rewarded by “successes” and “failures” which can be predicted through an unknown relationship that depends on a partially controllable vector of attributes for each instance. The learner takes an active role in selecting samples from the instance pool. The goal is to maximize the probability of success, either after the offline training phase or minimizing regret in online learning. Our problem is motivated by real-world applications where observations are time consuming and/or expensive. With the adaptation of an online Bayesian linear classifier, we develop a knowledge-gradient type policy to guide the experiment by maximizing the expected value of information of labeling each alternative, in order to reduce the number of expensive physical experiments. We provide a finite-time analysis of the estimated error and demonstrate the performance of the proposed algorithm on both synthetic problems and benchmark UCI datasets.

Details

JMLR Journal 2011 Journal Article

Dirichlet Process Mixtures of Generalized Linear Models

Lauren A. Hannah
David M. Blei
Warren B. Powell

We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new class of methods for nonparametric regression. Given a data set of input-response pairs, the DP-GLM produces a global model of the joint distribution through a mixture of local generalized linear models. DP-GLMs allow both continuous and categorical inputs, and can model the same class of responses that can be modeled with a generalized linear model. We study the properties of the DP-GLM, and show why it provides better predictions and density estimates than existing Dirichlet process mixture regression models. We give conditions for weak consistency of the joint distribution and pointwise consistency of the regression estimate. [abs] [ pdf ][ bib ] &copy JMLR 2011. ( edit, beta )

PDF Details

JMLR Journal 2011 Journal Article

Hierarchical Knowledge Gradient for Sequential Sampling

Martijn R.K. Mes
Warren B. Powell
Peter I. Frazier

We propose a sequential sampling policy for noisy discrete global optimization and ranking and selection, in which we aim to efficiently explore a finite set of alternatives before selecting an alternative as best when exploration stops. Each alternative may be characterized by a multi-dimensional vector of categorical and numerical attributes and has independent normal rewards. We use a Bayesian probability model for the unknown reward of each alternative and follow a fully sequential sampling policy called the knowledge-gradient policy. This policy myopically optimizes the expected increment in the value of sampling information in each time period. We propose a hierarchical aggregation technique that uses the common features shared by alternatives to learn about many alternatives from even a single measurement. This approach greatly reduces the measurement effort required, but it requires some prior knowledge on the smoothness of the function in the form of an aggregation function and computational issues limit the number of alternatives that can be easily considered to the thousands. We prove that our policy is consistent, finding a globally optimal alternative when given enough measurements, and show through simulations that it performs competitively with or significantly better than other policies. [abs] [ pdf ][ bib ] &copy JMLR 2011. ( edit, beta )

PDF Details

JMLR Journal 2008 Journal Article

Value Function Approximation using Multiple Aggregation for Multiattribute Resource Management

Abraham George
Warren B. Powell
Sanjeev R. Kulkarni

We consider the problem of estimating the value of a multiattribute resource, where the attributes are categorical or discrete in nature and the number of potential attribute vectors is very large. The problem arises in approximate dynamic programming when we need to estimate the value of a multiattribute resource from estimates based on Monte-Carlo simulation. These problems have been traditionally solved using aggregation, but choosing the right level of aggregation requires resolving the classic tradeoff between aggregation error and sampling error. We propose a method that estimates the value of a resource at different levels of aggregation simultaneously, and then uses a weighted combination of the estimates. Using the optimal weights, which minimizes the variance of the estimate while accounting for correlations between the estimates, is computationally too expensive for practical applications. We have found that a simple inverse variance formula (adjusted for bias), which effectively assumes the estimates are independent, produces near-optimal estimates. We use the setting of two levels of aggregation to explain why this approximation works so well. [abs] [ pdf ][ bib ] &copy JMLR 2008. ( edit, beta )

PDF Details