Author name cluster

Paul Barford

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

2 author rows

AAAI Conference 2017 Conference Paper

Explicit Defense Actions Against Test-Set Attacks

Scott Alfeld
Xiaojin Zhu
Paul Barford

Automated learning and decision making systems in publicfacing applications are vulnerable to malicious attacks. Examples of such systems include spam detectors, credit card fraud detectors, and network intrusion detection systems. These systems are at further risk of attack when money is directly involved, such as market forecasters or decision systems used in determining insurance or loan rates. In this paper, we consider the setting where a predictor Bob has a ﬁxed model, and an unknown attacker Alice aims to perturb (or poison) future test instances so as to alter Bob’s prediction to her beneﬁt. We focus speciﬁcally on Bob’s optimal defense actions to limit Alice’s effectiveness. We deﬁne a general framework for determining Bob’s optimal defense action against Alice’s worst-case attack. We then demonstrate our framework by considering linear predictors, where we provide tractable methods of determining the optimal defense action. Using these methods, we perform an empirical investigation of optimal defense actions for a particular class of linear models — autoregressive forecasters — and ﬁnd that for ten real world futures markets, the optimal defense action reduces the Bob’s loss by between 78 and 97%.

PDF Details

AAAI Conference 2016 Conference Paper

Data Poisoning Attacks against Autoregressive Models

Scott Alfeld
Xiaojin Zhu
Paul Barford

Forecasting models play a key role in money-making ventures in many different markets. Such models are often trained on data from various sources, some of which may be untrustworthy. An actor in a given market may be incentivised to drive predictions in a certain direction to their own beneﬁt. Prior analyses of intelligent adversaries in a machinelearning context have focused on regression and classiﬁcation. In this paper we address the non-iid setting of time series forecasting. We consider a forecaster, Bob, using a ﬁxed, known model and a recursive forecasting method. An adversary, Alice, aims to pull Bob’s forecasts toward her desired target series, and may exercise limited inﬂuence on the initial values fed into Bob’s model. We consider the class of linear autoregressive models, and a ﬂexible framework of encoding Alice’s desires and constraints. We describe a method of calculating Alice’s optimal attack that is computationally tractable, and empirically demonstrate its effectiveness compared to random and greedy baselines on synthetic and realworld time series data. We conclude by discussing defensive strategies in the face of Alice-like adversaries.

PDF Details

SoCS Conference 2016 Conference Paper

Machine Teaching as Search

Scott Alfeld
Xiaojin Zhu 0001
Paul Barford

Machine teaching (MT) studies the task of designing a training set. Specifically, given a learner (e. g. , an artificial neural network or a human) and a target model, a teacher aims to create a training set which results in the target model being learned. MT applications include optimal education design for human learners and computer security where adversaries aim to attack learning-based systems. In this work, we formulate pool-based MT as a state space search problem. We discuss the properties and challenges of the resulting problem and highlight opportunities for novel search techniques. In our preliminary study we use a beam search approach, and find that training and evaluating empirical risk of models dominate the run time of the search. Toward the goal of better search techniques for future work, we develop optimizations ranging from implementation details for specific learners to algorithm changes applicable to general blackbox learners. We conclude with a discussion of open problems and research directions.

Details