Harris Drucker

ICML Conference 2001 Conference Paper

Relevance Feedback using Support Vector Machines

• Harris Drucker
• Behzad Shahraray
• David C. Gibbon

ICML Conference 1997 Conference Paper

Improving Regressors using Boosting Techniques

• Harris Drucker

NeurIPS Conference 1996 Conference Paper

Support Vector Regression Machines

• Harris Drucker
• Christopher Burges
• Linda Kaufman
• Alex Smola
• Vladimir Vapnik

A new regression technique based on Vapnik's concept of support vectors is introduced. We compare support vector regression (SVR) with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimensionality of the input space.

NeurIPS Conference 1995 Conference Paper

Boosting Decision Trees

• Harris Drucker
• Corinna Cortes

We introduce a constructive, incremental learning system for regression problems that models data by means of locally linear experts. In contrast to other approaches, the experts are trained independently and do not compete for data during learning. Only when a prediction for a query is required do the experts cooperate by blending their individual predic(cid: 173) tions. Each expert is trained by minimizing a penalized local cross vali(cid: 173) dation error using second order methods. In this way, an expert is able to find a local distance metric by adjusting the size and shape of the recep(cid: 173) tive field in which its predictions are valid, and also to detect relevant in(cid: 173) put features by adjusting its bias on the importance of individual input dimensions. We derive asymptotic results for our method. In a variety of simulations the properties of the algorithm are demonstrated with respect to interference, learning speed, prediction accuracy, feature detection, and task oriented incremental learning.

ICML Conference 1994 Conference Paper

Boosting and Other Machine Learning Algorithms

• Harris Drucker
• Corinna Cortes
• Larry D. Jackel
• Yann LeCun
• Vladimir Vapnik

NeurIPS Conference 1992 Conference Paper

Improving Performance in Neural Networks Using a Boosting Algorithm

• Harris Drucker
• Robert Schapire
• Patrice Simard

A boosting algorithm converts a learning machine with error rate less than 50% to one with an arbitrarily low error rate. However, the algorithm discussed here depends on having a large supply of independent training samples. We show how to circumvent this problem and generate an ensemble of learning machines whose performance in optical character recognition problems is dramatically improved over that of a single network. We report the effect of boosting on four databases (all handwritten) consisting of 12, 000 digits from segmented ZIP codes from the United State Postal Service (USPS) and the following from the National Institute of Standards and Testing (NIST): 220, 000 digits, 45, 000 upper case alphas, and 45, 000 lower case alphas. We use two performance measures: the raw error rate (no rejects) and the reject rate required to achieve a 1% error rate on the patterns not rejected. Boosting improved performance in some cases by a factor of three.