Author name cluster

Kristin Bennett

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

JMLR Journal 2003 Journal Article

Dimensionality Reduction via Sparse Support Vector Machines (Kernel Machines Section)

Jinbo Bi
Kristin Bennett
Mark Embrechts
Curt Breneman
Minghu Song

We describe a methodology for performing variable ranking and selection using support vector machines (SVMs). The method constructs a series of sparse linear SVMs to generate linear models that can generalize well, and uses a subset of nonzero weighted variables found by the linear models to produce a final nonlinear model. The method exploits the fact that a linear SVM (no kernels) with l 1 -norm regularization inherently performs variable selection as a side-effect of minimizing capacity of the SVM model. The distribution of the linear model weights provides a mechanism for ranking and interpreting the effects of variables. Starplots are used to visualize the magnitude and variance of the weights for each variable. We illustrate the effectiveness of the methodology on synthetic data, benchmark problems, and challenging regression problems in drug design. This method can dramatically reduce the number of variables and outperforms SVMs trained using all attributes and using the attributes selected according to correlation coefficients. The visualization of the resulting models is useful for understanding the role of underlying variables.

PDF Details

NeurIPS Conference 2001 Conference Paper

Duality, Geometry, and Support Vector Regression

J. Bi
Kristin Bennett

We develop an intuitive geometric framework for support vector regression (SVR). By examining when (cid: 15)-tubes exist, we show that SVR can be regarded as a classi(cid: 12)cation problem in the dual space. Hard and soft (cid: 15)-tubes are constructed by separating the convex or reduced convex hulls respectively of the training data with the response variable shifted up and down by (cid: 15). A novel SVR model is proposed based on choosing the max-margin plane between the two shifted datasets. Maximizing the margin corresponds to shrinking the e(cid: 11)ective (cid: 15)-tube. In the proposed approach the e(cid: 11)ects of the choices of all parameters become clear geometrically.

PDF Details

NeurIPS Conference 2000 Conference Paper

A Linear Programming Approach to Novelty Detection

Colin Campbell
Kristin Bennett

Novelty detection involves modeling the normal behaviour of a sys(cid: 173) tem hence enabling detection of any divergence from normality. It has potential applications in many areas such as detection of ma(cid: 173) chine damage or highlighting abnormal features in medical data. One approach is to build a hypothesis estimating the support of the normal data i. e. constructing a function which is positive in the region where the data is located and negative elsewhere. Recently kernel methods have been proposed for estimating the support of a distribution and they have performed well in practice - training involves solution of a quadratic programming problem. In this pa(cid: 173) per we propose a simpler kernel method for estimating the support based on linear programming. The method is easy to implement and can learn large datasets rapidly. We demonstrate the method on medical and fault detection datasets. 1

PDF Details

NeurIPS Conference 1998 Conference Paper

Semi-Supervised Support Vector Machines

Kristin Bennett
Ayhan Demiriz

We introduce a semi-supervised support vector machine (S3yM) method. Given a training set of labeled data and a working set of unlabeled data, S3YM constructs a support vector machine us(cid: 173) ing both the training and working sets. We use S3 YM to solve the transduction problem using overall risk minimization (ORM) posed by Yapnik. The transduction problem is to estimate the value of a classification function at the given points in the working set. This contrasts with the standard inductive learning problem of estimating the classification function at all possible values and then using the fixed function to deduce the classes of the working set data. We propose a general S3YM model that minimizes both the misclassification error and the function capacity based on all the available data. We show how the S3YM model for I-norm lin(cid: 173) ear support vector machines can be converted to a mixed-integer program and then solved exactly using integer programming. Re(cid: 173) sults of S3YM and the standard I-norm support vector machine approach are compared on ten data sets. Our computational re(cid: 173) sults support the statistical learning theory results showing that incorporating working data improves generalization when insuffi(cid: 173) cient training information is available. In every case, S3YM either improved or showed no significant difference in generalization com(cid: 173) pared to the traditional approach. Semi-Supervised Support Vector Machines

PDF Details