Author name cluster

S. Keerthi

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

1 author row

NeurIPS Conference 2006 Conference Paper

An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models

S. Keerthi
Vikas Sindhwani
Olivier Chapelle

We consider the task of tuning hyperparameters in SVM models based on minimizing a smooth performance validation function, e. g. , smoothed k-fold crossvalidation error, using non-linear optimization techniques. The key computation in this approach is that of the gradient of the validation function with respect to hyperparameters. We show that for large-scale problems involving a wide choice of kernel-based models and validation functions, this computation can be very efficiently done; often within just a fraction of the training time. Empirical results show that a near-optimal set of hyperparameters can be identified by our approach with very few training rounds and gradient computations. .

PDF Details

NeurIPS Conference 2006 Conference Paper

Branch and Bound for Semi-Supervised Support Vector Machines

Olivier Chapelle
Vikas Sindhwani
S. Keerthi

Semi-supervised SVMs (S3 VM) attempt to learn low-density separators by maximizing the margin over labeled and unlabeled examples. The associated optimization problem is non-convex. To examine the full potential of S3 VMs modulo local minima problems in current implementations, we apply branch and bound techniques for obtaining exact, global ly optimal solutions. Empirical evidence suggests that the globally optimal solution can return excellent generalization performance in situations where other implementations fail completely. While our current implementation is only applicable to small datasets, we discuss variants that can potentially lead to practically useful algorithms.

PDF Details

NeurIPS Conference 2006 Conference Paper

Relational Learning with Gaussian Processes

Wei Chu
Vikas Sindhwani
Zoubin Ghahramani
S. Keerthi

Correlation between instances is often modelled via a kernel function using in- put attributes of the instances. Relational knowledge can further reveal additional pairwise correlations between variables of interest. In this paper, we develop a class of models which incorporates both reciprocal relational information and in- put attributes using Gaussian process techniques. This approach provides a novel non-parametric Bayesian framework with a data-dependent covariance function for supervised learning tasks. We also apply this framework to semi-supervised learning. Experimental results on several real world data sets verify the usefulness of this algorithm.

PDF Details

NeurIPS Conference 2000 Conference Paper

A Variational Mean-Field Theory for Sigmoidal Belief Networks

Chiranjib Bhattacharyya
S. Keerthi

A variational derivation of Plefka's mean-field theory is presented. This theory is then applied to sigmoidal belief networks with the aid of further approximations. Empirical evaluation on small scale networks show that the proposed approximations are quite com(cid: 173) petitive.

PDF Details

NeurIPS Conference 1999 Conference Paper

Predictive App roaches for Choosing Hyperparameters in Gaussian Processes

S. Sundararajan
S. Keerthi

Gaussian Processes are powerful regression models specified by parametrized mean and covariance functions. Standard approaches to estimate these parameters (known by the name Hyperparam(cid: 173) eters) are Maximum Likelihood (ML) and Maximum APosterior (MAP) approaches. In this paper, we propose and investigate pre(cid: 173) dictive approaches, namely, maximization of Geisser's Surrogate Predictive Probability (GPP) and minimization of mean square er(cid: 173) ror with respect to GPP (referred to as Geisser's Predictive mean square Error (GPE)) to estimate the hyperparameters. We also derive results for the standard Cross-Validation (CV) error and make a comparison. These approaches are tested on a number of problems and experimental results show that these approaches are strongly competitive to existing approaches.

PDF Details