Author name cluster

Michael Tipping

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

NeurIPS Conference 2002 Conference Paper

Bayesian Image Super-Resolution

Michael Tipping
Christopher Bishop

The extraction of a single high-quality image from a set of low(cid: 173) resolution images is an important problem which arises in fields such as remote sensing, surveillance, medical imaging and the ex(cid: 173) traction of still images from video. Typical approaches are based on the use of cross-correlation to register the images followed by the inversion of the transformation from the unknown high reso(cid: 173) lution image to the observed low resolution images, using regular(cid: 173) ization to resolve the ill-posed nature of the inversion process. In this paper we develop a Bayesian treatment of the super-resolution problem in which the likelihood function for the image registra(cid: 173) tion parameters is based on a marginalization over the unknown high-resolution image. This approach allows us to estimate the unknown point spread function, and is rendered tractable through the introduction of a Gaussian process prior over images. Results indicate a significant improvement over techniques based on MAP (maximum a-posteriori) point optimization of the high resolution image and associated registration parameters.

PDF Details

NeurIPS Conference 2001 Conference Paper

Analysis of Sparse Bayesian Learning

Anita Faul
Michael Tipping

The recent introduction of the 'relevance vector machine' has effec(cid: 173) tively demonstrated how sparsity may be obtained in generalised linear models within a Bayesian framework. Using a particular form of Gaussian parameter prior, 'learning' is the maximisation, with respect to hyperparameters, of the marginal likelihood of the data. This paper studies the properties of that objective func(cid: 173) tion, and demonstrates that conditioned on an individual hyper(cid: 173) parameter, the marginal likelihood has a unique maximum which is computable in closed form. It is further shown that if a derived 'sparsity criterion' is satisfied, this maximum is exactly equivalent to 'pruning' the corresponding parameter from the model.

PDF Details

NeurIPS Conference 2000 Conference Paper

Sparse Kernel Principal Component Analysis

Michael Tipping

'Kernel' principal component analysis (PCA) is an elegant non(cid: 173) linear generalisation of the popular linear data analysis method, where a kernel function implicitly defines a nonlinear transforma(cid: 173) tion into a feature space wherein standard PCA is performed. Un(cid: 173) fortunately, the technique is not 'sparse', since the components thus obtained are expressed in terms of kernels associated with ev(cid: 173) ery training vector. This paper shows that by approximating the covariance matrix in feature space by a reduced number of exam(cid: 173) ple vectors, using a maximum-likelihood approach, we may obtain a highly sparse form of kernel PCA without loss of effectiveness.

PDF Details

NeurIPS Conference 1999 Conference Paper

The Relevance Vector Machine

Michael Tipping

The support vector machine (SVM) is a state-of-the-art technique for regression and classification, combining excellent generalisation properties with a sparse kernel representation. However, it does suffer from a number of disadvantages, notably the absence of prob(cid: 173) abilistic outputs, the requirement to estimate a trade-off parameter and the need to utilise 'Mercer' kernel functions. In this paper we introduce the Relevance Vector Machine (RVM), a Bayesian treat(cid: 173) ment of a generalised linear model of identical functional form to the SVM. The RVM suffers from none of the above disadvantages, and examples demonstrate that for comparable generalisation per(cid: 173) formance, the RVM requires dramatically fewer kernel functions. 1

PDF Details

NeurIPS Conference 1998 Conference Paper

Probabilistic Visualisation of High-Dimensional Binary Data

Michael Tipping

We present a probabilistic latent-variable framework for data visu(cid: 173) alisation, a key feature of which is its applicability to binary and categorical data types for which few established methods exist. A variational approximation to the likelihood is exploited to derive a fast algorithm for determining the model parameters. Illustrations of application to real and synthetic binary data sets are given.

PDF Details

NeurIPS Conference 1996 Conference Paper

NeuroScale: Novel Topographic Feature Extraction using RBF Networks

David Lowe
Michael Tipping

Dimension-reducing feature extraction neural network techniques which also preserve neighbourhood relationships in data have tra(cid: 173) ditionally been the exclusive domain of Kohonen self organising maps. Recently, we introduced a novel dimension-reducing feature extraction process, which is also topographic, based upon a Radial Basis Function architecture. It has been observed that the gener(cid: 173) alisation performance of the system is broadly insensitive to model order complexity and other smoothing factors such as the kernel widths, contrary to intuition derived from supervised neural net(cid: 173) work models. In this paper we provide an effective demonstration of this property and give a theoretical justification for the apparent 'self-regularising' behaviour of the 'NEUROSCALE' architecture. 1 'NeuroScale': A Feed-forward Neural Network Topographic Transformation Recently an important class of topographic neural network based feature extraction approaches, which can be related to the traditional statistical methods of Sammon Mappings (Sammon, 1969) and Multidimensional Scaling (Kruskal, 1964), have been introduced (Mao and Jain, 1995; Lowe, 1993; Webb, 1995; Lowe and Tipping, 1996). These novel alternatives to Kohonen-like approaches for topographic feature extraction possess several interesting properties. For instance, the NEuROSCALE architecture has the empirically observed property that the generalisation perfor- 544 D. Lowe and M. E. Tipping mance does not seem to depend critically on model order complexity, contrary to intuition based upon knowledge of its supervised counterparts. This paper presents evidence for their 'self-regularising' behaviour and provides an explanation in terms of the curvature of the trained models. We now provide a brief introduction to the NEUROSCALE philosophy of nonlinear topographic feature extraction. Further details may be found in (Lowe, 1993; Lowe and Tipping, 1996). We seek a dimension-reducing, topographic transformation of data for the purposes of visualisation and analysis. By 'topographic', we imply that the geometric structure of the data be optimally preserved in the transformation, and the embodiment of this constraint is that the inter-point distances in the feature space should correspond as closely as possible to those distances in the data space. The implementation of this principle by a neural network is very simple. A Radial Basis Function (RBF) neural network is utilised to predict the coordinates of the data point in the transformed feature space. The locations of the feature points are indirectly determined by adjusting the weights of the network. The transformation is determined by optimising the network parameters in order to minimise a suitable error measure that embodies the topographic principle. The specific details of this alternative approach are as follows. Given an m(cid: 173) dimensional input space of N data points x q, an n-dimensional feature space of points Yq is generated such that the relative positions of the feature space points minimise the error, or 'STRESS', term: N E = 2: 2: (d~p - dqp)2,

PDF Details