Author name cluster

Joachim Buhmann

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

21 papers

1 author row

NeurIPS Conference 2017 Conference Paper

Continuous DR-submodular Maximization: Structure and Algorithms

An Bian
Kfir Levy
Andreas Krause
Joachim Buhmann

DR-submodular continuous functions are important objectives with wide real-world applications spanning MAP inference in determinantal point processes (DPPs), and mean-field inference for probabilistic submodular models, amongst others. DR-submodularity captures a subclass of non-convex functions that enables both exact minimization and approximate maximization in polynomial time. In this work we study the problem of maximizing non-monotone DR-submodular continuous functions under general down-closed convex constraints. We start by investigating geometric properties that underlie such objectives, e. g. , a strong relation between (approximately) stationary points and global optimum is proved. These properties are then used to devise two optimization algorithms with provable guarantees. Concretely, we first devise a "two-phase'' algorithm with 1/4 approximation guarantee. This algorithm allows the use of existing methods for finding (approximately) stationary points as a subroutine, thus, harnessing recent progress in non-convex optimization. Then we present a non-monotone Frank-Wolfe variant with 1/e approximation guarantee and sublinear convergence rate. Finally, we extend our approach to a broader class of generalized DR-submodular continuous functions, which captures a wider spectrum of applications. Our theoretical findings are validated on synthetic and real-world problem instances.

NeurIPS Conference 2017 Conference Paper

Efficient and Flexible Inference for Stochastic Systems

Stefan Bauer
Nico Gorbach
Djordje Miladinovic
Joachim Buhmann

Many real world dynamical systems are described by stochastic differential equations. Thus parameter inference is a challenging and important problem in many disciplines. We provide a grid free and flexible algorithm offering parameter and state inference for stochastic systems and compare our approch based on variational approximations to state of the art methods showing significant advantages both in runtime and accuracy.

NeurIPS Conference 2017 Conference Paper

Scalable Variational Inference for Dynamical Systems

Nico Gorbach
Stefan Bauer
Joachim Buhmann

Gradient matching is a promising tool for learning parameters and state dynamics of ordinary differential equations. It is a grid free inference approach, which, for fully observable systems is at times competitive with numerical integration. However, for many real-world applications, only sparse observations are available or even unobserved variables are included in the model description. In these cases most gradient matching methods are difficult to apply or simply do not provide satisfactory results. That is why, despite the high computational cost, numerical integration is still the gold standard in many applications. Using an existing gradient matching approach, we propose a scalable variational inference framework which can infer states and parameters simultaneously, offers computational speedups, improved accuracy and works well even under model misspecifications in a partially observable system.

NeurIPS Conference 2016 Conference Paper

Scalable Adaptive Stochastic Optimization Using Random Projections

Gabriel Krummenacher
Brian McWilliams
Yannic Kilcher
Joachim Buhmann
Nicolai Meinshausen

Adaptive stochastic gradient methods such as AdaGrad have gained popularity in particular for training deep neural networks. The most commonly used and studied variant maintains a diagonal matrix approximation to second order information by accumulating past gradients which are used to tune the step size adaptively. In certain situations the full-matrix variant of AdaGrad is expected to attain better performance, however in high dimensions it is computationally impractical. We present Ada-LR and RadaGrad two computationally efficient approximations to full-matrix AdaGrad based on randomized dimensionality reduction. They are able to capture dependencies between features and achieve similar performance to full-matrix AdaGrad but at a much smaller computational cost. We show that the regret of Ada-LR is close to the regret of full-matrix AdaGrad which can have an up-to exponentially smaller dependence on the dimension than the diagonal variant. Empirically, we show that Ada-LR and RadaGrad perform similarly to full-matrix AdaGrad. On the task of training convolutional neural networks as well as recurrent neural networks, RadaGrad achieves faster convergence than diagonal AdaGrad.

AAAI Conference 2015 Conference Paper

Kickback Cuts Backprop’s Red-Tape: Biologically Plausible Credit Assignment in Neural Networks

David Balduzzi
Hastagiri Vanchinathan
Joachim Buhmann

Error backpropagation is an extremely effective algorithm for assigning credit in artificial neural networks. However, weight updates under Backprop depend on lengthy recursive computations and require separate output and error messages – features not shared by biological neurons, that are perhaps unnecessary. In this paper, we revisit Backprop and the credit assignment problem. We first decompose Backprop into a collection of interacting learning algorithms; provide regret bounds on the performance of these sub-algorithms; and factorize Backprop’s error signals. Using these results, we derive a new credit assignment algorithm for nonparametric regression, Kickback, that is significantly simpler than Backprop. Finally, we provide a sufficient condition for Kickback to follow error gradients, and show that Kickback matches Backprop’s performance on real-world regression benchmarks.

NeurIPS Conference 2014 Conference Paper

Fast and Robust Least Squares Estimation in Corrupted Linear Models

Brian McWilliams
Gabriel Krummenacher
Mario Lucic
Joachim Buhmann

Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings. However, these algorithms are typically not robust to outliers or corruptions in the observed covariates. The concept of influence that was developed for regression diagnostics can be used to detect such corrupted observations as shown in this paper. This property of influence -- for which we also develop a randomized approximation -- motivates our proposed subsampling algorithm for large scale corrupted linear regression which limits the influence of data points since highly influential points contribute most to the residual error. Under a general model of corrupted observations, we show theoretically and empirically on a variety of simulated and real datasets that our algorithm improves over the current state-of-the-art approximation schemes for ordinary least squares.

NeurIPS Conference 2013 Conference Paper

Correlated random features for fast semi-supervised learning

Brian McWilliams
David Balduzzi
Joachim Buhmann

This paper presents Correlated Nystrom Views (XNV), a fast semi-supervised algorithm for regression and classification. The algorithm draws on two main ideas. First, it generates two views consisting of computationally inexpensive random features. Second, multiview regression, using Canonical Correlation Analysis (CCA) on unlabeled data, biases the regression towards useful features. It has been shown that CCA regression can substantially reduce variance with a minimal increase in bias if the views contains accurate estimators. Recent theoretical and empirical work shows that regression with random features closely approximates kernel regression, implying that the accuracy requirement holds for random views. We show that XNV consistently outperforms a state-of-the-art algorithm for semi-supervised learning: substantially improving predictive performance and reducing the variability of performance on a wide variety of real-world datasets, whilst also reducing runtime by orders of magnitude.

NeurIPS Conference 2006 Conference Paper

Denoising and Dimension Reduction in Feature Space

Mikio Braun
Klaus-Robert Müller
Joachim Buhmann

We show that the relevant information about a classification problem in feature space is contained up to negligible error in a finite number of leading kernel PCA components if the kernel matches the underlying learning problem. Thus, kernels not only transform data sets such that good generalization can be achieved even by linear discriminant functions, but this transformation is also performed in a manner which makes economic use of feature space dimensions. In the best case, kernels provide efficient implicit representations of the data to perform classification. Practically, we propose an algorithm which enables us to recover the subspace and dimensionality relevant for good classification. Our algorithm can therefore be applied (1) to analyze the interplay of data set and kernel in a geometric fashion, (2) to help in model selection, and to (3) de-noise in feature space in order to yield better classification results.

NeurIPS Conference 2005 Conference Paper

Fusion of Similarity Data in Clustering

Tilman Lange
Joachim Buhmann

Fusing multiple information sources can yield signiﬁcant beneﬁts to suc- cessfully accomplish learning tasks. Many studies have focussed on fus- ing information in supervised learning contexts. We present an approach to utilize multiple information sources in the form of similarity data for unsupervised learning. Based on similarity information, the clustering task is phrased as a non-negative matrix factorization problem of a mix- ture of similarity measurements. The tradeoff between the informative- ness of data sources and the sparseness of their mixture is controlled by an entropy-based weighting mechanism. For the purpose of model se- lection, a stability-based approach is employed to ensure the selection of the most self-consistent hypothesis. The experiments demonstrate the performance of the method on toy as well as real world data sets.

NeurIPS Conference 2004 Conference Paper

A Hidden Markov Model for de Novo Peptide Sequencing

Bernd Fischer
Volker Roth
Jonas Grossmann
Sacha Baginsky
Wilhelm Gruissem
Franz Roos
Peter Widmayer
Joachim Buhmann

De novo Sequencing of peptides is a challenging task in proteome re- search. While there exist reliable DNA-sequencing methods, the high- throughput de novo sequencing of proteins by mass spectrometry is still an open problem. Current approaches suffer from a lack in precision to detect mass peaks in the spectrograms. In this paper we present a novel method for de novo peptide sequencing based on a hidden Markov model. Experiments effectively demonstrate that this new method signif- icantly outperforms standard approaches in matching quality.

NeurIPS Conference 2003 Conference Paper

Clustering with the Connectivity Kernel

Bernd Fischer
Volker Roth
Joachim Buhmann

Clustering aims at extracting hidden structure in dataset. While the prob- lem of ﬁnding compact clusters has been widely studied in the litera- ture, extracting arbitrarily formed elongated structures is considered a much harder problem. In this paper we present a novel clustering algo- rithm which tackles the problem by a two step procedure: ﬁrst the data are transformed in such a way that elongated structures become compact ones. In a second step, these new objects are clustered by optimizing a compactness-based criterion. The advantages of the method over related approaches are threefold: (i) robustness properties of compactness-based criteria naturally transfer to the problem of extracting elongated struc- tures, leading to a model which is highly robust against outlier objects; (ii) the transformed distances induce a Mercer kernel which allows us to formulate a polynomial approximation scheme to the generally NP- hard clustering problem; (iii) the new method does not contain free kernel parameters in contrast to methods like spectral clustering or mean-shift clustering.

NeurIPS Conference 2002 Conference Paper

Going Metric: Denoising Pairwise Data

Volker Roth
Julian Laub
Klaus-Robert Müller
Joachim Buhmann

Pairwise data in empirical sciences typically violate metricity, ei(cid: 173) ther due to noise or due to fallible estimates, and therefore are hard to analyze by conventional machine learning technology. In this paper we therefore study ways to work around this problem. First, we present an alternative embedding to multi-dimensional scaling (MDS) that allows us to apply a variety of classical ma(cid: 173) chine learning and signal processing algorithms. The class of pair(cid: 173) wise grouping algorithms which share the shift-invariance property is statistically invariant under this embedding procedure, leading to identical assignments of objects to clusters. Based on this new vectorial representation, denoising methods are applied in a sec(cid: 173) ond step. Both steps provide a theoretically well controlled setup to translate from pairwise data to the respective denoised met(cid: 173) ric representation. We demonstrate the practical usefulness of our theoretical reasoning by discovering structure in protein sequence data bases, visibly improving performance upon existing automatic methods.

NeurIPS Conference 2002 Conference Paper

Stability-Based Model Selection

Tilman Lange
Mikio Braun
Volker Roth
Joachim Buhmann

Model selection is linked to model assessment, which is the problem of comparing different models, or model parameters, for a speciﬁc learning task. For supervised learning, the standard practical technique is cross- validation, which is not applicable for semi-supervised and unsupervised settings. In this paper, a new model assessment scheme is introduced which is based on a notion of stability. The stability measure yields an upper bound to cross-validation in the supervised case, but extends to semi-supervised and unsupervised problems. In the experimental part, the performance of the stability measure is studied for model order se- lection in comparison to standard techniques in this area.

NeurIPS Conference 2001 Conference Paper

The Noisy Euclidean Traveling Salesman Problem and Learning

Mikio Braun
Joachim Buhmann

We consider noisy Euclidean traveling salesman problems in the plane, which are random combinatorial problems with underlying structure. Gibbs sampling is used to compute average trajectories, which estimate the underlying structure common to all instances. This procedure requires identifying the exact relationship between permutations and tours. In a learning setting, the average trajec(cid: 173) tory is used as a model to construct solutions to new instances sampled from the same source. Experimental results show that the average trajectory can in fact estimate the underlying structure and that overfitting effects occur if the trajectory adapts too closely to a single instance.

NeurIPS Conference 1999 Conference Paper

Model Selection in Clustering by Uniform Convergence Bounds

Joachim Buhmann
Marcus Held

Unsupervised learning algorithms are designed to extract struc(cid: 173) ture from data samples. Reliable and robust inference requires a guarantee that extracted structures are typical for the data source, Le. , similar structures have to be inferred from a second sample set of the same data source. The overfitting phenomenon in max(cid: 173) imum entropy based annealing algorithms is exemplarily studied for a class of histogram clustering models. Bernstein's inequality for large deviations is used to determine the maximally achievable approximation quality parameterized by a minimal temperature. Monte Carlo simulations support the proposed model selection cri(cid: 173) terion by finite temperature annealing.

NeurIPS Conference 1998 Conference Paper

Visualizing Group Structure

Marcus Held
Jan Puzicha
Joachim Buhmann

Cluster analysis is a fundamental principle in exploratory data analysis, providing the user with a description of the group struc(cid: 173) ture of given data. A key problem in this context is the interpreta(cid: 173) tion and visualization of clustering solutions in high- dimensional or abstract data spaces. In particular, probabilistic descriptions of the group structure, essential to capture inter-cluster relation(cid: 173) ships, are hardly assessable by simple inspection ofthe probabilistic assignment variables. VVe present a novel approach to the visual(cid: 173) ization of group structure. It is based on a statistical model of the object assignments which have been observed or estimated by a probabilistic clustering procedure. The objects or data points are embedded in a low dimensional Euclidean space by approximating the observed data statistics with a Gaussian mixture model. The algorithm provides a new approach to the visualization of the inher(cid: 173) ent structure for a broad variety of data types, e. g. histogram data, proximity data and co-occurrence data. To demonstrate the power of the approach, histograms of textured images are visualized as an example of a large-scale data mining application.

NeurIPS Conference 1997 Conference Paper

Active Data Clustering

Thomas Hofmann
Joachim Buhmann

Active data clustering is a novel technique for clustering of proxim(cid: 173) ity data which utilizes principles from sequential experiment design in order to interleave data generation and data analysis. The pro(cid: 173) posed active data sampling strategy is based on the expected value of information, a concept rooting in statistical decision theory. This is considered to be an important step towards the analysis of large(cid: 173) scale data sets, because it offers a way to overcome the inherent data sparseness of proximity data. '''Ie present applications to unsu(cid: 173) pervised texture segmentation in computer vision and information retrieval in document databases.

NeurIPS Conference 1997 Conference Paper

Unsupervised On-line Learning of Decision Trees for Hierarchical Data Analysis

Marcus Held
Joachim Buhmann

An adaptive on-line algorithm is proposed to estimate hierarchical data structures for non-stationary data sources. The approach is based on the principle of minimum cross entropy to derive a decision tree for data clustering and it employs a metalearning idea (learning to learn) to adapt to changes in data characteristics. Its efficiency is demonstrated by grouping non-stationary artifical data and by hierarchical segmentation of LANDSAT images.

NeurIPS Conference 1994 Conference Paper

Multidimensional Scaling and Data Clustering

Thomas Hofmann
Joachim Buhmann

Visualizing and structuring pairwise dissimilarity data are difficult combinatorial op(cid: 173) timization problems known as multidimensional scaling or pairwise data clustering. Algorithms for embedding dissimilarity data set in a Euclidian space, for clustering these data and for actively selecting data to support the clustering process are discussed in the maximum entropy framework. Active data selection provides a strategy to discover structure in a data set efficiently with partially unknown data.

NeurIPS Conference 1993 Conference Paper

Central and Pairwise Data Clustering by Competitive Neural Networks

Joachim Buhmann
Thomas Hofmann

Data clustering amounts to a combinatorial optimization problem to re(cid: 173) duce the complexity of a data representation and to increase its precision. Central and pairwise data clustering are studied in the maximum en(cid: 173) tropy framework. For central clustering we derive a set of reestimation equations and a minimization procedure which yields an optimal num(cid: 173) ber of clusters, their centers and their cluster probabilities. A meanfield approximation for pairwise clustering is used to estimate assignment probabilities. A se1fconsistent solution to multidimensional scaling and pairwise clustering is derived which yields an optimal embedding and clustering of data points in a d-dimensional Euclidian space.

NeurIPS Conference 1993 Conference Paper

Illumination-Invariant Face Recognition with a Contrast Sensitive Silicon Retina

Joachim Buhmann
Martin Lades
Frank Eeckman

Changes in lighting conditions strongly effect the performance and reli(cid: 173) ability of computer vision systems. We report face recognition results under drastically changing lighting conditions for a computer vision sys(cid: 173) tem which concurrently uses a contrast sensitive silicon retina and a con(cid: 173) ventional, gain controlled CCO camera. For both input devices the face recognition system employs an elastic matching algorithm with wavelet based features to classify unknown faces. To assess the effect of analog on -chip preprocessing by the silicon retina the CCO images have been "digitally preprocessed" with a bandpass filter to adjust the power spec(cid: 173) trum. The silicon retina with its ability to adjust sensitivity increases the recognition rate up to 50 percent. These comparative experiments demonstrate that preprocessing with an analog VLSI silicon retina gen(cid: 173) erates image data enriched with object-constant features.