Subutai Ahmad

NeurIPS Conference 1994 Conference Paper

Efficient Methods for Dealing with Missing Data in Supervised Learning

• Volker Tresp
• Ralph Neuneier
• Subutai Ahmad

We present efficient algorithms for dealing with the problem of mis(cid: 173) sing inputs (incomplete feature vectors) during training and recall. Our approach is based on the approximation of the input data dis(cid: 173) tribution using Parzen windows. For recall, we obtain closed form solutions for arbitrary feedforward networks. For training, we show how the backpropagation step for an incomplete pattern can be approximated by a weighted averaged backpropagation step. The complexity of the solutions for training and recall is independent of the number of missing features. We verify our theoretical results using one classification and one regression problem.

AIJ Journal 1993 Journal Article

Connectionist models: Proceedings of the 1990 summer school

• Subutai Ahmad

NeurIPS Conference 1993 Conference Paper

Feature Densities are Required for Computing Feature Correspondences

• Subutai Ahmad

The feature correspondence problem is a classic hurdle in visual object-recognition concerned with determining the correct mapping between the features measured from the image and the features ex(cid: 173) pected by the model. In this paper we show that determining good correspondences requires information about the joint probability density over the image features. We propose "likelihood based correspondence matching" as a general principle for selecting op(cid: 173) timal correspondences. The approach is applicable to non-rigid models, allows nonlinear perspective transformations, and can op(cid: 173) timally deal with occlusions and missing features. Experiments with rigid and non-rigid 3D hand gesture recognition support the theory. The likelihood based techniques show almost no decrease in classification performance when compared to performance with perfect correspondence knowledge.

NeurIPS Conference 1993 Conference Paper

Training Neural Networks with Deficient Data

• Volker Tresp
• Subutai Ahmad
• Ralph Neuneier

We analyze how data with uncertain or missing input features can be incorporated into the training of a neural network. The gen(cid: 173) eral solution requires a weighted integration over the unknown or uncertain input although computationally cheaper closed-form so(cid: 173) lutions can be found for certain Gaussian Basis Function (GBF) networks. We also discuss cases in which heuristical solutions such as substituting the mean of an unknown input can be harmful.

NeurIPS Conference 1992 Conference Paper

Network Structuring and Training Using Rule-based Knowledge

• Volker Tresp
• Jürgen Hollatz
• Subutai Ahmad

We demonstrate in this paper how certain forms of rule-based knowledge can be used to prestructure a neural network of nor(cid: 173) malized basis functions and give a probabilistic interpretation of the network architecture. We describe several ways to assure that rule-based knowledge is preserved during training and present a method for complexity reduction that tries to minimize the num(cid: 173) ber of rules and the number of conjuncts. After training the refined rules are extracted and analyzed.

NeurIPS Conference 1992 Conference Paper

Some Solutions to the Missing Feature Problem in Vision

• Subutai Ahmad
• Volker Tresp

In visual processing the ability to deal with missing and noisy informa(cid: 173) tion is crucial. Occlusions and unreliable feature detectors often lead to situations where little or no direct information about features is availa(cid: 173) ble. However the available information is usually sufficient to highly constrain the outputs. We discuss Bayesian techniques for extracting class probabilities given partial data. The optimal solution involves inte(cid: 173) grating over the missing dimensions weighted by the local probability densities. We show how to obtain closed-form approximations to the Bayesian solution using Gaussian basis function networks. The frame(cid: 173) work extends naturally to the case of noisy features. Simulations on a complex task (3D hand gesture recognition) validate the theory. When both integration and weighting by input densities are used, performance decreases gracefully with the number of missing or noisy features. Per(cid: 173) formance is substantially degraded if either step is omitted.

NeurIPS Conference 1991 Conference Paper

VISIT: A Neural Model of Covert Visual Attention

• Subutai Ahmad

Visual attention is the ability to dynamically restrict processing to a subset of the visual field. Researchers have long argued that such a mechanism is necessary to efficiently perform many intermediate level visual tasks. This paper describes VISIT, a novel neural network model of visual attention. The current system models the search for target objects in scenes contain(cid: 173) ing multiple distractors. This is a natural task for people, it is studied extensively by psychologists, and it requires attention. The network's be(cid: 173) havior closely matches the known psychophysical data on visual search and visual attention. VISIT also matches much of the physiological data on attention and provides a novel view of the functionality of a number of visual areas. This paper concentrates on the biological plausibility of the model and its relationship to the primary visual cortex, pulvinar, superior colliculus and posterior parietal areas.

NeurIPS Conference 1989 Conference Paper

Asymptotic Convergence of Backpropagation: Numerical Experiments

• Subutai Ahmad
• Gerald Tesauro
• Yu He

Yu He Dept. of Physics Ohio State Univ. Columbus, OH 43212 We have calculated, both analytically and in simulations, the rate of convergence at long times in the backpropagation learning al(cid: 173) gorithm for networks with and without hidden units. Our basic finding for units using the standard sigmoid transfer function is lit convergence of the error for large t, with at most logarithmic cor(cid: 173) rections for networks with hidden units. Other transfer functions may lead to a 8lower polynomial rate of convergence. Our analytic calculations were presented in (Tesauro, He & Ahamd, 1989). Here we focus in more detail on our empirical measurements of the con(cid: 173) vergence rate in numerical simulations, which confirm our analytic results.

NeurIPS Conference 1988 Conference Paper

Scaling and Generalization in Neural Networks: A Case Study

• Subutai Ahmad
• Gerald Tesauro

The issues of scaling and generalization have emerged as key issues in current studies of supervised learning from examples in neural networks. Questions such as how many training patterns and training cycles are needed for a problem of a given size and difficulty, how to represent the inllUh and how to choose useful training exemplars, are of considerable theoretical and practical importance. Several intuitive rules of thumb have been obtained from empirical studies, but as yet there are few rig(cid: 173) orous results. In this paper we summarize a study Qf generalization in the simplest possible case-perceptron networks learning linearly separa(cid: 173) ble functions. The task chosen was the majority function (i. e. return a 1 if a majority of the input units are on), a predicate with a num(cid: 173) ber of useful properties. We find that many aspects of. generalization in multilayer networks learning large, difficult tasks are reproduced in this simple domain, in which concrete numerical results and even some analytic understanding can be achieved.