Author name cluster

Bill Horne

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

1 author row

NeurIPS Conference 1997 Conference Paper

Function Approximation with the Sweeping Hinge Algorithm

Don Hush
Fernando Lozano
Bill Horne

We present a computationally efficient algorithm for function ap(cid: 173) proximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the method of fitting the residual. The task of fitting individual nodes is accom(cid: 173) plished using a new algorithm that searchs for the best fit by solving a sequence of Quadratic Programming problems. This approach of(cid: 173) fers significant advantages over derivative-based search algorithms (e. g. backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stop(cid: 173) ping criterion, a deterministic methodology for seeking "good" local minima, good scaling properties and a robust numerical implemen(cid: 173) tation.

PDF Details

NeurIPS Conference 1996 Conference Paper

Representation and Induction of Finite State Machines using Time-Delay Neural Networks

Daniel Clouse
C. Giles
Bill Horne
Garrison Cottrell

This work investigates the representational and inductive capabili(cid: 173) ties of time-delay neural networks (TDNNs) in general, and of two subclasses of TDNN, those with delays only on the inputs (IDNN), and those which include delays on hidden units (HDNN). Both ar(cid: 173) chitectures are capable of representing the same class of languages, the definite memory machine (DMM) languages, but the delays on the hidden units in the HDNN helps it outperform the IDNN on problems composed of repeated features over short time windows.

PDF Details

NeurIPS Conference 1995 Conference Paper

Learning long-term dependencies is not as difficult with NARX networks

Tsungnan Lin
Bill Horne
Peter Tiño
C. Giles

It has recently been shown that gradient descent learning algo(cid: 173) rithms for recurrent neural networks can perform poorly on tasks that involve long-term dependencies. In this paper we explore this problem for a class of architectures called NARX networks, which have powerful representational capabilities. Previous work reported that gradient descent learning is more effective in NARX networks than in recurrent networks with "hidden states". We show that although NARX networks do not circumvent the prob(cid: 173) lem of long-term dependencies, they can greatly improve perfor(cid: 173) mance on such problems. We present some experimental 'results that show that NARX networks can often retain information for two to three times as long as conventional recurrent networks.

PDF Details

NeurIPS Conference 1994 Conference Paper

An experimental comparison of recurrent neural networks

Bill Horne
C. Giles

Many different discrete-time recurrent neural network architec(cid: 173) tures have been proposed. However, there has been virtually no effort to compare these arch: tectures experimentally. In this paper we review and categorize many of these architectures and compare how they perform on various classes of simple problems including grammatical inference and nonlinear system identification.

PDF Details

NeurIPS Conference 1994 Conference Paper

Effects of Noise on Convergence and Generalization in Recurrent Networks

Kam Jim
Bill Horne
C. Giles

We introduce and study methods of inserting synaptic noise into dynamically-driven recurrent neural networks and show that ap(cid: 173) plying a controlled amount of noise during training may improve convergence and generalization. In addition, we analyze the effects of each noise parameter (additive vs. multiplicative, cumulative vs. non-cumulative, per time step vs. per string) and predict that best overall performance can be achieved by injecting additive noise at each time step. Extensive simulations on learning the dual parity grammar from temporal strings substantiate these predictions.

PDF Details

NeurIPS Conference 1994 Conference Paper

Learning with Product Units

Laurens Leerink
C. Giles
Bill Horne
Marwan Jabri

The TNM staging system has been used since the early 1960's to predict breast cancer patient outcome. In an attempt to in(cid: 173) crease prognostic accuracy, many putative prognostic factors have been identified. Because the TNM stage model can not accom(cid: 173) modate these new factors, the proliferation of factors in breast cancer has lead to clinical confusion. What is required is a new computerized prognostic system that can test putative prognostic factors and integrate the predictive factors with the TNM vari(cid: 173) ables in order to increase prognostic accuracy. Using the area un(cid: 173) der the curve of the receiver operating characteristic, we compare the accuracy of the following predictive models in terms of five year breast cancer-specific survival: pTNM staging system, princi(cid: 173) pal component analysis, classification and regression trees, logistic regression, cascade correlation neural network, conjugate gradient descent neural, probabilistic neural network, and backpropagation neural network. Several statistical models are significantly more ac-

PDF Details

NeurIPS Conference 1993 Conference Paper

Bounds on the complexity of recurrent neural network implementations of finite state machines

Bill Horne
Don Hush

In this paper the efficiency of recurrent neural network implementa(cid: 173) tions of m-state finite state machines will be explored. Specifically, it will be shown that the node complexity for the unrestricted case can be bounded above by 0 ( fo). It will also be shown that the node complexity is 0 (y'm log m) when the weights and thresholds are restricted to the set {-I, I}, and 0 (m) when the fan-in is re(cid: 173) stricted to two. Matching lower bounds will be provided for each of these upper bounds assuming that the state of the FSM can be encoded in a subset of the nodes of size rlog m 1.

PDF Details