Author name cluster

C. Giles

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

15 papers

1 author row

AAAI Conference 2016 Conference Paper

BBookX: Building Online Open Books for Personalized Learning

Chen Liang
Shuting Wang
Zhaohui Wu
Kyle Williams
Bart Pursel
Benjamin Brautigam
Sherwyn Saul
Hannah Williams

We demonstrate BBookX1, a novel system that automatically builds in collaboration with a user online open books by searching open educational resources (OER). This system explores the use of retrieval technologies to dynamically generate zero-cost materials such as textbooks for personalized learning.

AAAI Conference 2016 Conference Paper

Exploring Multiple Feature Spaces for Novel Entity Discovery

Zhaohui Wu
Yang Song
C. Giles

Continuously discovering novel entities in news and Web data is important for Knowledge Base (KB) maintenance. One of the key challenges is to decide whether an entity mention refers to an in-KB or out-of-KB entity. We propose a principled approach that learns a novel entity classiﬁer by modeling mention and entity representation into multiple feature spaces, including contextual, topical, lexical, neural embedding and query spaces. Different from most previous studies that address novel entity discovery as a submodule of entity linking systems, our model is more a generalized approach and can be applied as a pre-ﬁltering step of novel entities for any entity linking systems. Experiments on three real-world datasets show that our method signiﬁcantly outperforms existing methods on identifying novel entities.

AAAI Conference 2015 Conference Paper

A Neural Probabilistic Model for Context Based Citation Recommendation

Wenyi Huang
Zhaohui Wu
Chen Liang
Prasenjit Mitra
C. Giles

Automatic citation recommendation can be very useful for authoring a paper and is an AI-complete problem due to the challenge of bridging the semantic gap between citation context and the cited paper. It is not always easy for knowledgeable researchers to give an accurate citation context for a cited paper or to find the right paper to cite given context. To help with this problem, we propose a novel neural probabilistic model that jointly learns the semantic representations of citation contexts and cited papers. The probability of citing a paper given a citation context is estimated by training a multi-layer neural network. We implement and evaluate our model on the entire CiteSeer dataset, which at the time of this work consists of 10, 760, 318 citation contexts from 1, 017, 457 papers. We show that the proposed model significantly outperforms other stateof-the-art models in recall, MAP, MRR, and nDCG.

AAAI Conference 2015 Conference Paper

Sense-Aaware Semantic Analysis: A Multi-Prototype Word Representation Model Using Wikipedia

Zhaohui Wu
C. Giles

Human languages are naturally ambiguous, which makes it difficult to automatically understand the semantics of text. Most vector space models (VSM) treat all occurrences of a word as the same and build a single vector to represent the meaning of a word, which fails to capture any ambiguity. We present sense-aware semantic analysis (SaSA), a multi-prototype VSM for word representation based on Wikipedia, which could account for homonymy and polysemy. The “sense-specific” prototypes of a word are produced by clustering Wikipedia pages based on both local and global contexts of the word in Wikipedia. Experimental evaluation on semantic relatedness for both isolated words and words in sentential contexts and word sense induction demonstrate its effectiveness.

NeurIPS Conference 2000 Conference Paper

Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping

Rich Caruana
Steve Lawrence
C. Giles

The conventional wisdom is that backprop nets with excess hidden units generalize poorly. We show that nets with excess capacity generalize well when trained with backprop and early stopping. Experiments sug(cid: 173) gest two reasons for this: 1) Overfitting can vary significantly in different regions of the model. Excess capacity allows better fit to regions of high non-linearity, and backprop often avoids overfitting the regions of low non-linearity. 2) Regardless of size, nets learn task subcomponents in similar sequence. Big nets pass through stages similar to those learned by smaller nets. Early stopping can stop training the large net when it generalizes comparably to a smaller net. We also show that conjugate gradient can yield worse generalization because it overfits regions of low non-linearity when learning to fit regions of high non-linearity.

NeurIPS Conference 1999 Conference Paper

Robust Learning of Chaotic Attractors

Rembrandt Bakker
Jaap Schouten
Marc-Olivier Coppens
Floris Takens
C. Giles
Cor van den Bleek

A fundamental problem with the modeling of chaotic time series data is that minimizing short-term prediction errors does not guarantee a match between the reconstructed attractors of model and experiments. We introduce a modeling paradigm that simultaneously learns to short-tenn predict and to locate the outlines of the attractor by a new way of nonlinear principal component analysis. Closed-loop predictions are constrained to stay within these outlines, to prevent divergence from the attractor. Learning is exceptionally fast: parameter estimation for the 1000 sample laser data from the 1991 Santa Fe time series competition took less than a minute on a 166 MHz Pentium PC.

NeurIPS Conference 1996 Conference Paper

Representation and Induction of Finite State Machines using Time-Delay Neural Networks

Daniel Clouse
C. Giles
Bill Horne
Garrison Cottrell

This work investigates the representational and inductive capabili(cid: 173) ties of time-delay neural networks (TDNNs) in general, and of two subclasses of TDNN, those with delays only on the inputs (IDNN), and those which include delays on hidden units (HDNN). Both ar(cid: 173) chitectures are capable of representing the same class of languages, the definite memory machine (DMM) languages, but the delays on the hidden units in the HDNN helps it outperform the IDNN on problems composed of repeated features over short time windows.

NeurIPS Conference 1995 Conference Paper

Learning long-term dependencies is not as difficult with NARX networks

Tsungnan Lin
Bill Horne
Peter Tiño
C. Giles

It has recently been shown that gradient descent learning algo(cid: 173) rithms for recurrent neural networks can perform poorly on tasks that involve long-term dependencies. In this paper we explore this problem for a class of architectures called NARX networks, which have powerful representational capabilities. Previous work reported that gradient descent learning is more effective in NARX networks than in recurrent networks with "hidden states". We show that although NARX networks do not circumvent the prob(cid: 173) lem of long-term dependencies, they can greatly improve perfor(cid: 173) mance on such problems. We present some experimental 'results that show that NARX networks can often retain information for two to three times as long as conventional recurrent networks.

NeurIPS Conference 1994 Conference Paper

An experimental comparison of recurrent neural networks

Bill Horne
C. Giles

Many different discrete-time recurrent neural network architec(cid: 173) tures have been proposed. However, there has been virtually no effort to compare these arch: tectures experimentally. In this paper we review and categorize many of these architectures and compare how they perform on various classes of simple problems including grammatical inference and nonlinear system identification.

NeurIPS Conference 1994 Conference Paper

Effects of Noise on Convergence and Generalization in Recurrent Networks

Kam Jim
Bill Horne
C. Giles

We introduce and study methods of inserting synaptic noise into dynamically-driven recurrent neural networks and show that ap(cid: 173) plying a controlled amount of noise during training may improve convergence and generalization. In addition, we analyze the effects of each noise parameter (additive vs. multiplicative, cumulative vs. non-cumulative, per time step vs. per string) and predict that best overall performance can be achieved by injecting additive noise at each time step. Extensive simulations on learning the dual parity grammar from temporal strings substantiate these predictions.

NeurIPS Conference 1994 Conference Paper

Learning with Product Units

Laurens Leerink
C. Giles
Bill Horne
Marwan Jabri

The TNM staging system has been used since the early 1960's to predict breast cancer patient outcome. In an attempt to in(cid: 173) crease prognostic accuracy, many putative prognostic factors have been identified. Because the TNM stage model can not accom(cid: 173) modate these new factors, the proliferation of factors in breast cancer has lead to clinical confusion. What is required is a new computerized prognostic system that can test putative prognostic factors and integrate the predictive factors with the TNM vari(cid: 173) ables in order to increase prognostic accuracy. Using the area un(cid: 173) der the curve of the receiver operating characteristic, we compare the accuracy of the following predictive models in terms of five year breast cancer-specific survival: pTNM staging system, princi(cid: 173) pal component analysis, classification and regression trees, logistic regression, cascade correlation neural network, conjugate gradient descent neural, probabilistic neural network, and backpropagation neural network. Several statistical models are significantly more ac-

NeurIPS Conference 1992 Conference Paper

Using Prior Knowledge in a NNPDA to Learn Context-Free Languages

Sreerupa Das
C. Giles
Guo-Zheng Sun

Although considerable interest has been shown in language inference and automata induction using recurrent neural networks, success of these models has mostly been limited to regular languages. We have previ(cid: 173) ously demonstrated that Neural Network Pushdown Automaton (NNPDA) model is capable of learning deterministic context-free languages (e. g. , anbn and parenthesis languages) from examples. However, the learning task is computationally intensive. In this paper we discus some ways in which a priori knowledge about the task and data could be used for efficient learning. We also observe that such knowledge is often an experimental prerequisite for learning nontrivial languages (eg. anbncbmam ).

NeurIPS Conference 1991 Conference Paper

Neural Network Routing for Random Multistage Interconnection Networks

Mark Goudreau
C. Giles

A routing scheme that uses a neural network has been developed that can aid in establishing point-to-point communication routes through multi(cid: 173) stage interconnection networks (MINs). The neural network is a network of the type that was examined by Hopfield (Hopfield, 1984 and 1985). In this work, the problem of establishing routes through random MINs (RMINs) in a shared-memory, distributed computing system is addressed. The performance of the neural network routing scheme is compared to two more traditional approaches - exhaustive search routing and greedy rout(cid: 173) ing. The results suggest that a neural network router may be competitive for certain RMIN s.

NeurIPS Conference 1989 Conference Paper

Higher Order Recurrent Networks and Grammatical Inference

C. Giles
Guo-Zheng Sun
Hsing-Hen Chen
Yee-Chun Lee
Dong Chen

A higher order single layer recursive network easily learns to simulate a deterministic finite state machine and recognize regular grammars. When an enhanced version of this neural net state machine is connected through a common error term to an external analog stack memory, the combination can be interpreted as a neural net pushdown automata. The neural net finite state machine is given the primitives, push and POP. and is able to read the top of the stack. Through a gradient descent learning rule derived from the common error function, the hybrid network learns to effectively use the stack actions to manipUlate the stack memory and to learn simple context(cid: 173) free grammars. INTRODUCTION Biological networks readily and easily process temporal information; artificial neural networks should do the same. Recurrent neural network models permit the encoding and learning of temporal sequences. There are many recurrent neural net models. for ex(cid: 173) ample see [Jordan 1986. Pineda 1987, Williams & Zipser 1988]. Nearly all encode the current state representation of the models in the activity of the neuron and the next state is determined by the current state and input. From an automata perspective, this dynamical structure is a state machine. One formal model of sequences and machines that generate and recognize them are formal grammars and their respective automata. These models formalize some of the foundations of computer science. In the Chomsky hierarchy of formal grammars [Hopcroft & Ullman 1979] the simplest level of com(cid: 173) plexity is defmed by the finite state machine and its regular grammars. (All machines Higher Order Recurrent Networks and Grammatical Inference 381 and grammars described here are deterministic. } The next level of complexity is de(cid: 173) scribed by pushdown automata and their associated context-free grammars. The push(cid: 173) down automaton is a fmite state machine with the added power to use a stack memory. Nemal networks should be able to perform the same type of computation and thus solve such learning problems as grammatical inference [pu 1982]. Simple grammatical inference is defined as the problem of finding (learning) a grammar from a fmite set of strings, often called the teaching sample. Recall that a grammar {phrase-structured} is defined as a 4-tuple (N, V, P, S) where N and V are a nonterm i(cid: 173) na1 and terminal vocabularies, P is a finite set of production rules and S is the start sym(cid: 173) bol. Here grammatical inference is also defined as the learning of the machine that recognizes the teaching and testing samples. Potential applications of grammatical in(cid: 173) ference include such various areas as pattern recognition, information retrieval, pro(cid: 173) gramming language design, translation and compiling and graphics languages [pu 1982]. There has been a great deal of interest in teaching nemal nets to recognize grammars and simulate automata [Allen 1989. Jordan 1986. Pollack 1989. Servant-Schreiber et. a1. 1989, Williams & Zipser 1988]. Some important extensions of that work are discussed here. In particular we construct recurrent higher order nemal net state machines which have no hidden layers and seem to be at least as powerful as any nemal net multilayer state machine discussed so far. For example, the learning time and training sample size are significantly reduced. In addition, we integrate this neural net fmite state machine with an external stack memory and inform the network through a common objective function that it has at its disposal the symbol at the top of the stack and the operation primitives of push and pop. By devising a common error function which integrates the stack and the nemal net state machine, this hybrid structure learns to effectively use the the interesting work of [Williams & stack to recognize context-free grammars. Zipser 1988] a recurrent net learns only the state machine part of a Turing Machine. since the associated move, read, write operations for each input string are known and are given as part of the training set. However, the model we present learns how to manipu(cid: 173) late the push, POP. and read primitives of an external stack memory plus learns the ad(cid: 173) ditional necessary state operations and structure. HIGHER ORDER RECURRENT NETWORK The recurrent neural network utilized can be considered as a higher order modification of the network model developed by [Williams & Zipser 1988]. Recall that in a recur(cid: 173) rent net the activation state S of the neurons at time (t+l) is defined as in a state ma(cid: 173) chine automata: In (1) where F maps the state S and the input I at time t to the next state. The weight matrix W forms the mapping and is usually learned. We use a higher order form for this map(cid: 173) ping: S(t+ 1) = F ( S(t), I(t); W }

NeurIPS Conference 1987 Conference Paper

Encoding Geometric Invariances in Higher-Order Neural Networks

C. Giles
R. Griffin
T. Maxwell

We describe a method of constructing higher-order neural networks that respond invariantly under geometric transformations on the input space. By requiring each unit to satisfy a set of constraints on the interconnection weights, a particular structure is imposed on the network. A network built using such an architecture maintains its invariant performance independent of the values the weights assume, of the learning rules used, and of the form of the nonlinearities in the network. The invariance exhibited by a first(cid: 173) order network is usually of a trivial sort, e. g. , responding only to the average input in the case of translation invariance, whereas higher-order networks can perform useful functions and still exhibit the invariance. We derive the weight constraints for translation, rotation, scale, and several combinations of these transformations, and report results of simulation studies.