Author name cluster

John Moody

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

15 papers

1 author row

NeurIPS Conference 1999 Conference Paper

Constructing Heterogeneous Committees Using Input Feature Grouping: Application to Economic Forecasting

Yuansong Liao
John Moody

The committee approach has been proposed for reducing model uncertainty and improving generalization performance. The ad(cid: 173) vantage of committees depends on (1) the performance of individ(cid: 173) ual members and (2) the correlational structure of errors between members. This paper presents an input grouping technique for de(cid: 173) signing a heterogeneous committee. With this technique, all input variables are first grouped based on their mutual information. Sta(cid: 173) tistically similar variables are assigned to the same group. Each member's input set is then formed by input variables extracted from different groups. Our designed committees have less error cor(cid: 173) relation between its members, since each member observes different input variable combinations. The individual member's feature sets contain less redundant information, because highly correlated vari(cid: 173) ables will not be combined together. The member feature sets con(cid: 173) tain almost complete information, since each set contains a feature from each information group. An empirical study for a noisy and nonstationary economic forecasting problem shows that commit(cid: 173) tees constructed by our proposed technique outperform committees formed using several existing techniques.

NeurIPS Conference 1999 Conference Paper

Data Visualization and Feature Selection: New Algorithms for Nongaussian Data

Howard Yang
John Moody

Data visualization and feature selection methods are proposed based on the )oint mutual information and ICA. The visualization methods can find many good 2-D projections for high dimensional data interpretation, which cannot be easily found by the other ex(cid: 173) isting methods. The new variable selection method is found to be better in eliminating redundancy in the inputs than other methods based on simple mutual information. The efficacy of the methods is illustrated on a radar signal analysis problem to find 2-D viewing coordinates for data visualization and to select inputs for a neural network classifier. Keywords: feature selection, joint mutual information, ICA, vi(cid: 173) sualization, classification.

NeurIPS Conference 1998 Conference Paper

Reinforcement Learning for Trading

John Moody
Matthew Saffell

We propose to train trading systems by optimizing financial objec(cid: 173) tive functions via reinforcement learning. The performance func(cid: 173) tions that we consider are profit or wealth, the Sharpe ratio and our recently proposed differential Sharpe ratio for online learn(cid: 173) ing. In Moody & Wu (1997), we presented empirical results that demonstrate the advantages of reinforcement learning relative to supervised learning. Here we extend our previous work to com(cid: 173) pare Q-Learning to our Recurrent Reinforcement Learning (RRL) algorithm. We provide new simulation results that demonstrate the presence of predictability in the monthly S&P 500 Stock Index for the 25 year period 1970 through 1994, as well as a sensitivity analysis that provides economic insight into the trader's structure. Introduction: Reinforcement Learning for Thading 1 The investor's or trader's ultimate goal is to optimize some relevant measure of trading system performance, such as profit, economic utility or risk-adjusted re(cid: 173) turn. In this paper, we propose to use recurrent reinforcement learning to directly optimize such trading system performance functions, and we compare two differ(cid: 173) ent reinforcement learning methods. The first, Recurrent Reinforcement Learning, uses immediate rewards to train the trading systems, while the second (Q-Learning (Watkins 1989)) approximates discounted future rewards. These methodologies can be applied to optimizing systems designed to trade a single security or to trade port(cid: 173) folios. In addition, we propose a novel value function for risk-adjusted return that enables learning to be done online: the differential Sharpe ratio. Trading system profits depend upon sequences of interdependent decisions, and are thus path-dependent. Optimal trading decisions when the effects of transactions costs, market impact and taxes are included require knowledge of the current system state. In Moody, Wu, Liao & Saffell (1998), we demonstrate that reinforcement learning provides a more elegant and effective means for training trading systems when transaction costs are included, than do more standard supervised approaches. • The authors are also with Nonlinear Prediction Systems. 918 J. Moody and M Saffell Though much theoretical progress has been made in recent years in the area of rein(cid: 173) forcement learning, there have been relatively few successful, practical applications of the techniques. Notable examples include Neuro-gammon (Tesauro 1989), the asset trader of Neuneier (1996), an elevator scheduler (Crites & Barto 1996) and a space-shuttle payload scheduler (Zhang & Dietterich 1996). In this paper we present results for reinforcement learning trading systems that outperform the S&P 500 Stock Index over a 25-year test period, thus demonstrating the presence of predictable structure in US stock prices. The reinforcement learning algorithms compared here include our new recurrent reinforcement learning (RRL) method (Moody & Wu 1997, Moody et ai. 1998) and Q-Learning (Watkins 1989). 2 Trading Systems and Financial Performance Functions 2. 1 Structure, Profit and Wealth for Trading Systems We consider performance functions for systems that trade a single 1 security with price series Zt. The trader is assumed to take only long, neutral or short positions Ft E {-I, 0, I} of constant magnitude. The constant magnitude assumption can be easily relaxed to enable better risk control. The position Ft is established or maintained at the end of each time interval t, and is re-assessed at the end of period t + 1. A trade is thus possible at the end of each time period, although nonzero trading costs will discourage excessive trading. A trading system return R t is realized at the end of the time interval (t - 1, t] and includes the profit or loss resulting from the position F t - 1 held during that interval and any transaction cost incurred at time t due to a difference in the positions Ft- 1 and Ft. In order to properly incorporate the effects of transactions costs, market impact and taxes in a trader's decision making, the trader must have internal state information and must therefore be recurrent. An example of a single asset trading system that takes into account transactions costs and market impact has following decision function: Ft = F((}t; Ft-l. It) with It = {Zt, Zt-1, Zt-2, ··. ; Yt, Yt-1, Yt-2, .. . } where (}t denotes the (learned) system parameters at time t and It denotes the information set at time t, which includes present and past values of the price series Zt and an arbitrary number of other external variables denoted Yt. Trading systems can be optimized by maximizing performance functions U 0 such as profit, wealth, utility functions of wealth or performance ratios like the Sharpe ratio. The simplest and most natural performance function for a risk-insensitive trader is profit. The transactions cost rate is denoted 6. Additive profits are appropriate to consider if each trade is for a fixed number of shares or contracts of security Zt. This is often the case, for example, when trading small futures accounts or when trading standard US$ FX contracts in dollar(cid: 173) denominated foreign currencies. With the definitions rt = Zt - Zt-1 and r{ = 4 - 4-1 for the price returns of a risky (traded) asset and a risk-free asset (like T(cid: 173) Bills) respectively, the additive profit accumulated over T time periods with trading position size Jl > 0 is then defined as:

NeurIPS Conference 1996 Conference Paper

Multi-effect Decompositions for Financial Data Modeling

Lizhong Wu
John Moody

High frequency foreign exchange data can be decomposed into three components: the inventory effect component, the surprise infonnation (news) component and the regular infonnation component. The presence of the inventory effect and news can make analysis of trends due to the diffusion of infonnation (regular information component) difficult. We propose a neural-net-based, independent component analysis to sep(cid: 173) arate high frequency foreign exchange data into these three components. Our empirical results show that our proposed multi-effect decomposition can reveal the intrinsic price behavior.

NeurIPS Conference 1996 Conference Paper

Smoothing Regularizers for Projective Basis Function Networks

John Moody
Thorsteinn Rögnvaldsson

Smoothing regularizers for radial basis functions have been studied extensively, but no general smoothing regularizers for projective basis junctions (PBFs), such as the widely-used sigmoidal PBFs, have heretofore been proposed. We de(cid: 173) rive new classes of algebraically-simple mH'-order smoothing regularizers for networks of the form f(W, x) = L7=1 Ujg [x T Vj + Vjol + uo, with general projective basis functions g[. ]. These regularizers are: Ra(W, m) = LU; lIvjIl2m-1 GlobalForm

NeurIPS Conference 1995 Conference Paper

A Smoothing Regularizer for Recurrent Neural Networks

Lizhong Wu
John Moody

We derive a smoothing regularizer for recurrent network models by requiring robustness in prediction performance to perturbations of the training data. The regularizer can be viewed as a generaliza(cid: 173) tion of the first order Tikhonov stabilizer to dynamic models. The closed-form expression of the regularizer covers both time-lagged and simultaneous recurrent nets, with feedforward nets and one(cid: 173) layer linear nets as special cases. We have successfully tested this regularizer in a number of case studies and found that it performs better than standard quadratic weight decay. 1

NeurIPS Conference 1993 Conference Paper

Fast Pruning Using Principal Components

Asriel Levin
Todd Leen
John Moody

We present a new algorithm for eliminating excess parameters and improving network generalization after supervised training. The method, "Principal Components Pruning (PCP)", is based on prin(cid: 173) cipal component analysis of the node activations of successive layers of the network. It is simple, cheap to implement, and effective. It requires no network retraining, and does not involve calculating the full Hessian of the cost function. Only the weight and the node activity correlation matrices for each layer of nodes are required. We demonstrate the efficacy of the method on a regression problem using polynomial basis functions, and on an economic time series prediction problem using a two-layer, feedforward network.

NeurIPS Conference 1992 Conference Paper

Weight Space Probability Densities in Stochastic Learning: I. Dynamics and Equilibria

Todd Leen
John Moody

The ensemble dynamics of stochastic learning algorithms can be studied using theoretical techniques from statistical physics. We develop the equations of motion for the weight space probability densities for stochastic learning algorithms. We discuss equilibria in the diffusion approximation and provide expressions for special cases of the LMS algorithm. The equilibrium densities are not in general thermal (Gibbs) distributions in the objective function be(cid: 173) ing minimized, but rather depend upon an effective potential that includes diffusion effects. Finally we present an exact analytical expression for the time evolution of the density for a learning algo(cid: 173) rithm with weight updates proportional to the sign of the gradient. 1 Introduction: Theoretical Framework Stochastic learning algorithms involve weight updates of the form w(n+1) = w(n) + /-l(n)H[w(n), x(n)] (1) where w E 7£m is the vector of m weights, /-l is the learning rate, H[. ] E 7£m is the update function, and x(n) is the exemplar (input or input/target pair) presented

NeurIPS Conference 1991 Conference Paper

Networks with Learned Unit Response Functions

John Moody
Norman Yarvin

Feedforward networks composed of units which compute a sigmoidal func(cid: 173) tion of a weighted sum of their inputs have been much investigated. We tested the approximation and estimation capabilities of networks using functions more complex than sigmoids. Three classes of functions were tested: polynomials, rational functions, and flexible Fourier series. Un(cid: 173) like sigmoids, these classes can fit non-monotonic functions. They were compared on three problems: prediction of Boston housing prices, the sunspot count, and robot arm inverse dynamics. The complex units at(cid: 173) tained clearly superior performance on the robot arm problem, which is a highly non-monotonic, pure approximation problem. On the noisy and only mildly nonlinear Boston housing and sunspot problems, differences among the complex units were revealed; polynomials did poorly, whereas rationals and flexible Fourier series were comparable to sigmoids.

NeurIPS Conference 1991 Conference Paper

Principled Architecture Selection for Neural Networks: Application to Corporate Bond Rating Prediction

John Moody
Joachim Utans

The notion of generalization ability can be defined precisely as the pre(cid: 173) diction risk, the expected performance of an estimator in predicting new observations. In this paper, we propose the prediction risk as a measure of the generalization ability of multi-layer perceptron networks and use it to select an optimal network architecture from a set of possible architec(cid: 173) tures. We also propose a heuristic search strategy to explore the space of possible architectures. The prediction risk is estimated from the available data; here we estimate the prediction risk by v-fold cross-validation and by asymptotic approximations of generalized cross-validation or Akaike's final prediction error. We apply the technique to the problem of predicting corporate bond ratings. This problem is very attractive as a case study, since it is characterized by the limited availability of the data and by the lack of a complete a priori model which could be used to impose a structure to the network architecture. 1 Generalization and Prediction Risk The notion of generalization ability can be defined precisely as the prediction risk, the expected performance of an estimator is predicting new observations. Consider a set of observations D = {(Xj, tj); j = 1. .. N} that are assumed to be generated 683 684

NeurIPS Conference 1991 Conference Paper

The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems

John Moody

We present an analysis of how the generalization performance (expected test set error) relates to the expected training set error for nonlinear learn(cid: 173) ing systems, such as multilayer perceptrons and radial basis functions. The principal result is the following relationship (computed to second order) between the expected test set and tlaining set errors: (1) Here, n is the size of the training sample e, u; f f is the effective noise variance in the response variable( s), ,x is a regularization or weight decay parameter, and Peff(, x) is the effective number of parameters in the non(cid: 173) linear model. The expectations ( ) of training set and test set errors are taken over possible training sets e and training and test sets e' respec(cid: 173) tively. The effective number of parameters Peff(, x) usually differs from the true number of model parameters P for nonlinear or regularized models; this theoretical conclusion is supported by Monte Carlo experiments. In addition to the surprising result that Peff(, x); /; p, we propose an estimate of (1) called the generalized prediction error (GPE) which generalizes well established estimates of prediction risk such as Akaike's F P E and AI C, Mallows Cp, and Barron's PSE to the nonlinear setting. ! lCPE and Peff(>") were previously introduced in Moody (1991).

NeurIPS Conference 1991 Conference Paper

Towards Faster Stochastic Gradient Search

Christian Darken
John Moody

Stochastic gradient descent is a general algorithm which includes LMS, on-line backpropagation, and adaptive k-means clustering as special cases. The standard choices of the learning rate 1] (both adaptive and fixed func(cid: 173) tions of time) often perform quite poorly. In contrast, our recently pro(cid: 173) posed class of "search then converge" learning rate schedules (Darken and Moody, 1990) display the theoretically optimal asymptotic convergence rate and a superior ability to escape from poor local minima. However, the user is responsible for setting a key parameter. We propose here a new method(cid: 173) ology for creating the first completely automatic adaptive learning rates which achieve the optimal rate of convergence.

NeurIPS Conference 1990 Conference Paper

Note on Learning Rate Schedules for Stochastic Optimization

Christian Darken
John Moody

We present and compare learning rate schedules for stochastic gradient descent, a general algorithm which includes LMS, on-line backpropaga(cid: 173) tion and k-means clustering as special cases. We introduce "search-then(cid: 173) converge" type schedules which outperform the classical constant and "running average" (1ft) schedules both in speed of convergence and quality of solution. Introduction: Stochastic Gradient Descent 1 tion G(W). In the context of learning systems typically G(W) = £x E(W, X), i. e. The optimization task is to find a parameter vector W which minimizes a func(cid: 173) G is the average of an objective function over the exemplars, labeled E and X respectively. The stochastic gradient descent algorithm is Ll Wet) = -1](t)V'w E(W(t), X(t)). where t is the "time", and X(t) is the most recent independently-chosen random exemplar. For comparison, the deterministic gradient descent algorithm is Ll Wet) = -1](t)V'w£x E(W(t), X).

NeurIPS Conference 1989 Conference Paper

Note on Development of Modularity in Simple Cortical Models

Alex Chernajvsky
John Moody

The existence of modularity in the organization of nervous systems (e. g. cortical columns and olfactory glomeruli) is well known. We show that localized activity patterns in a layer of cells, collective excitations, can induce the formation of modular structures in the anatomical connections via a Hebbian learning mechanism. The networks are spatially homogeneous before learning, but the spon(cid: 173) taneous emergence of localized collective excitations and subse(cid: 173) quently modularity in the connection patterns breaks translational symmetry. This spontaneous symmetry breaking phenomenon is similar to those which drive pattern formation in reaction-diffusion systems. We have identified requirements on the patterns of lateral connections and on the gains of internal units which are essential for the development of modularity. These essential requirements will most likely remain operative when more complicated (and bi(cid: 173) ologically realistic) models are considered. 1 Present Address: Molecular and Cellular Physiology, Beckman Center, Stanford University, Stanford, CA 94305. 2 Please address correspondence to John Moody. 134 Chernjavsky and Moody 1 Modularity in Nervous Systems Modular organization exists throughout the nervous system on many different spa(cid: 173) tial scales. On the very small scale, synapses appear to be clustered on dendrites. On the very large scale, the brain as a whole is composed of many anatomically and functionally distinct regions. At intermediate scales, the scales of networks and maps, the brain exhibits columnar structures. The purpose of this work is to suggest possible mechanisms for the development of modular structures at the intermediate scales of networks and maps. The best known modular structure at this scale is the column. Many modality- specific variations of columnar organization are known, for example orientation selective columns, ocular dominance columns, color sensitive blobs, somatosensory barrels, and olfactory glomeruli. In addition to these anatomically well-established struc(cid: 173) tures, other more speculative modular anatomical structures may exist. These include the frontal eye fields of association cortex whose modular structure is in(cid: 173) ferred only from electrophysiology and the hypothetical existence of minicolumns and possibly neuronal groups. Although a complete biophysical picture of the development of modular structures is still unavailable, it is well established that electrical activity is crucial for the development of certain modular structures such as complex synaptic zones and oc(cid: 173) ular dominance columns (see Kalil 1989 and references therein). It is also generally conjectured that a Hebb-like mechanism is operative in this development. These observations form a basis for our operating hypothesis described below. 2 Operating Hypothesis and Modeling Approach Our hypothesis in this work is that localized activity patterns in a layer of cells induce the development of modular anatomical structure within the layer. We further hypothesize that the emergence of localized activity patterns in a layer is due to the properties of the intrinsic network dynamics and does not necessarily depend upon the system receiving localized patterns of afferent activity. Our work therefore has two parts. First, we show that localized patterns of ac(cid: 173) tivity on a preferred spatial scale, collective excitations, spontaneously emerge in homogeneous networks with appropriate lateral connectivity and cellular response properties when driven with arbitrary stimulus (see Moody 1990). Secondly, we show that these collective excitations induce the formation of modular structures in the connectivity patterns when coupled to a Hebbian learning mechanism. The emergence of collective excitations at a preferred spatial scale in a homogeneous network breaks translational symmetry and is an example of spontaneous symmetry breaking. The Hebbian learning freezes the modular structure into the anatomy. The time scale of collective excitations is short, while the Hebbian learning process occurs over a longer time scale. The spontaneous symmetry breaking mechanism is similar to that which drives pattern formation in reaction-diffusion systems (Turing 1952, Meinhardt 1982). Reaction-diffusion models have been applied to pattern for- Note on Development or Modularity in Simple Cortical Models 135

NeurIPS Conference 1988 Conference Paper

Fast Learning in Multi-Resolution Hierarchies

John Moody

A class of fast, supervised learning algorithms is presented. They use lo(cid: 173) cal representations, hashing, atld multiple scales of resolution to approximate functions which are piece-wise continuous. Inspired by Albus's CMAC model, the algorithms learn orders of magnitude more rapidly than typical imple(cid: 173) mentations of back propagation, while often achieving comparable qualities of generalization. Furthermore, unlike most traditional function approximation methods, the algorithms are well suited for use in real time adaptive signal processing. Unlike simpler adaptive systems, such as linear predictive cod(cid: 173) ing, the adaptive linear combiner, and the Kalman filter, the new algorithms are capable of efficiently capturing the structure of complicated non-linear systems. As an illustration, the algorithm is applied to the prediction of a chaotic timeseries.