Generalization by Weight-Elimination with Application to Forecasting

Inspired by the information theoretic idea of minimum description length, we add a term to the back propagation cost function that penalizes network complexity. We give the details of the procedure, called weight-elimination, describe its dynamics, and clarify the meaning of the parameters involved. From a Bayesian perspective, the complexity term can be usefully interpreted as an assumption about prior distribution of the weights. We use this procedure to predict the sunspot time series and the notoriously noisy series of currency exchange rates.

Authors

• Andreas Weigend
• David Rumelhart
• Bernardo Huberman

Keywords

No keywords are indexed for this paper.