Backforward Propagation (Student Abstract)

In this paper we introduce Backforward Propagation, a method of completely eliminating Internal Covariate Shift (ICS). Unlike previous methods, which only indirectly reduce the impact of ICS while introducing other biases, we are able to have a surgical view at the effects ICS has on training neural networks. Our experiments show that ICS has a weight regularizing effect on models, and completely removing it enables for faster convergence of the neural network.

Authors

• George Stoica "Alexandru Ioan Cuza" University, Iasi, Romania
• Cristian Simionescu "Alexandru Ioan Cuza" University, Iasi, Romania

Keywords

• Backpropagation
• Internal Covariate Shift
• Optimization
• Regularization