Gradient and Mangitude Based Pruning for Sparse Deep Neural Networks

Deep Neural Networks have memory and computational demands that often render them difficult to use in lowresource environments. Also, highly dense networks are over-parameterized and thus prone to overfitting. To address these problems, we introduce a novel algorithm that prunes (sparsifies) weights from the network by taking into account their magnitudes and gradients taken against a validation dataset. Unlike existing pruning methods, our method does not require the network model to be retrained once initial training is completed. On the CIFAR-10 dataset, our method reduced the number of para1 meters of MobileNet by a factor of 9X, from 14 million to 1. 5 million, with just a 3. 8% drop in accuracy.

Authors

• Kaleab Belay

Keywords

No keywords are indexed for this paper.