Employing Bayesian Optimization for Hyperparameter Tuning in Convolutional Neural Networks

This article focuses on optimizing machine learning when each solution eval- uation is costly and/or time-consuming. In particular, current deep learning models, which are complex and computationally intensive, rely on a large num- ber of hyperparameters that need to be optimized. The purpose is to maximize the performance of these models by optimizing their hyperparameters and sav- ing computational resources. In this paper, an informed search using Bayesian optimization is applied to tune the hyperparameters of the Convolutional Neural Network (CNN). Firstly, we analyze the performance of the Stochastic Gradi- ent Descent with Momentum (SGDM) optimizer to learn the weights of CNN faster and more accurately. Then, we show the importance of the momentum hyperparameter for CNN learning. Finally, Bayesian optimization is applied Special thanks to the anonymous referees ∗ This work was supported by the Polish National Science Centre under the grant No. 2019/35/B/ST6/04442

Authors

• Amgad M. Mohammed
• Enrique Onieva
• Michał Woźniak

Keywords

• Hyperparameter optimization
• Convolutional neural network
• Bayesian optimization
• Automated Machine Learning
• Stochastic gradient descent
• Stochastic gradient descent with momentum
• Deep learning optimizers