Genetic Algorithm Based Deep Learning Neural Network Structure and Hyperparameter Optimization
Source abstract
Abstract
Alzheimer’s disease is one of the major challenges of population ageing, and diagnosis and prediction of the disease through various biomarkers is the key. While the application of deep learning as imaging technologies has recently expanded across the medical industry, empirical design of these technologies is very difficult. The main reason for this problem is that the performance of the Convolutional Neural Networks (CNN) differ greatly depending on the statistical distribution of the input dataset. Different hyperparameters also greatly affect the convergence of the CNN models. With this amount of information, selecting appropriate parameters for the network structure has became a large research area. Genetic Algorithm (GA), is a very popular technique to automatically select a high-performance network architecture. In this paper, we show the possibility of optimising the network architecture using GA, where its search space includes both network structure configuration and hyperparameters. To verify the performance of our Algorithm, we used an amyloid brain image dataset that is used for Alzheimer’s disease diagnosis. As a result, our algorithm outperforms Genetic CNN by 11.73% on a given classification task.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Brain Tumor Detection and Classification · Advanced Neural Network Applications · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electronic Engineering, Kyungsung University, Busan 48434, Korea
Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, UK
Institute of Convergence Bio-Health, Dong-A University, Busan 49315, Korea
Department of Nuclear Medicine, Dong-A University College of Medicine, Busan 49201, Korea
Department of Translational Biomedical Sciences, Dong-A University College of Medicine, Busan 49201, Korea
Funding
National Research Foundation of Korea
References (33)
Show source-provided references
1. Amyloid β-Protein Assembly and Alzheimer Disease10.1074/jbc.R800036200 ↗
2. Yoder, K.K. (2013). Basic PET Data Analysis Techniques. Positron Emission Tomography-Recent Developments in Instrumentation, Research and Clinical Oncological Practice, IntechOpen.
3. Co-authorship network analysis in cardiovascular research utilizing machine learning (2009–2019)10.1016/j.ijmedinf.2020.104274 ↗
4. Applications of machine learning to diagnosis and treatment of neurodegenerative diseases10.1038/s41582-020-0377-8 ↗
5. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18–22). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00907 ↗
6. Face recognition based on genetic algorithm10.1016/j.jvcir.2018.12.012 ↗
7. Xie, L., and Yuille, A. (2017, January 22–29). Genetic CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.154 ↗
8. Backpropagation Applied to Handwritten Zip Code Recognition10.1162/neco.1989.1.4.541 ↗
9. The method of steepest descent for non-linear minimization problems10.1090/qam/10667 ↗
10. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd Int. Conf. Learn. Representat, San Diego, CA, USA.
11. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
12. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
13. Srivastava, R.K., Greff, K., and Schmidhuber, J. (2015). Training Very Deep Networks. Advances in Neural Information Processing Systems 28, Curran Associates, Inc.
14. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 22–25). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
15. Mohandes, M., Codrington, C.W., and Gelfand, S.B. (July, January 28). Two adaptive stepsize rules for gradient descent and their application to the training of feedforward artificial neural networks. Proceedings of the 1994 IEEE International Conference on Neural Networks, Orlando, FL, USA.10.1109/ICNN.1994.374225 ↗
16. A method for self-determination of adaptive learning rates in back propagation10.1016/0893-6080(91)90073-E ↗
17. Nair, V., and Hinton, G.E. (2010, January 21–24). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel.
18. Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 17–19). Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30th Int. Conf. Mach. Learn, Atlanta, GA, USA.
19. Hannun, A.Y., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., and Coates, A. (2014). Deep Speech: Scaling up end-to-end speech recognition. arXiv.
20. Bottou, L. (1999). Online Learning and Stochastic Approximations. On-Line Learning in Neural Networks, Publications of the Newton Institute, Cambridge University Press.10.1017/CBO9780511569920.003 ↗
21. On the momentum term in gradient descent learning algorithms10.1016/S0893-6080(98)00116-6 ↗
22. Tieleman, T., and Hinton, G. (2012). Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, COURSERA.
23. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd Int. Conf. Learn. Representat, San Diego, CA, USA.
24. Neural Architecture Search: A Survey
25. Zoph, B., and Le, Q.V. (2017). Neural Architecture Search with Reinforcement Learning. arXiv.
26. Liu, H., Simonyan, K., and Yang, Y. (2019). DARTS: Differentiable Architecture Search. arXiv.
27. Elsken, T., Metzen, J.H., and Hutter, F. (2017). Simple And Efficient Architecture Search for Convolutional Neural Networks. arXiv.
28. Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., and Xing, E.P. (2018, January 3–8). Neural Architecture Search with Bayesian Optimisation and Optimal Transport. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
29. VGG-based BAPL Score Classification of 18F-Florbetaben Amyloid Brain PET10.15616/BSL.2018.24.4.418 ↗
30. Kang, H., Park, J.S., Cho, K., and Kang, D.Y. (2020). Visual and Quantitative Evaluation of Amyloid Brain PET Image Synthesis with Generative Adversarial Network. Appl. Sci., 10.10.3390/app10072628 ↗
31. ImageNet Classification with Deep Convolutional Neural Networks
32. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model size. arXiv.
33. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Autoencoders. Deep Learning, MIT Press.