Improving the Robustness and Quality of Biomedical CNN Models through Adaptive Hyperparameter Tuning
Source abstract
Abstract
Deep learning is an obvious method for the detection of disease, analyzing medical images and many researchers have looked into it. However, the performance of deep learning algorithms is frequently influenced by hyperparameter selection, the question of which combination of hyperparameters are best emerges. To address this challenge, we proposed a novel algorithm for Adaptive Hyperparameter Tuning (AHT) that automates the selection of optimal hyperparameters for Convolutional Neural Network (CNN) training. All of the optimal hyperparameters for the CNN models were instantaneously selected and allocated using a novel proposed algorithm Adaptive Hyperparameter Tuning (AHT). Using AHT, enables CNN models to be highly autonomous to choose optimal hyperparameters for classifying medical images into various classifications. The CNN model (Deep-Hist) categorizes medical images into basic classes: malignant and benign, with an accuracy of 95.71%. The most dominant CNN models such as ResNet, DenseNet, and MobileNetV2 are all compared to the already proposed CNN model (Deep-Hist). Plausible classification results were obtained using large, publicly available clinical datasets such as BreakHis, BraTS, NIH-Xray and COVID-19 X-ray. Medical practitioners and clinicians can utilize the CNN model to corroborate their first malignant and benign classification assessment. The recommended Adaptive high F1 score and precision, as well as its excellent generalization and accuracy, imply that it might be used to build a pathologist’s aid tool.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: COVID-19 diagnosis using AI · AI in cancer detection · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
Department of Computer Science, Faculty of Information Technology & Computer Science, University of Central Punjab, Lahore 54000, Pakistan
Beijing Engineering Research Center for IoT Software and Systems, Beijing 100124, China
Division of Science and Technology, University of Education, Lahore 54000, Pakistan
Funding
National Key R&D Program of China
References (49)
Show source-provided references
1. Ilievski, I., Akhtar, T., Feng, J., and Shoemaker, C. (2017, January 4–9). Efficient hyperparameter optimization for deep learning algorithms using deterministic rbf surrogates. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10647 ↗
2. Ogundokun, R.O., Misra, S., Douglas, M., Damaševičius, R., and Maskeliūnas, R. (2022). Medical Internet-of-Things Based Breast Cancer Diagnosis Using Hyperparameter-Optimized Neural Networks. Future Internet, 14.10.3390/fi14050153 ↗
3. Random search for hyper-parameter optimization
4. Algorithms for hyper-parameter optimization
5. Bayesian methods in global optimization10.1007/BF00120661 ↗
6. A taxonomy of global optimization methods based on response surfaces10.1023/A:1012771025575 ↗
7. Deep-Hist: Breast cancer diagnosis through histopathological images using convolution neural network
8. A dataset for breast cancer histopathological image classification10.1109/TBME.2015.2496264 ↗
9. Henry, T., Carré, A., Lerousseau, M., Estienne, T., Robert, C., Paragios, N., and Deutsch, E. (2020, January 4). Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net neural networks: A BraTS 2020 challenge solution. Proceedings of the International MICCAI Brainlesion Workshop, Lima, Peru.10.1007/978-3-030-72084-1_30 ↗
10. Cohen, J.P., Morrison, P., Dao, L., Roth, K., Duong, T.Q., and Ghassemi, M. (2020). COVID-19 Image Data Collection: Prospective Predictions Are the Future. arXiv.10.59275/j.melba.2020-48g7 ↗
11. Automatic tuberculosis screening using chest radiographs10.1109/TMI.2013.2284099 ↗
12. Feurer, M., Eggensperger, K., Falkner, S., Lindauer, M., and Hutter, F. (2020). Auto-sklearn 2.0: Hands-free automl via meta-learning. arXiv.
13. Erickson, N., Mueller, J., Shirkov, A., Zhang, H., Larroy, P., Li, M., and Smola, A. (2020). AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. arXiv.
14. Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J.E., and Stoica, I. (2018). Tune: A research platform for distributed model selection and training. arXiv.
15. Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., and Jordan, M.I. (2018, January 8–10). Ray: A distributed framework for emerging {AI} applications. Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), Carlsbad, CA, USA.
16. The CMA evolution strategy: A comparing review10.1007/3-540-32494-1_4 ↗
17. Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019, January 4–8). Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330701 ↗
18. Li, L., Jamieson, K., Rostamizadeh, A., Gonina, E., Hardt, M., Recht, B., and Talwalkar, A. (2018). Massively Parallel Hyperparameter Tuning. arXiv.
19. Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W.M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., and Simonyan, K. (2017). Population based training of neural networks. arXiv.
20. SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
21. Falkner, S., Klein, A., and Hutter, F. (2018, January 19–21). BOHB: Robust and efficient hyperparameter optimization at scale. Proceedings of the International Conference on Machine Learning, Jinan, China.
22. Hyperband: A novel bandit-based approach to hyperparameter optimization
23. Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
24. Li, Y., Shen, Y., Jiang, H., Zhang, W., Li, J., Liu, J., Zhang, C., and Cui, B. (2022). Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale. arXiv.10.14778/3514061.3514071 ↗
25. BoTorch: A framework for efficient Monte-Carlo Bayesian optimization
26. Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Automated Machine Learning: Methods, Systems, Challenges, Springer.10.1007/978-3-030-05318-5 ↗
27. Jamieson, K., and Talwalkar, A. (2016). Non-stochastic best arm identification and hyperparameter optimization. Artificial Intelligence and Statistics, PMLR.
28. Domhan, T., Springenberg, J.T., and Hutter, F. (2015, January 25–31). Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina.
29. Klein, A., Falkner, S., Springenberg, J.T., and Hutter, F. (2017, January 24–26). Learning Curve Prediction with Bayesian Neural Networks. Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Toulon, France.
30. Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., and Sculley, D. (2017, January 13–17). Google vizier: A service for black-box optimization. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.10.1145/3097983.3098043 ↗
31. Li, Y., Shen, Y., Zhang, W., Chen, Y., Jiang, H., Liu, M., Jiang, J., Gao, J., Wu, W., and Yang, Z. (2021, January 14–18). Openbox: A generalized black-box optimization service. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Virtual.10.1145/3447548.3467061 ↗
32. Hu, Y.Q., Yu, Y., Tu, W.W., Yang, Q., Chen, Y., and Dai, W. (2019, January 8–12). Multi-fidelity automatic hyper-parameter tuning via transfer series expansion. Proceedings of the AAAI Conference on Artificial Intelligence, Atlanta, Georgia.10.1609/aaai.v33i01.33013846 ↗
33. Li, Y., Shen, Y., Jiang, J., Gao, J., Zhang, C., and Cui, B. (2021, January 2–9). Mfes-hb: Efficient hyperband with multi-fidelity quality measurements. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i10.17031 ↗
34. Asynchronous Parallel Bayesian Optimisation via Thompson Sampling
35. González, J., Dai, Z., Hennig, P., and Lawrence, N. (2016, January 9–11). Batch Bayesian optimization via local penalization. Proceedings of the Artificial Intelligence and Statistics, 19th International Conference on Artificial Intelligence and Statistics (AISTATS) 2016, Cadiz, Spain.
36. Alvi, A., Ru, B., Calliess, J.P., Roberts, S., and Osborne, M.A. (2019, January 9–15). Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
37. Klein, A., Tiao, L.C.C., Lienart, T., Archambeau, C., and Seeger, M. (2020). Model-based Asynchronous Hyperparameter and Neural Architecture Search. arXiv.
38. Siems, J., Zimmer, L., Zela, A., Lukasik, J., Keuper, M., and Hutter, F. (2020). NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search. arXiv.
39. Ma, L., Cui, J., and Yang, B. (2019, January 14–17). Deep Neural Architecture Search with Deep Graph Bayesian Optimization. Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, Thessaloniki, Greece.10.1145/3350546.3360740 ↗
40. Neural architecture search with bayesian optimisation and optimal transport
41. Real, E., Aggarwal, A., Huang, Y., and Le, Q.V. (February, January 27). Regularized evolution for image classifier architecture search. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
42. Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.J., Tian, Q., and Xiong, H. (2019, January 6–9). PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
43. Practical bayesian optimization of machine learning algorithms
44. Olson, R.S., and Moore, J.H. (2016, January 24). TPOT: A tree-based pipeline optimization tool for automating machine learning. Proceedings of the Workshop on Automatic Machine Learning, New York, NY, USA.
45. Dong, X., and Yang, Y. (2019, January 6–9). NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
46. Atteia, G., Abdel Samee, N., El-Kenawy, E.S.M., and Ibrahim, A. (2022). CNN-Hyperparameter Optimization for Diabetic Maculopathy Diagnosis in Optical Coherence Tomography and Fundus Retinography. Mathematics, 10.10.3390/math10183274 ↗
47. Hyper-parameter optimization of convolutional neural networks for classifying COVID-19 X-ray images10.2298/CSIS210209056V ↗
48. A novel adaptive momentum method for medical image classification using convolutional neural network
49. Atteia, G., Alhussan, A.A., and Samee, N.A. (2022). BO-ALLCNN: Bayesian-Based Optimized CNN for Acute Lymphoblastic Leukemia Detection in Microscopic Blood Smear Images. Sensors, 22.10.3390/s22155520 ↗