Classification of Similar Sports Images Using Convolutional Neural Network with Hyper-Parameter Optimization
Source abstract
Abstract
With the exponential growth of the presence of sport in the media, the need for effective classification of sports images has become crucial. The traditional approaches require carefully hand-crafted features, which make them impractical for massive-scale data and less accurate in distinguishing images that are very similar in appearance. As the deep learning methods can automatically extract deep representation of training data and have achieved impressive performance in image classification, our goal was to apply them to automatic classification of very similar sports disciplines. For this purpose, we developed a CNN-TL-DE method for image classification using the fine-tuning of transfer learning for training a convolutional neural network model with the use of hyper-parameter optimization based on differential evolution. Through the automatic optimization of neural network topology and essential training parameters, we significantly improved the classification performance evaluated on a dataset composed from images of four similar sports—American football, rugby, soccer, and field hockey. The analysis of interpretable representation of the trained model additionally revealed interesting insights into how our model perceives images which contributed to a greater confidence in the model prediction. The performed experiments showed our proposed method to be a very competitive image classification method for distinguishing very similar sports and sport situations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI and Multimedia in Education · Anomaly Detection Techniques and Applications · Video Analysis and Summarization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Informatics, Faculty of Electrical Engineering and Computer Science, University of Maribor, SI-2000 Maribor, Slovenia
Funding
Javna Agencija za Raziskovalno Dejavnost RS
References (61)
Show source-provided references
1. Exploring feature dimensionality reduction methods for enhancing automatic sport image annotation10.1007/s11042-017-5417-z ↗
2. Pilar, P.M., Rafael, M.C., Félix, Z.O., and Gabriel, G.V. (2019). Impact of sports mass media on the behavior and health of society. A systematic review. Int. J. Environ. Res. Public Health, 16.10.3390/ijerph16030486 ↗
3. Minhas, R.A., Javed, A., Irtaza, A., Mahmood, M.T., and Joo, Y.B. (2019). Shot classification of field sports videos using AlexNet Convolutional Neural Network. Appl. Sci., 9.10.3390/app9030483 ↗
4. Fister, I., Fister, I., and Fister, D. (2019). Computational Intelligence in Sports, Springer.10.1007/978-3-030-03490-0 ↗
5. Learning by expansion: Exploiting social media for image classification with few training examples10.1016/j.neucom.2011.05.043 ↗
6. Weakly-supervised large-scale image modeling for sport scenes and its applications10.1016/j.jvcir.2019.102718 ↗
7. Ciregan, D., Meier, U., and Schmidhuber, J. (2012, January 16–21). Multi-column deep neural networks for image classification. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248110 ↗
8. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei, L. (2014, January 24–27). Large-scale video classification with convolutional neural networks. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.223 ↗
9. Moreira, D., Vasconcelos, C., Paes, A., and Velho, L. (2020, August 20). Deep Image Classification of a Wild Data Set for Olympic Sports. Available online: https://www.visgraf.impa.br/Data/RefBib/PS_PDF/workpedia2016/Workpedia_2016_paper_18.pdf.
10. Hussain, M., Bird, J.J., and Faria, D.R. (2018). A study on cnn transfer learning for image classification. UK Workshop on Computational Intelligence, Springer.10.1007/978-3-319-97982-3_16 ↗
11. On hyperparameter optimization of machine learning algorithms: Theory and practice10.1016/j.neucom.2020.07.061 ↗
12. Parameter Setting for Deep Neural Networks Using Swarm Intelligence on Phishing Websites Classification10.1142/S021821301960008X ↗
13. Vrbančič, G., Zorman, M., and Podgorelec, V. (2019, January 10). Transfer Learning Tuning Utilizing Grey Wolf Optimizer for Identification of Brain Hemorrhage from Head CT Images. Proceedings of the 2019 6th Student Computer Science Research Conference, Koper, Slovenia.10.26493/978-961-7055-82-5.61-66 ↗
14. Vrbančič, G., Pečnik, Š., and Podgorelec, V. (2020, January 24–26). Identification of COVID-19 X-ray Images using CNN with Optimized Tuning of Transfer Learning. Proceedings of the 2020 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), Novi Sad, Serbia.10.1109/INISTA49547.2020.9194615 ↗
15. Jung, Y., Hwang, E., and Kim, W. (2003). Sports image classification through Bayesian classifier. Lecture Notes in Computer Science, Proceedings of the Conference on Technology Transfer, San Sebastian, Spain, 12–14 November 2003, Springer.
16. Capodiferro, L., Costantini, L., Mangiatordi, F., and Pallotti, E. (2012, January 2–4). SVM for historical sport video classification. Proceedings of the 2012 5th International Symposium on Communications, Control and Signal Processing, Rome, Italy.10.1109/ISCCSP.2012.6217817 ↗
17. Hatem, Y., Rady, S., Ismail, R., and Bahnasy, K. (2016, January 9–11). Automatic content description and annotation of sport images using classification techniques. Proceedings of the 10th International Conference on Informatics and Systems, Cairo, Egypt.10.1145/2908446.2908458 ↗
18. Breen, C., Khan, L., and Ponnusamy, A. (2002, January 6). Image classification using neural networks and ontologies. Proceedings of the 13th International Workshop on Database and Expert Systems Applications, Aix-en-Provence, France.
19. Strategies and best practice for neural network image classification10.1080/014311697218719 ↗
20. Supervised image classification by MLP and RBF neural networks with and without an exhaustively defined set of classes10.1080/01431160310001648019 ↗
21. Design of effective neural network ensembles for image classification purposes10.1016/S0262-8856(01)00045-2 ↗
22. Amari, S., and Arbib, M.A. (1982). Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. Competition and Cooperation in nEural Nets, Springer.10.1007/978-3-642-46466-9 ↗
23. Deep convolutional neural networks for image classification: A comprehensive review10.1162/neco_a_00990 ↗
24. Automatic Classification of Motor Impairment Neural Disorders from EEG Signals Using Deep Convolutional Neural Networks
25. Automatic Detection of Heartbeats in Heart Sound Signals Using Deep Convolutional Neural Networks10.5755/j01.eie.25.3.23680 ↗
26. McQuin, C., Goodman, A., Chernyshev, V., Kamentsky, L., Cimini, B.A., Karhohs, K.W., Doan, M., Ding, L., Rafelski, S.M., and Thirstrup, D. (2018). CellProfiler 3.0: Next-generation image processing for biology. PLoS Biol., 16.10.1371/journal.pbio.2005970 ↗
27. Javaid, A., Niyaz, Q., Sun, W., and Alam, M. (2015, January 3–5). A deep learning approach for network intrusion detection system. Proceedings of the 9th EAI International Conference on Bio-Inspired Information and Communications Technologies (Formerly BIONETICS), New York, NY, USA.10.4108/eai.3-12-2015.2262516 ↗
28. Deep learning in agriculture: A survey10.1016/j.compag.2018.02.016 ↗
29. Class-dependent discretization for inductive learning from continuous and mixed-mode data10.1109/34.391407 ↗
30. Dumais, S., Platt, J., Heckerman, D., and Sahami, M. (1998, January 3–7). Inductive Learning Algorithms and Representations for Text Categorization. Proceedings of the 7th International Conference on Information and Knowledge Management, Bethesda, MD, USA.10.1145/288627.288651 ↗
31. Class noise handling for effective cost-sensitive learning by cost-guided iterative classification filtering10.1109/TKDE.2006.155 ↗
32. Test-cost sensitive classification on data with missing values10.1109/TKDE.2006.84 ↗
33. An effective algorithm for hyperparameter optimization of neural networks10.1147/JRD.2017.2709578 ↗
34. Gambella, C., Ghaddar, B., and Naoum-Sawaya, J. (2019). Optimization models for machine learning: A survey. arXiv.
35. Bergstra, J.S., Bardenet, R., Bengio, Y., and Kégl, B. (2011, January 12–17). Algorithms for hyper-parameter optimization. Proceedings of the Advances in Neural Information Processing Systems, Granada, Spain.
36. Random search for hyper-parameter optimization
37. Bacanin, N., Bezdan, T., Tuba, E., Strumberger, I., and Tuba, M. (2020). Optimizing Convolutional Neural Network Hyperparameters by Enhanced Swarm Intelligence Metaheuristics. Algorithms, 13.10.3390/a13030067 ↗
38. Bibaeva, V. (2018, January 17–20). Using metaheuristics for hyper-parameter optimization of convolutional neural networks. Proceedings of the 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), Aalborg, Denmark.10.1109/MLSP.2018.8516989 ↗
39. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces10.1023/A:1008202821328 ↗
40. Long short term memory hyperparameter optimization for a neural network based emotion recognition framework10.1109/ACCESS.2018.2868361 ↗
41. Xue, N., Triguero, I., Figueredo, G.P., and Landa-Silva, D. (2019, January 10–13). Evolving Deep CNN-LSTMs for Inventory Time Series Prediction. Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand.10.1109/CEC.2019.8789957 ↗
42. Welzer, T., Eder, J., Podgorelec, V., Wrembel, R., Ivanović, M., Gamper, J., Morzy, M., Tzouramanis, T., Darmont, J., and Kamišalić Latifić, A. (2019). Applying Differential Evolution with Threshold Mechanism for Feature Selection on a Phishing Websites Classification. New Trends in Databases and Information Systems, Springer International Publishing.10.1007/978-3-030-30278-8 ↗
43. Review of differential evolution population size10.1016/j.swevo.2016.05.003 ↗
44. A memetic differential evolution approach in noisy optimization10.1007/s12293-009-0029-4 ↗
45. Differential Evolution: A Survey of the State-of-the-Art10.1109/TEVC.2010.2059031 ↗
46. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why should i trust you?” Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
47. Zhang, Y., Song, K., Sun, Y., Tan, S., and Udell, M. (2014). “Why Should You Trust My Explanation?” Understanding Uncertainty in LIME Explanations. arXiv.
48. Schallner, L., Rabold, J., Scholz, O., and Schmid, U. (2019). Effect of Superpixel Aggregation on Explanations in LIME–A Case Study with Biological Data. Communications in Computer and Information Science, Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Würzburg, Germany, 16–20 September 2019, Springer.10.1007/978-3-030-43823-4_13 ↗
49. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30, Curran Associates, Inc.
50. Lundberg, S. (2020, September 28). SHAP Documentation. Available online: https://shap.readthedocs.io/en/latest/index.html.
51. Molnar, C. (2019). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable, Lulu. [1st ed.].
52. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
53. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
54. Automatic glaucoma classification using color fundus images based on convolutional neural networks and transfer learning10.1364/BOE.10.000892 ↗
55. Differential evolution–an easy and efficient evolutionary algorithm for model optimisation10.1016/j.agsy.2004.05.002 ↗
56. Fawcett, T. (December, January 29). Using rule sets to maximize ROC performance. Proceedings of the 2001 IEEE International Conference on Data Mining, San Jose, CA, USA.
57. Fleiss, J.L., Levin, B., and Paik, M.C. (2003). Statistical Methods for Rates and Proportions, Wiley-Interscience. [3rd ed.].10.1002/0471445428 ↗
58. Demsar, J. (2006). Statistical Comparisons of Classifiers over Multiple Data Sets. J. Mach. Learn. Res., 1–30.
59. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
60. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hoholulu, HI, USA.10.1109/CVPR.2017.243 ↗
61. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗