Ontology with Deep Learning for Forest Image Classification
Source abstract
Abstract
Most existing approaches to image classification neglect the concept of semantics, resulting in two major shortcomings. Firstly, categories are treated as independent even when they have a strong semantic overlap. Secondly, the features used to classify images into different categories can be the same. It has been demonstrated that the integration of ontologies and semantic relationships greatly improves image classification accuracy. In this study, a hybrid ontological bagging algorithm and an ensemble technique of convolutional neural network (CNN) models have been developed to improve forest image classification accuracy. The ontological bagging approach learns discriminative weak attributes over multiple learning instances, and the bagging concept is adopted to minimize the error propagation of the classifiers. An ensemble of ResNet50, VGG16, and Xception models is used to generate a set of features for the classifiers trained through an ontology to perform the image classification process. To the authors’ best knowledge, there are no publicly available datasets for forest-type images; hence, the images used in this study were obtained from the internet. Obtained images were put into eight categories, namely: orchards, bare land, grassland, woodland, sea, buildings, shrubs, and logged forest. Each category comprised 100 images for training and 19 images for testing; thus, in total, the dataset contained 800 images for training and 152 images for testing. Our ensemble deep learning approach with an ontology model was successfully used to classify forest images into their respective categories. The classification was based on the semantic relationship between image categories. The experimental results show that our proposed model with ontology outperformed other baseline classifiers without ontology with 96% accuracy and the lowest root-mean-square error (RMSE) of 0.532 compared to 88.8%, 86.2%, 81.6%, 64.5%, and 63.8% accuracy and 1.048, 1.094, 1.530, 1.678, and 2.090 RMSE for support-vector machines, random forest, k-nearest neighbours, Gaussian naive Bayes, and decision trees, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote-Sensing Image Classification · Advanced Image and Video Retrieval Techniques · Wood and Agarwood Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mathematics, Statistics and Computer Science, University of KwaZulu Natal, Pietermaritzburg 3209, South Africa
References (40)
Show source-provided references
1. Xu, N., Wang, J., Qi, G., Huang, T.S., and Lin, W. (2018). Computer Vision: Concepts, Methodologies, Tools, and Applications, IGI Global.
2. Subordinate-level categorization relies on high spatial frequencies to a greater degree than basic-level categorization10.3758/BF03206498 ↗
3. Griffin, G., Holub, A., and Perona, P. (2022, September 02). Caltech-256 Object Category Dataset. Available online: https://resolver.caltech.edu/CaltechAUTHORS:CNS-TR-2007-001.
4. Fei-Fei, L., Fergus, R., and Perona, P. (July, January 27). Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop, Washington, DC, USA.
5. Shao, M., Li, S., Liu, T., Tao, D., Huang, T.S., and Fu, Y. (2014, January 14–18). Learning relative features through adaptive pooling for image classification. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME), Chengdu, China.10.1109/ICME.2014.6890269 ↗
6. Griffin, G., Holub, A., and Perona, P. (2022, September 02). Caltech-UCSD Birds 200. Available online: https://resolver.caltech.edu/CaltechAUTHORS:20111026-155425465.
7. Biswas, S., Chatterjee, S., Majee, A., Sen, S., Schwenker, F., and Sarkar, R. (2021). Prediction of covid-19 from chest ct images using an ensemble of deep learning models. Appl. Sci., 11.10.3390/app11157004 ↗
8. He, T., Zhou, H., Xu, C., Hu, J., Xue, X., Xu, L., Lou, X., Zeng, K., and Wang, Q. (2023). Deep Learning in Forest Tree Species Classification Using Sentinel-2 on Google Earth Engine: A Case Study of Qingyuan County. Sustainability, 15.10.3390/su15032741 ↗
9. Ahmad, A.M., Minallah, N., Ahmed, N., Ahmad, A.M., and Fazal, N. (2020, January 10). Remote sensing based vegetation classification using machine learning algorithms. Proceedings of the 2019 International Conference on Advances in the Emerging Computing Technologies (AECT), Al Madinah Al Munawwarah, Saudi Arabia.10.1109/AECT47998.2020.9194217 ↗
10. Joseph, J.L., Kumar, V.A., and Mathew, S.P. (2021). Innovations in Electrical and Electronic Engineering, Proceedings of the ICEEE 2021, Torino, Italy, 2–3 January 2021, Springer.
11. Filali, J., Zghal, H.B., and Martinet, J. (2019, January 25–27). Ontology and hmax features-based image classification using merged classifiers. Proceedings of the International Conference on Computer Vision Theory and Applications 2019 (VISAPP’19), Prague, Czech Republic.10.5220/0007444101240134 ↗
12. Comparing HMAX and BoVW Models for Large-Scale Image Classification10.1016/j.procs.2021.08.117 ↗
13. Guo, Y., and Gu, S. (2011, January 16–22). Multi-label classification using conditional dependency networks. Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, Barcelona, Spain.
14. Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Ranzato, M., and Mikolov, T. (2013, January 5–8). Devise: A deep visual-semantic embedding model. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
15. Cisse, M.M., Usunier, N., Artieres, T., and Gallinari, P. (2013, January 5–8). Robust bloom filters for large multilabel classification tasks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
16. Cabral, R., Torre, F., Costeira, J.P., and Bernardino, A. (2011, January 12–15). Matrix completion for multi-label image classification. Proceedings of the Advances in Neural Information Processing Systems, Granada, Spain.
17. Griffin, G., and Perona, P. (2008, January 23–28). Learning and using taxonomies for fast visual categorization. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587410 ↗
18. Marszalek, M., and Schmid, C. (2007, January 17–22). Semantic hierarchies for visual object recognition. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383272 ↗
19. Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., and Adam, H. (2014, January 6–12). Large-scale object classification using label relation graphs. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10590-1_4 ↗
20. Wu, J., Yu, Y., Huang, C., and Yu, K. (2015, January 7–12). Deep multiple instance learning for image classification and auto-annotation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298968 ↗
21. Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., and Xu, W. (2016, January 27–30). Cnn-rnn: A unified framework for multi-label image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.251 ↗
22. Aggarwal, S., Gupta, S., Gupta, D., Gulzar, Y., Juneja, S., Alwan, A.A., and Nauman, A. (2023). An Artificial Intelligence-Based Stacked Ensemble Approach for Prediction of Protein Subcellular Localization in Confocal Microscopy Images. Sustainability, 15.10.3390/su15021695 ↗
23. Gulzar, Y. (2023). Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique. Sustainability, 15.10.3390/su15031906 ↗
24. Mamat, N., Othman, M.F., Abdulghafor, R., Alwan, A.A., and Gulzar, Y. (2023). Enhancing Image Annotation Technique of Fruit Classification Using a Deep Learning Approach. Sustainability, 15.10.3390/su15020901 ↗
25. Hamid, Y., Wani, S., Soomro, A.B., Alwan, A.A., and Gulzar, Y. (2022, January 25–27). Smart seed classification system based on MobileNetV2 architecture. Proceedings of the 2022 2nd International Conference on Computing and Information Technology (ICCIT), Tabuk, Saudi Arabia.10.1109/ICCIT52419.2022.9711662 ↗
26. Ontology-based image classification and annotation10.1142/S0218001420400029 ↗
27. Image classification and recognition based on deep learning and random forest algorithm10.1155/2022/2013181 ↗
28. Deep learning based supervised image classification using UAV images for forest areas classification10.1007/s12524-020-01231-3 ↗
29. ForestResNet: A deep learning algorithm for forest image classification10.1088/1742-6596/2024/1/012053 ↗
30. Images, G. (2023, January 02). Forest. Available online: https://www.istockphoto.com/photos/forest.
31. Punnet, B. (2022, August 30). Intel Image Classification Image Scene Classification of Multiclass. Available online: https://www.kaggle.com/datasets/puneet6060/intel-image-classification?resource=download.
32. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
33. Qassim, H., Verma, A., and Feinzimer, D. (2018, January 8–10). Compressed residual-VGG16 CNN model for big data places image recognition. Proceedings of the 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.10.1109/CCWC.2018.8301729 ↗
34. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
35. Lei, J., Guo, Z., and Wang, Y. (2017, January 17–19). Weakly supervised image classification with coarse and fine labels. Proceedings of the 2017 14th Conference on Computer and Robot Vision (CRV), Edmonton, AB, Canada.10.1109/CRV.2017.21 ↗
36. Transfer learning for image classification using VGG19: Caltech-101 image data set10.1007/s12652-021-03488-z ↗
37. Durand, N., Derivaux, S., Forestier, G., Wemmert, C., Gançarski, P., Boussaid, O., and Puissant, A. (2007, January 29–31). Ontology-based object recognition for remote sensing image interpretation. Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2007), Patras, Greece.10.1109/ICTAI.2007.111 ↗
38. Tan, S., Pan, J., Zhang, J., and Liu, Y. (2022). CASVM: An Efficient Deep Learning Image Classification Method Combined with SVM. Appl. Sci., 12.10.3390/app122211690 ↗
39. Abdollahpour, Z., Samani, Z.R., and Moghaddam, M.E. (2015, January 10–14). Image classification using ontology based improved visual words. Proceedings of the 2015 23rd Iranian Conference on Electrical Engineering, Tehran, Iran.10.1109/IranianCEE.2015.7146303 ↗
40. Alzubaidi, L., Zhang, J., Humaidi, A.J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M.A., Al-Amidie, M., and Farhan, L. (2021). Review of Deep Learning: Concepts, CNN Architectures, Challenges, Applications, Future Directions, Springer International Publishing.10.1186/s40537-021-00444-8 ↗