journal article·2023-10-29Open access

Optimizing Multimodal Scene Recognition through Mutual Information-Based Feature Selection in Deep Learning Models

Mohamed Hammad, Samia Allaoua Chelloug, Walaa Alayed, Ahmed A. Abd El-Latif

Applied Sciences· Volume 13· Issue 21

Abstract

The field of scene recognition, which lies at the crossroads of computer vision and artificial intelligence, has experienced notable progress because of scholarly pursuits. This article introduces a novel methodology for scene recognition by combining convolutional neural networks (CNNs) with feature selection techniques based on mutual information (MI). The main goal of our study is to address the limitations inherent in conventional unimodal methods, with the aim of improving the precision and dependability of scene classification. The focus of our research is around the formulation of a comprehensive approach for scene detection, utilizing multimodal deep learning methodologies implemented on a solitary input image. Our work distinguishes itself by the innovative amalgamation of CNN- and MI-based feature selection. This integration provides distinct advantages and enhanced capabilities when compared to prevailing methodologies. In order to assess the effectiveness of our methodology, we performed tests on two openly accessible datasets, namely, the scene categorization dataset and the AID dataset. The results of these studies exhibited notable levels of precision, with accuracies of 100% and 98.83% achieved for the corresponding datasets. These findings surpass the performance of other established techniques. The primary objective of our end-to-end approach is to reduce complexity and resource requirements, hence creating a robust framework for the task of scene categorization. This work significantly advances the practical application of computer vision in various real-world scenarios, leading to a large improvement in the accuracy of scene recognition and interpretation.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Image and Video Retrieval Techniques · Advanced Neural Network Applications · Remote-Sensing Image Classification

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

EIAS Data Science Lab, College of Computer and Information Sciences, Prince Sultan University, Riyadh 11586, Saudi Arabia

Department of Information Technology, Faculty of Computers and Information, Menoufia University, Shebin El Kom 32511, Egypt

Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh 11671, Saudi Arabia

Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin (UniSZA), Besut Campus, Besut 22200, Malaysia

Department of Mathematics and Computer Science, Faculty of Science, Menoufia University, Shebin Elkom 32511, Egypt

Funding

Deputyship for Research & Innovation, Ministry of Education, in Saudi Arabia

References (30)

Show source-provided references
1. Scene recognition: A comprehensive survey10.1016/j.patcog.2020.107205 ↗
2. A survey of advances in vision-based human motion capture and analysis10.1016/j.cviu.2006.08.002 ↗
3. Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues10.1016/j.array.2021.100057 ↗
4. ROSEFusion: Random optimization for online dense reconstruction under fast camera motion10.1145/3478513.3480500 ↗
5. Graph convolutional network with triplet attention learning for person re-identification10.1016/j.ins.2022.10.105 ↗
6. A Multi-Attention Approach for Person Re-Identification Using Deep Learning10.3390/s23073678 ↗
7. Registration based on scene recognition and natural features tracking techniques for wide-area augmented reality systems10.1109/TMM.2009.2032684 ↗
8. Pawar, P.G., and Devendran, V. (2019, January 28–29). Scene understanding: A survey to see the world at a single glance. Proceedings of the 2019 2nd International Conference on Intelligent Communication and Computational Techniques (ICCT), Jaipur, India.10.1109/ICCT46177.2019.8969051 ↗
9. Joint cross-modal and unimodal features for RGB-D salient object detection10.1109/TMM.2020.3011327 ↗
10. Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks10.1016/j.isprsjprs.2021.04.006 ↗
11. Aerial scene classification through fine-tuning with adaptive learning rates and label smoothing10.3390/app10175792 ↗
12. CSDS: End-to-end aerial scenes classification with depthwise separable convolution and an attention mechanism10.1109/JSTARS.2021.3117857 ↗
13. Residual dense network based on channel-spatial attention for the scene classification of a high-resolution remote sensing image10.3390/rs12111887 ↗
14. Vision transformers for remote sensing image classification10.3390/rs13030516 ↗
15. Deep feature fusion for high-resolution aerial scene classification10.1007/s11063-019-10119-4 ↗
16. S-MAT: Semantic-Driven Masked Attention Transformer for Multi-Label Aerial Image Classification10.3390/s22145433 ↗
17. Convolutional neural network for remote-sensing scene classification: Transfer learning analysis10.3390/rs12010086 ↗
18. Deep learning-based object detection and scene perception under bad weather conditions10.3390/electronics11040563 ↗
19. Intelligent scene recognition based on deep learning10.1109/ACCESS.2021.3057075 ↗
20. Deep learning based application for indoor scene recognition10.1007/s11063-020-10231-w ↗
21. Dhanaraj, M., Sharma, M., Sarkar, T., Karnam, S., Chachlakis, D., Ptucha, R., Markopoulos, P.P., and Saber, E. (2020, January 15). Vehicle detection from multi-modal aerial imagery using YOLOv3 with mid-level fusion. Proceedings of the Big Data II: Learning, Analytics, and Applications, Online.10.1117/12.2558115 ↗
22. Multi-Modal CNN Features Fusion for Emotion Recognition: A Modified Xception Model10.1109/ACCESS.2023.3310428 ↗
23. Multi-modal multi-concept-based deep neural network for automatic image annotation10.1007/s11042-018-6555-7 ↗
24. Mutual information-based feature selection for multilabel classification10.1016/j.neucom.2013.06.035 ↗
25. Measuring relevance between discrete and continuous features based on neighborhood mutual information10.1016/j.eswa.2011.01.023 ↗
26. Adapting Feature Selection Algorithms for the Classification of Chinese Texts10.3390/systems11090483 ↗
27. Multiscale feature extraction and fusion of image and text in VQA10.1007/s44196-023-00233-6 ↗
28. Nitisha (2023, September 17). Scene Classification. Available online: https://www.kaggle.com/datasets/nitishabharathi/scene-classification.
29. JayChen (2023, September 17). AID: A Scene Classification Dataset. Available online: https://www.kaggle.com/datasets/jiayuanchengala/aid-scene-classification-datasets.
30. Manning, C.D. (2009). An Introduction to Information Retrieval, Cambridge University Press.