journal article·2023Open access

Natural Language Processing with Optimal Deep Learning-Enabled Intelligent Image Captioning System

Radwa Marzouk, Eatedal Alabdulkreem, Mohamed K. Nour, Mesfer Al Duhayyim, Mahmoud Othman, Abu Sarwar Zamani, Ishfaq Yaseen, Abdelwahed Motwakel

Computers, Materials & Continua· Volume 74· Issue 2

Abstract

The recent developments in Multimedia Internet of Things (MIoT) devices, empowered with Natural Language Processing (NLP) model, seem to be a promising future of smart devices. It plays an important role in industrial models such as speech understanding, emotion detection, home automation, and so on. If an image needs to be captioned, then the objects in that image, its actions and connections, and any silent feature that remains under-projected or missing from the images should be identified. The aim of the image captioning process is to generate a caption for image. In next step, the image should be provided with one of the most significant and detailed descriptions that is syntactically as well as semantically correct. In this scenario, computer vision model is used to identify the objects and NLP approaches are followed to describe the image. The current study develops a Natural Language Processing with Optimal Deep Learning Enabled Intelligent Image Captioning System (NLPODL-IICS). The aim of the presented NLPODL-IICS model is to produce a proper description for input image. To attain this, the proposed NLPODL-IICS follows two stages such as encoding and decoding processes. Initially, at the encoding side, the proposed NLPODL-IICS model makes use of Hunger Games Search (HGS) with Neural Search Architecture Network (NASNet) model. This model represents the input data appropriately by inserting it into a predefined length vector. Besides, during decoding phase, Chimp Optimization Algorithm (COA) with deeper Long Short Term Memory (LSTM) approach is followed to concatenate the description sentences produced by the method. The application of HGS and COA algorithms helps in accomplishing proper parameter tuning for NASNet and LSTM models respectively. The proposed NLPODL-IICS model was experimentally validated with the help of two benchmark datasets. A widespread comparative analysis confirmed the superior performance of NLPODL-IICS model over other models.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Multimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Human Pose and Action Recognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (27)

Show source-provided references
1. Image captioning: A comprehensive survey
2. Image captioning using deep learning: A systematic literature review10.14569/IJACSA.2020.0110537 ↗
3. From show to tell: A survey on deep learning-based image captioning
4. Learning to evaluate image captioning
5. Exploring visual relationship for image captioning
6. Pointing novel objects in image captioning
7. A comprehensive survey of deep learning for image captioning10.1145/3295748 ↗
8. Stack-captioning: Coarse-to-fine learning for image captioning
9. Federated learning with blockchain assisted image classification for clustered UAV networks10.32604/cmc.2022.025473 ↗
10. Toward remote sensing image retrieval under a deep image captioning perspective10.1109/JSTARS.2020.3013818 ↗
11. Fuzzy cognitive maps with bird swarm intelligence optimization-based remote sensing image classification
12. A detailed review of prevailing image captioning methods using deep learning techniques10.1007/s11042-021-11293-1 ↗
13. Image captioning with deep bidirectional LSTMs and multi-task learning
14. Deep learning for military image captioning
15. Image captioning–A deep learning approach
16. Deep learning approaches based on transformer architectures for image captioning tasks10.1109/ACCESS.2022.3161428 ↗
17. A Reference-based model using deep learning for image captioning
18. Modeling of hyperparameter tuned deep learning model for automated image captioning10.3390/math10030288 ↗
19. Features to text: A comprehensive survey of deep learning on semantic segmentation and image captioning10.1155/2021/5538927 ↗
20. Radhika · Nature Inspired Computing for Data Science, Studies in Computational Intelligence Book Series · 2020 · 871 · 57
21. A novel hunger games search optimization-based artificial neural network for predicting ground vibration intensity induced by mine blasting10.1007/s11053-021-09903-8 ↗
22. Unsupervised pre-training of a deep LSTM-based stacked autoencoder for multivariate time series forecasting problems10.1038/s41598-019-55320-6 ↗
23. Chimp optimization algorithm10.1016/j.eswa.2020.113338 ↗
24. SChoA: A newly fusion of sine and cosine with chimp optimization algorithm for HLS of datapaths in digital filters and engineering applications
25. Adaptive combination of tag and link-based user similarity in flickr
26. Show and tell: Lessons learned from the 2015 MSCOCO image captioning challenge10.1109/TPAMI.2016.2587640 ↗
27. Multilayer dense attention model for image caption10.1109/ACCESS.2019.2917771 ↗