Enhanced Robotic Vision System Based on Deep Learning and Image Fusion
Source abstract
Abstract
Image fusion has become one of the interesting fields that attract researchers to integrate information from different image sources. It is involved in several applications. One of the recent applications is the robotic vision. This application necessitates image enhancement of both infrared (IR) and visible images. This paper presents a Robot Human Interaction System (RHIS) based on image fusion and deep learning. The basic objective of this system is to fuse visual and IR images for efficient feature extraction from the captured images. Then, an enhancement model is carried out on the fused image to increase its quality. Several image enhancement models such as fuzzy logic, Convolutional Neural Network (CNN) and residual network (ResNet) pre-trained model are utilized on the fusion results and they are compared with each other and with the state-of-the-art works. Simulation results prove that the fuzzy logic enhancement gives the best results from the image quality perspective. Hence, the proposed system can be considered as an efficient solution for the robotic vision problem with multi-modality images.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Advanced Image Fusion Techniques · Remote-Sensing Image Classification · Image Enhancement Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (43)
Show source-provided references
1. From multi-scale decomposition to non-multi-scale decomposition methods: A comprehensive survey of image fusion techniques and its applications10.1109/ACCESS.2017.2735865 ↗
2. An overview of multi-modal medical image fusion10.1016/j.neucom.2015.07.160 ↗
3. Facial expression recognition with fusion features extracted from salient facial areas10.3390/s17040712 ↗
4. Pixel convolutional neural network for multi-focus image fusion10.1016/j.ins.2017.12.043 ↗
5. Active and dynamic information fusion for facial expression understanding from image sequences10.1109/TPAMI.2005.93 ↗
6. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition10.1016/j.ins.2019.08.066 ↗
7. Optimal fusion aided face recognition from visible and thermal face images10.1007/s11042-020-08628-9 ↗
8. Infrared and visible image fusion methods and applications: A survey10.1016/j.inffus.2018.02.004 ↗
9. A new deep learning based multi-spectral image fusion method10.3390/e21060570 ↗
10. Infrared and visible image fusion via detail preserving adversarial learning10.1016/j.inffus.2019.07.005 ↗
11. Multisensor image fusion and enhancement in spectral total variation domain10.1109/TMM.2017.2760100 ↗
12. IFCNN: A general image fusion framework based on convolutional neural network10.1016/j.inffus.2019.07.011 ↗
13. Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure10.1016/j.inffus.2016.09.006 ↗
14. Multi-scale weighted gradient-based fusion for multi-focus images10.1016/j.inffus.2013.11.005 ↗
15. A general framework for image fusion based on multi-scale transform and sparse representation10.1016/j.inffus.2014.09.004 ↗
16. Infrared and visible image fusion using a deep learning framework
17. Multi-focus image fusion with a deep convolutional neural network10.1016/j.inffus.2016.12.001 ↗
18. Deep learning for pixel-level image fusion: Recent advances and future prospects10.1016/j.inffus.2017.10.007 ↗
19. Low contrast image enhancement using convolutional neural network with simple reflection model
20. Image enhancement based on fuzzy logic10.1109/ICIP.1995.529066 ↗
21. An improved method for image enhancement of remote sensed images using fusion methods
22. Multimodal medical image fusion using nsct and dwt fusion frame work
23. Fusion method for infrared and visible images by using non-negative sparse representation10.1016/j.infrared.2014.09.019 ↗
24. Fusion of infrared and visible images for night-vision context enhancement10.1364/AO.55.006480 ↗
25. Infrared image segmentation based on multi-information fused fuzzy clustering method for electrical equipment10.1177/1729881420909600 ↗
26. A Fusion-based enhancing method for weakly illuminated images10.1016/j.sigpro.2016.05.031 ↗
27. Quantitative assessment of laser-dazzling effects through wavelet-weighted multi-scale SSIM measurements10.1016/j.optlastec.2014.10.001 ↗
28. Fuzzy C-means clustering through SSIM and patch for image segmentation10.1016/j.asoc.2019.105928 ↗
29. T. Chaira, “Medical image processing: Advanced fuzzy set theoretic techniques,” Boca Raton, Florida, USA, CRC Press, 2015.10.1201/b18019 ↗
30. Low contrast image enhancement technique by using fuzzy method
31. Chaira · Fuzzy Image Processing and Applications with MATLAB · 2009 · 45
32. Chaira · Medical Image Processing-Advanced Fuzzy Set Theoretic Techniques · 2015 · 83
33. Near-infrared shadow detection based on HDR image10.1007/s11042-022-12996-9 ↗
34. Automatic contrast enhancement using fuzzy logic for real time object recognition system
35. Efficient implementation of image interpolation as an inverse problem10.1016/j.dsp.2004.10.003 ↗
36. New techniques to conquer the image resolution enhancement problem10.2528/PIERB08020404 ↗
37. A review of quality metrics for fused image10.1016/j.aqpro.2015.02.019 ↗
38. https://www.trb.org/InformationServices/ResourcesfortheTRISDatabases.aspx.
39. Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform10.1016/j.neucom.2008.02.025 ↗
40. Multimodal fusion for multimedia analysis: A survey10.1007/s00530-010-0182-0 ↗
41. Performance evaluation of botnet DDoS attack detection using machine learning
42. The application of wavelet transform to multi-modality medical image fusion
43. Guest editorial: Image fusion: Advances in the state of the art10.1016/j.inffus.2006.04.001 ↗