Multimodal Gas Detection Using E-Nose and Thermal Images: An Approach Utilizing SRGAN and Sparse Autoencoder

Electronic nose and thermal images are effective ways to diagnose the presence of gases in real-time real-time. Multimodal fusion of these modalities can result in the development of highly accurate diagnostic systems. The low-cost thermal imaging software produces low-resolution thermal images in grayscale format, hence necessitating methods for improving the resolution and colorizing the images. The objective of this paper is to develop and train a super-resolution generative adversarial network for improving the resolution of the thermal images, followed by a sparse autoencoder for colorization of thermal images and a multimodal convolutional neural network for gas detection using electronic nose and thermal images. The dataset used comprises 6400 thermal images and electronic nose measurements for four classes. A multimodal Convolutional Neural Network (CNN) comprising an EfficientNetB2 pre-trained model was developed using both early and late feature fusion. The Super Resolution Generative Adversarial Network (SRGAN) model was developed and trained on low and high-resolution thermal images. A sparse autoencoder was trained on the grayscale and colorized thermal images. The SRGAN was trained on low and high-resolution thermal images, achieving a Structural Similarity Index (SSIM) of 90.28, a Peak Signal-to-Noise Ratio (PSNR) of 68.74, and a Mean Absolute Error (MAE) of 0.066. The autoencoder model produced an MAE of 0.035, a Mean Squared Error (MSE) of 0.006, and a Root Mean Squared Error (RMSE) of 0.0705. The multimodal CNN, trained on these images and electronic nose measurements using both early and late fusion techniques, achieved accuracies of 97.89% and 98.55%, respectively. Hence, the proposed framework can be of great aid for the integration with low-cost software to generate high quality thermal camera images and highly accurate detection of gases in real-time.