A Study on Enhancing the Visual Fidelity of Aviation Simulators Using WGAN-GP for Remote Sensing Image Color Correction
Source abstract
Abstract
When implementing outside-the-window (OTW) visuals in aviation tactical simulators, maintaining terrain image color consistency is critical for enhancing pilot immersion and focus. However, due to various environmental factors, inconsistent image colors in terrain can cause visual confusion and diminish realism. To address these issues, a color correction technique based on a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) is proposed. The proposed WGAN-GP model utilizes multi-scale feature extraction and Wasserstein distance to effectively measure and adjust the color distribution difference between the input image and the reference image. This approach can preserve the texture and structural characteristics of the image while maintaining color consistency. In particular, by converting Bands 2, 3, and 4 of the BigEarthNet-S2 dataset into RGB images as the reference image and preprocessing the reference image to serve as the input image, it is demonstrated that the proposed WGAN-GP model can handle large-scale remote sensing images containing various lighting conditions and color differences. The experimental results showed that the proposed WGAN-GP model outperformed traditional methods, such as histogram matching and color transfer, and was effective in reflecting the style of the reference image to the target image while maintaining the structural elements of the target image during the training process. Quantitative analysis demonstrated that the mid-stage model achieved a PSNR of 28.93 dB and an SSIM of 0.7116, which significantly outperforms traditional methods. Furthermore, the LPIPS score was reduced to 0.3978, indicating improved perceptual similarity. This approach can contribute to improving the visual elements of the simulator to enhance pilot immersion and has the potential to significantly reduce time and costs compared to the manual methods currently used by the Republic of Korea Air Force.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Enhancement Techniques · Advanced Image Fusion Techniques · Color Science and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Defense Artificial Intelligence Applications, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea
Avionics Software Development Center of Republic of Korea Air Force Logistics Command, Seosan 32024, Republic of Korea
AI Convergence Center, Sungkyunkwan University, Suwon 16419, Republic of Korea
Department of Computer Engineering, Sejong University, Seoul 05006, Republic of Korea
Department of Convergence Engineering for Intelligent Drones, Sejong University, Seoul 05006, Republic of Korea
Funding
Defense Acquisition Program Administration(DAPA)
References (31)
Show source-provided references
1. Randomized Histogram Matching: A Simple Augmentation for Unsupervised Domain Adaptation in Overhead Imagery10.1109/JSTARS.2023.3340412 ↗
2. Niu, H., Lu, Q., and Wang, C. (2018, January 27–29). Color Correction Based on Histogram Matching and Polynomial Regression for Image Stitching. Proceedings of the 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), Chongqing, China.10.1109/ICIVC.2018.8492895 ↗
3. A Color Correction Method Based on Incremental Multi-Level Iterative Histogram Matching10.1109/JSEN.2024.3432277 ↗
4. Color histogram specification by histogram warping10.1117/12.596953 ↗
5. Color transfer based remote sensing image fusion using non-separable wavelet frame transform10.1016/j.patrec.2005.02.010 ↗
6. Xiao, X.Z., and Ma, L.Z. (2006, January 12–26). Color transfer in correlated color space. Proceedings of the 2006 ACM International Conference on Virtual Reality Continuum and Its Applications (VRCIA’06), New York, NY, USA.10.1145/1128923.1128974 ↗
7. Progressive color transfer for images of arbitrary dynamic range10.1016/j.cag.2010.11.003 ↗
8. Tai, Y.W., Jia, J., and Tang, C.K. (2005, January 20–26). Local Color Transfer via Probabilistic Segmentation by Expectation-Maximization. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, San Diego, CA, USA.
9. Fabbri, C., Islam, M.J., and Sattar, J. (2018, January 21–25). Enhancing underwater imagery using generative adversarial networks. Proceedings of the 2018 IEEE International Conference on Robotics and Automation(ICRA), Brisbane, Australia.10.1109/ICRA.2018.8460552 ↗
10. MLFcGAN: Multilevel feature fusion-based conditional gan for underwater image color correction10.1109/LGRS.2019.2950056 ↗
11. Watergan: Unsupervised generative network to enable real-time color correction of monocular underwater images
12. Wang, N., Zhou, Y., Han, F., Zhu, H., and Zheng, Y. (2019). UWGAN: Underwater GAN for real-world underwater color restoration and dehazing. arXiv.
13. Yang, F.H., Guo, R.R., Cheung, R.C., and Lau, C.P. (2022, January 1–4). F-GAN: Real-Time Color Correction Model of Underwater Images. Proceedings of the TENCON 2022—2022 IEEE Region 10 Conference (TENCON), Hong Kong, China.10.1109/TENCON55691.2022.9977993 ↗
14. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C. (2017). Improved training of wasserstein gans. Advances in Neural Information Processing Systems 30 (NIPS 2017), NeurIPS.
15. Arjovsky, M., Chintala, S., and Bottou, L. (2017). Wasserstein gan. arXiv.
16. Nyberg, D. (2021). Exploring the Capabilities of Generative Adversarial Networks in Remote Sensing Applications, Linkoping University Electronic Press.
17. Clasen, K.N., Hackel, L., Burgert, T., Sumbul, G., Demir, B., and Markl, V. (2024). reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysis. arXiv.
18. Efficient Global Color, Luminance, and Contrast Consistency Optimization for Multiple Remote Sensing Images10.1109/JSTARS.2022.3229392 ↗
19. Fan, C., Chen, X., Zhong, L., Zhou, M., Shi, Y., and Duan, Y. (2017). Improved Wallis Dodging Algorithm for Large-Scale Super-Resolution Reconstruction Remote Sensing Images. Sensors, 17.10.3390/s17030623 ↗
20. Dodging Research for Digital Aerial Images
21. Accurate 3d image colour histogram transformation10.1016/S0167-8655(02)00328-8 ↗
22. Neumann, L., and Neumann, A. (2005, January 18–20). Color style transfer techniques using hue, lightness and saturation histogram matching. Proceedings of the Computational Aesthetics in Graphics, Visualization and Imaging, Girona, Spain.
23. Color transfer between images10.1109/38.946629 ↗
24. Pitie, F., Kokaram, A.C., and Dahyot, R. (2005, January 17–21). N-dimensional probability density function transfer and its application to color transfer. Proceedings of the Tenth IEEE International Conference on Computer Vision, Beijing, China.10.1109/ICCV.2005.166 ↗
25. Katayama, T., Song, T., Shimamoto, T., and Jiang, X. (2019, January 27–31). GAN-based Color Correction for Underwater Object Detection. Proceedings of the OCEANS 2019 MTS/IEEE SEATTLE, Seattle, WA, USA.10.23919/OCEANS40490.2019.8962561 ↗
26. Afifi, M., Brubaker, M.A., and Brown, M.S. (2021, January 20–25). HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00785 ↗
27. Jason, A. (2022, January 16). Colorizing and Restoring Photo and Video. Available online: https://github.com/jantic/DeOldify.
28. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
29. The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations10.1002/col.20070 ↗
30. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
31. Karras, T., Laine, S., and Aila, T. (2019, January 15–20). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00453 ↗