Panchromatic and Hyperspectral Image Fusion Using Ratio Residual Attention Networks
Source abstract
Abstract
Hyperspectral remote sensing images provide rich spectral information about land surface features and are widely used in fields such as environmental monitoring, disaster assessment, and land classification. However, effectively leveraging the spectral information in hyperspectral images remains a significant challenge. In this paper, we propose a hyperspectral pansharpening method based on ratio transformation and residual networks, which significantly enhances both spatial details and spectral fidelity. The method generates an initial image through ratio transformation and refines it using a residual attention network. Additionally, specialized loss functions are designed to preserve both spatial and spectral details. Experimental results demonstrate that, when evaluated on the EO-1 and Chikusei datasets, the proposed method outperforms other methods in terms of both visual quality and quantitative metrics, particularly in spatial detail clarity and spectral fidelity. This approach effectively addresses the limitations of existing technologies and shows great potential for high-resolution remote sensing image processing applications.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Fusion Techniques · Remote-Sensing Image Classification · Infrared Target Detection Methodologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
Funding
the Natural Science Foundation of Chongqing
References (33)
Show source-provided references
1. An overview on spectral and spatial information fusion for hyperspectral image classification: Current trends and challenges10.1016/j.inffus.2020.01.007 ↗
2. Jörges, C., Vidal, H.S., Hank, T., and Bach, H. (2023). Detection of Solar Photovoltaic Power Plants Using Satellite and Airborne Hyperspectral Imaging. Remote Sens., 15.10.3390/rs15133403 ↗
3. Deville, Y., Drumetz, L., Mailhes, C., Hosseini, S., and Tourneret, J.Y. (2023). Modeling and Unsupervised Unmixing Based on Spectral Variability for Hyperspectral Oceanic Remote Sensing Data with Adjacency Effects. Remote Sens., 15.10.3390/rs15184583 ↗
4. Li, C. (2014). A Comparative Study of Component Replacement Methods for High-Resolution Remote Sensing Image Fusion, Water and Soil Conservation Research. [3rd ed.].
5. Luo, L. (2007). Research on Remote Sensing Image Fusion Algorithm Based on Multi-Resolution Analysis. [Ph.D. Thesis, Jilin University].
6. Hou, X.T. (2017). Research on the Application of Variational Methods in Image Fusion. [Ph.D. Thesis, Wuhan University].
7. Book review: Data Fusion, Definitions and Architectures—Fusion of Images of Different Spatial Resolutions10.1080/01431160310001599015 ↗
8. Yuhas, R.H., Goetz, A.F.H., and Boardman, J.W. (1992, January 1–5). Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm. Proceedings of the Third Annual JPL Airborne Geoscience Workshop, Pasadena, CA, USA. Technical Report_ Earth Resources and Remote Sensing.
9. Zheng, Y., Li, J., and Li, Y. (August, January 28). Hyperspectral pansharpening based on guided filter and deep residual learning. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.
10. Spectral-fidelity convolutional neural networks for hyperspectral pansharpening10.1109/JSTARS.2020.3025040 ↗
11. Dong, W., Zhang, T., Qu, J., Xiao, S., Liang, J., and Li, Y. (2022, January 17–22). Laplacian pyramid dense network for hyperspectral pansharpening. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.10.1109/TGRS.2021.3076768 ↗
12. HPGAN: Hyperspectral pansharpening using 3-D generative adversarial networks10.1109/TGRS.2020.2994238 ↗
13. Hyperspectral pansharpening using deep prior and dual attention residual network10.1109/TGRS.2020.2986313 ↗
14. Unsupervised Hyperspectral Pansharpening via Low-Rank Diffusion Model10.1016/j.inffus.2024.102325 ↗
15. MFT-GAN: A Multiscale Feature-Guided Transformer Network for Unsupervised Hyperspectral Pansharpening10.1109/TGRS.2024.3402058 ↗
16. Bandara, W.G.C., and Patel, V.M. (2022, January 19–24). HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00181 ↗
17. PSRT: Pyramid Shuffle-and-Reshuffle Transformer for Multispectral and Hyperspectral Image Fusion10.1109/TGRS.2023.3244750 ↗
18. Wang, Q., Li, Q., and Li, X. (2020). Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution. arXiv.
19. Wu, Y., Huang, M., Li, Y., Feng, S., and Wu, D. (2021). A Distributed Fusion Framework of Multispectral and Panchromatic Images Based on Residual Network. Remote Sens., 13.10.3390/rs13132556 ↗
20. Multistage Dual-Attention Guided Fusion Network for Hyperspectral Pansharpening10.1109/TGRS.2021.3114552 ↗
21. Xie, Q., Zhou, M., Zhao, Q., Meng, D., and Zuo, W. (2019, January 15–20). Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00168 ↗
22. Hyperspectral Image Fusion Using Dense Connected Networks
23. Attention-Based Hyperspectral and Panchromatic Image Fusion Network
24. Hyperspectral Image Fusion Using Generative Adversarial Networks
25. Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest10.1109/JSTARS.2015.2420582 ↗
26. Pansharpening Using Regression of Classified MS and Pan Images to Reduce Color Distortion10.1109/LGRS.2014.2324817 ↗
27. High-Fidelity Fusion Method for Panchromatic and Multispectral Images Based on Ratio Transformation
28. Targ, S., Almeida, D., and Lyman, K. (2016, January 27–30). Resnet in Resnet: Generalizing Residual Architectures. Proceedings of the IEEE International Conference on Computer Vision, Las Vegas, NV, USA.
29. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017). Aggregated Residual Transformations for Deep Neural Networks. arXiv.10.1109/CVPR.2017.634 ↗
30. Klonus, S., and Ehlers, M. (2009, January 6–9). Performance of evaluation methods in image fusion. Proceedings of the 2009 12th International Conference on Information Fusion, Seattle, WA, USA.
31. A universal image quality index10.1109/97.995823 ↗
32. A wavelet transform method to merge Landsat TM and SPOT panchromatic data10.1080/014311698215973 ↗
33. Hyperspectral pansharpening: A review10.1109/MGRS.2015.2440094 ↗