A Bidirectional Cross Spatiotemporal Fusion Network with Spectral Restoration for Remote Sensing Imagery
Source abstract
Abstract
Existing deep learning-based spatiotemporal fusion (STF) methods for remote sensing imagery often focus exclusively on capturing temporal changes or enhancing spatial details while failing to fully leverage spectral information from coarse images. To address these limitations, we propose a Bidirectional Cross Spatiotemporal Fusion Network with Spectral Restoration (BCSR-STF). The network integrates temporal and spatial information using a Bidirectional Cross Fusion (BCF) module and restores spectral fidelity through a Global Spectral Restoration and Feature Enhancement (GSRFE) module, which combines Adaptive Instance Normalization and spatial attention mechanisms. Additionally, a Progressive Spatiotemporal Feature Fusion and Restoration (PSTFR) module employs multi-scale iterative optimization to enhance the interaction between high- and low-level features. Experiments on three datasets demonstrate the superiority of BCSR-STF, achieving significant improvements in capturing seasonal variations and handling abrupt land cover changes compared to state-of-the-art methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Fusion Techniques · Remote-Sensing Image Classification · Infrared Target Detection Methodologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Geophysics and Geomatics, China University of Geosciences, Wuhan 430074, China
Institute of New Energy Equipment, Zhejiang College of Security Technology, Wenzhou 325000, China
Wenzhou Future City Research Institute, Wenzhou 325000, China
Wenzhou Collaborative Innovation Center for Space-Borne, Airborne and Ground Monitoring Situational Awareness Technology, Wenzhou 325000, China
Funding
National Natural Science Foundation of China · Open Fund of Key Laboratory of Space Ocean Remote Sensing and Application, MNR · Fundamental Research Funds for the Central Universities, China University of Geosciences (Wuhan) · Open Fund of Wenzhou Future City Research Institute · Zhejiang Provincial Philosophy and Social Sciences Planning · Major Science and Technology Research Projects in Wenzhou City in 2023 · Global Change and Air-Sea Interaction II · Foundation of State Key Laboratory of Public Big Data · Open Fund of Key Laboratory of Regional Development and Environmental Response · Open Fund of State Key Laboratory of Remote Sensing Science
References (43)
Show source-provided references
1. Zhu, X., Cai, F., Tian, J., and Williams, T. (2018). Spatiotemporal Fusion of Multisource Remote Sensing Data: Literature Survey, Taxonomy, Principles, Applications, and Future Directions. Remote Sens., 10.10.3390/rs10040527 ↗
2. Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review
3. Review of Pixel-Level Remote Sensing Image Fusion Based on Deep Learning10.1016/j.inffus.2022.09.008 ↗
4. High Spatio-Temporal Resolution Evapotranspiration Estimates within Large Agricultural Fields by Fusing Eddy Covariance and Landsat Based Data10.1016/j.agrformet.2023.109417 ↗
5. Assessment of Water Demands for Irrigation Using Energy Balance and Satellite Data Fusion Models in Cloud Computing: A Study in the Brazilian Semiarid Region10.1016/j.agwat.2023.108260 ↗
6. Integration of an Innovative Atmospheric Forecasting Simulator and Remote Sensing Data into a Geographical Information System in the Frame of Agriculture 4.0 Concept10.3390/agriengineering5030081 ↗
7. Spatiotemporal Fusion of Multi-Source Remote Sensing Data for Estimating Aboveground Biomass of Grassland10.1016/j.ecolind.2023.109892 ↗
8. A Comprehensive Flexible Spatiotemporal DAta Fusion Method (CFSDAF) for Generating High Spatiotemporal Resolution Land Surface Temperature in Urban Area10.1109/JSTARS.2022.3220897 ↗
9. Understanding Spatiotemporal Evolution of the Surface Urban Heat Island in the Bangkok Metropolitan Region from 2000 to 2020 Using Enhanced Land Surface Temperature10.1080/19475705.2023.2174904 ↗
10. Unmixing-Based Multisensor Multiresolution Image Fusion10.1109/36.763276 ↗
11. A Flexible Spatiotemporal Method for Fusing Satellite Images with Different Resolutions10.1016/j.rse.2015.11.016 ↗
12. Spectral Unmixing Based Spatiotemporal Downscaling Fusion Approach
13. On the Blending of the Landsat and MODIS Surface Reflectance: Predicting Daily Landsat Surface Reflectance10.1109/TGRS.2006.872081 ↗
14. An Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model for Complex Heterogeneous Regions10.1016/j.rse.2010.05.032 ↗
15. Spatio-Temporal Fusion for Daily Sentinel-2 Images10.1016/j.rse.2017.10.046 ↗
16. Blending Multi-Resolution Satellite Sea Surface Temperature (SST) Products Using Bayesian Maximum Entropy Method10.1016/j.rse.2013.03.021 ↗
17. Liao, L., Song, J., Wang, J., Xiao, Z., and Wang, J. (2016). Bayesian Method for Building Frequent Landsat-Like NDVI Datasets by Integrating MODIS and Landsat NDVI. Remote Sens., 8.10.3390/rs8060452 ↗
18. Spatiotemporal Reflectance Fusion via Sparse Representation10.1109/TGRS.2012.2186638 ↗
19. Spatiotemporal Satellite Image Fusion Through One-Pair Image Learning10.1109/TGRS.2012.2213095 ↗
20. Tan, Z., Yue, P., Di, L., and Tang, J. (2018). Deriving High Spatiotemporal Remote Sensing Images Using Deep Convolutional Network. Remote Sens., 10.10.3390/rs10071066 ↗
21. Tan, Z., Di, L., Zhang, M., Guo, L., and Gao, M. (2019). An Enhanced Deep Convolutional Model for Spatiotemporal Image Fusion. Remote Sens., 11.10.3390/rs11242898 ↗
22. A Flexible Reference-Insensitive Spatiotemporal Fusion Model for Remote Sensing Images Using Conditional Generative Adversarial Network10.1109/TGRS.2021.3050551 ↗
23. MLFF-GAN: A Multilevel Feature Fusion with GAN for Spatiotemporal Remote Sensing Images10.1109/TGRS.2022.3169916 ↗
24. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need 2017. Adv. Neural Inf. Process. Syst., 30.
25. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image Is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv.
26. SwinSTFM: Remote Sensing Spatiotemporal Fusion Using Swin Transformer
27. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV 2021), Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
28. STF-Trans: A Two-Stream Spatiotemporal Fusion Transformer for Very High Resolution Satellites Images10.1016/j.neucom.2023.126868 ↗
29. A CNN-Transformer Combined Remote Sensing Imagery Spatiotemporal Fusion Model10.1109/JSTARS.2024.3435739 ↗
30. StfNet: A Two-Stream Convolutional Neural Network for Spatiotemporal Image Fusion10.1109/TGRS.2019.2907310 ↗
31. A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion10.1109/JSTARS.2022.3143464 ↗
32. Spatiotemporal Remote Sensing Image Fusion Using Multiscale Two-Stream Convolutional Neural Networks
33. DMNet: A Network Architecture Using Dilated Convolution and Multiscale Mechanisms for Spatiotemporal Fusion of Remote Sensing Images10.1109/JSEN.2020.3000249 ↗
34. Spatiotemporal Fusion of Remote Sensing Images Using a Convolutional Neural Network with Attention and Multiscale Mechanisms10.1080/01431161.2020.1809742 ↗
35. Zhang, Y., Liu, J., Liang, S., and Li, M. (2022). A New Spatial–Temporal Depthwise Separable Convolutional Fusion Network for Generating Landsat 8-Day Surface Reflectance Time Series over Forest Regions. Remote Sens., 14.10.3390/rs14092199 ↗
36. PSTAF-GAN: Progressive Spatio-Temporal Attention Fusion Method Based on Generative Adversarial Network
37. Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks10.1109/JSTARS.2018.2797894 ↗
38. Liu, H., Yang, G., Deng, F., Qian, Y., and Fan, Y. (2023). MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images. Remote Sens., 15.10.3390/rs15061583 ↗
39. Jia, D., Song, C., Cheng, C., Shen, S., Ning, L., and Hui, C. (2020). A Novel Deep Learning-Based Spatiotemporal Fusion Method for Combining Satellite Images with Different Resolutions Using a Two-Stream Convolutional Neural Network. Remote Sens., 12.10.3390/rs12040698 ↗
40. Multitask Deep Learning Framework for Spatiotemporal Fusion of NDVI10.1109/TGRS.2021.3140144 ↗
41. CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention10.1109/TPAMI.2023.3341806 ↗
42. Assessing the Accuracy of Blending Landsat–MODIS Surface Reflectances in Two Landscapes with Contrasting Spatial and Temporal Dynamics: A Framework for Algorithm Selection10.1016/j.rse.2013.02.007 ↗
43. Wuhan Dataset: A High-Resolution Dataset of Spatiotemporal Fusion for Remote Sensing Images10.1109/LGRS.2024.3432285 ↗