Video Super-Resolution Combining Dual Motion Compensation and Multi-Scale Structure–Texture Prior
Source abstract
Abstract
Video super-resolution methods based on convolutional kernels or optical flow often face challenges such as limited utilization of multi-frame detail information or strong reliance on accurate optical flow estimation. To address these issues, this paper proposes a novel super-resolution reconstruction network named Dual Motion Compensation and Multi-scale Structure–Texture Prior (DCST-Net). Dual motion compensation performs direct and progressive motion mapping in parallel, effectively mitigating estimation bias in motion modeling. A multi-scale structure–texture prior is introduced to enhance high-frequency details through feature fusion, alleviating over-smoothing caused by warping and fusion processes. The proposed DCST-Net method is validated on datasets containing both large and small targets, demonstrating its effectiveness and robustness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Advanced Vision and Imaging · Image and Video Quality Assessment
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
China Academy of Space Technology, Beijing 100094, China
College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China
Key Laboratory of Space Photoelectric Detection and Sensing of Industry and Information Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China
References (26)
Show source-provided references
1. Swinir transformer applied for medical image super-resolution10.1016/j.procs.2022.08.110 ↗
2. DOVE: Doodled vessel enhancement for photoacoustic angiography super resolution10.1016/j.media.2024.103106 ↗
3. Nonpairwise-Trained Cycle Convolutional Neural Network for Single Remote Sensing Image Super-Resolution10.1109/TGRS.2020.3009224 ↗
4. Single-frame super-resolution in remote sensing: A practical overview10.1080/01431161.2016.1264027 ↗
5. Flow-based super-resolution reconstruction of remote sensing images
6. Super-resolution reconstruction of infrared image based on channel attention and transfer learning
7. Wang, X., Chan, K.C.K., Yu, K., Dong, C., and Loy, C.C. (2019, January 16–17). EDVR: Video restoration with enhanced deformable convolutional networks. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.10.1109/CVPRW.2019.00247 ↗
8. Haris, M., Shakhnarovich, G., and Ukita, N. (2019, January 15–20). Recurrent back-projection network for video super-resolution. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.10.1109/CVPR.2019.00402 ↗
9. Chan, K.C.K., Wang, X., Yu, K., Dong, C., and Loy, C.C. (2021, January 20–25). BasicVSR: The search for essential components in video super-resolution and beyond. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00491 ↗
10. Ranjan, A., and Black, M.J. (2017, January 21–26). Optical flow estimation using a spatial pyramid network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.291 ↗
11. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the 18th International Conference on Medical Image Computing and Computer Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
12. He, K.M., Zhang, X.Y., Ren, S.Q., and Sun, J. (2015, January 7–13). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.123 ↗
13. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27–30). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
14. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA.
15. He, K., Zhang, X.Y., Ren, S.Q., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
16. Liu, Z., Mao, H.Z., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S.N. (2022, January 18–24). A convnet for the 2020s. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
17. Video enhancement with task-oriented flow10.1007/s11263-018-01144-2 ↗
18. Tao, X., Gao, H.Y., Liao, R.J., Wang, J., and Jia, J. (2017, January 22–29). Detail-revealing Deep Video Super-resolution. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.479 ↗
19. On Bayesian Adaptive Video Super Resolution10.1109/TPAMI.2013.127 ↗
20. A method for stochastic optimization
21. Two deterministic half quadratic regularization algorithms for computed imaging
22. Johnson, J., Alahi, A., and Li, F.F. (2016, January 11–14). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. Proceedings of the 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46475-6_43 ↗
23. Zhang, R., Isola, P., and Efros, A.A. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
24. Jo, Y., Oh, S.W., Kang, J., and Kim, S.J. (2018, January 18–23). Deep video super-resolution network using dynamic upsampling filters without explicit motion compensation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00340 ↗
25. Xu, K., Yu, Z.W., Wang, X., Mi, M.B., and Yao, A. (2024, January 16–22). Enhancing Video Super-Resolution via Implicit Resampling-based Alignment. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00246 ↗
26. Shang, W., Ren, D.W., Zhang, W.Y., Fang, Y., Zuo, W., and Ma, K. (October, January 29). Arbitrary-Scale Video Super-Resolution with Structural and Textural Priors. Proceedings of the 18th European Conference, Milan, Italy.