LightVSR: A Lightweight Video Super-Resolution Model with Multi-Scale Feature Aggregation
Source abstract
Abstract
Video super-resolution aims to generate high-resolution video sequences with realistic details from existing low-resolution video sequences. However, most existing video super-resolution models require substantial computational power and are not suitable for resource-constrained devices such as smartphones and tablets. In this paper, we propose a lightweight video super-resolution (LightVSR) model that employs a novel feature aggregation module to enhance video quality by efficiently reconstructing high-resolution frames from compressed low-resolution inputs. LightVSR integrates several novel mechanisms, including head-tail convolution, cross-layer shortcut connections, and multi-input attention, to enhance computational efficiency while guaranteeing video super-resolution performance. Extensive experiments show that LightVSR achieves a frame rate of 28.57 FPS and a PSNR of 39.25 dB on the UDM10 dataset and 36.91 dB on the Vimeo-90k dataset, validating its efficiency and effectiveness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Advanced Vision and Imaging · Image Processing Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Post-Doctoral Research Station in Instrumentation Science and Technology, Guilin University of Electronic Technology, Guilin 541004, China
School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China
Funding
the Key Research and Development Program of Guangxi · the Guangxi Innovation Driven Development Special Fund Project · the Guangxi Key Laboratory of Trusted Software
References (42)
Show source-provided references
1. Jiang, Y., Nawała, J., Feng, C., Zhang, F., Zhu, X., Sole, J., and Bull, D. (2024). RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content. arXiv.10.1109/PCS60826.2024.10566374 ↗
2. Sun, J., Yuan, Q., Shen, H., Li, J., and Zhang, L. (2024). A Single-Frame and Multi-Frame Cascaded Image Super-Resolution Method. Sensors, 24.10.20944/preprints202408.0253.v2 ↗
3. Ko, H.-k., Park, D., Park, Y., Lee, B., Han, J., and Park, E. (2024). Sequence Matters: Harnessing Video Models in Super-Resolution. arXiv.
4. Li, Y., Yang, X., Liu, W., Jin, X., Jia, X., Lai, Y., Liu, H., Rosin, P.L., and Zhou, W. (2024). TINQ: Temporal Inconsistency Guided Blind Video Quality Assessment. arXiv.
5. Wen, Y., Zhao, Y., Liu, Y., Jia, F., Wang, Y., Luo, C., Zhang, C., Wang, T., Sun, X., and Zhang, X. (2024, January 16–22). Panacea: Panoramic and controllable video generation for autonomous driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00659 ↗
6. Feng, R., Li, C., and Loy, C.C. (2025, January 26–27). Kalman-inspired feature propagation for video face super-resolution. Proceedings of the European Conference on Computer Vision, Paris, France.10.1007/978-3-031-73347-5_12 ↗
7. Wang, Q., Yin, Q., Huang, Z., Jiang, W., Su, Y., Ma, S., and Zhang, J. (2025). Compressed Domain Prior-Guided Video Super-Resolution for Cloud Gaming Content. arXiv.
8. Ranjan, A., and Black, M.J. (2017, January 21–26). Optical Flow Estimation using a Spatial Pyramid Network. Proceedings of the 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.291 ↗
9. Shi, W.Z., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z.H. (2016, January 27–30). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR.2016.207 ↗
10. Cubic convolution interpolation for digital image processing10.1109/TASSP.1981.1163711 ↗
11. Image super-resolution using deep convolutional networks10.1109/TPAMI.2015.2439281 ↗
12. Wang, X., Chan, K.C., Yu, K., Dong, C., and Change Loy, C. (2019, January 16–17). Edvr: Video restoration with enhanced deformable convolutional networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00247 ↗
13. Li, W., Tao, X., Guo, T., Qi, L., Lu, J., and Jia, J. (2020, January 23–28). Mucan: Multi-correspondence aggregation network for video super-resolution. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part X 16.10.1007/978-3-030-58607-2_20 ↗
14. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22–29). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.89 ↗
15. Sajjadi, M.S., Vemulapalli, R., and Brown, M. (2018, January 18–23). Frame-recurrent video super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00693 ↗
16. Chu, M., Xie, Y., Leal-Taixé, L., and Thuerey, N. (2018). Temporally coherent gans for video super-resolution (tecogan). arXiv.
17. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems 27 (NIPS 2014), Montreal, QC, Canada.
18. Chan, K.C., Wang, X., Yu, K., Dong, C., and Loy, C.C. (2021, January 19–25). Basicvsr: The search for essential components in video super-resolution and beyond. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00491 ↗
19. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. (2021, January 10–17). Swinir: Image restoration using swin transformer. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCVW54120.2021.00210 ↗
20. Rota, C., Buzzelli, M., and van de Weijer, J. (2023). Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models. arXiv.10.1007/978-3-031-73254-6_3 ↗
21. Xu, K., Yu, Z., Wang, X., Mi, M.B., and Yao, A. (2024, January 16–22). Enhancing Video Super-Resolution via Implicit Resampling-based Alignment. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00246 ↗
22. Lu, Y., Wang, Z., Liu, M., Wang, H., and Wang, L. (2023, January 17–24). Learning spatial-temporal implicit neural representations for event-guided video super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00156 ↗
23. Yi, P., Wang, Z., Jiang, K., Jiang, J., Lu, T., Tian, X., and Ma, J. (2021, January 10–17). Omniscient video super-resolution. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00439 ↗
24. Jiang, L., Wang, N., Dang, Q., Liu, R., and Lai, B. (2021). PP-MSVSR: Multi-stage video super-resolution. arXiv.
25. Wang, X., Yang, X., Li, H., and Li, T. (2024). FDDCC-VSR: A lightweight video super-resolution network based on deformable 3D convolution and cheap convolution. Vis. Comput., 1–13.10.1007/s00371-024-03621-x ↗
26. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
27. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
28. Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., and Mu Lee, K. (2019, January 16–17). Ntire 2019 challenge on video deblurring and super-resolution: Dataset and study. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00251 ↗
29. Video enhancement with task-oriented flow10.1007/s11263-018-01144-2 ↗
30. Charbonnier, P., Blanc-Feraud, L., Aubert, G., and Barlaud, M. (1994, January 13–16). Two deterministic half-quadratic regularization algorithms for computed imaging. Proceedings of the 1st International Conference on Image Processing, Austin, TX, USA.
31. Caballero, J., Ledig, C., Aitken, A., Acosta, A., Totz, J., Wang, Z., and Shi, W. (2017, January 21–26). Real-time video super-resolution with spatio-temporal networks and motion compensation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.304 ↗
32. Tao, X., Gao, H., Liao, R., Wang, J., and Jia, J. (2017, January 22–29). Detail-revealing deep video super-resolution. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.479 ↗
33. Jo, Y., Oh, S.W., Kang, J., and Kim, S.J. (2018, January 18–23). Deep video super-resolution network using dynamic upsampling filters without explicit motion compensation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00340 ↗
34. Haris, M., Shakhnarovich, G., and Ukita, N. (2019, January 16–17). Recurrent back-projection network for video super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00402 ↗
35. Yi, P., Wang, Z., Jiang, K., Jiang, J., and Ma, J. (November, January 27). Progressive fusion video super-resolution network via exploiting non-local spatio-temporal correlations. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
36. Isobe, T., Li, S., Jia, X., Yuan, S., Slabaugh, G., Xu, C., Li, Y.-L., Wang, S., and Tian, Q. (2020, January 13–19). Video super-resolution with temporal group attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00803 ↗
37. Fuoli, D., Gu, S., and Timofte, R. (2019, January 27–28). Efficient video super-resolution through recurrent latent space propagation. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.10.1109/ICCVW.2019.00431 ↗
38. Isobe, T., Jia, X., Gu, S., Li, S., Wang, S., and Tian, Q. (2020, January 23–28). Video super-resolution with recurrent structure-detail network. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XII 16.
39. Isobe, T., Zhu, F., Jia, X., and Wang, S. (2020). Revisiting temporal modeling for video super-resolution. arXiv.
40. Deformable 3d convolution for video super-resolution10.1109/LSP.2020.3013518 ↗
41. Geng, Z., Liang, L., Ding, T., and Zharkov, I. (2022, January 18–24). Rstt: Real-time spatial temporal transformer for space-time video super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01692 ↗
42. Stdan: Deformable attention network for space-time video super-resolution