Improving Variable-Rate Learned Image Compression with Transformer-Based QR Prediction and Perceptual Optimization
Source abstract
Abstract
We present a variable-rate learned image compression (LIC) model that integrates Transformer-based quantization–reconstruction (QR) offset prediction, entropy-guided hyper-latent quantization, and perceptually informed multi-objective optimization. Unlike existing LIC frameworks that train separate networks for each bitrate, the proposed method achieves continuous rate adaptation within a single model by dynamically balancing rate, distortion and perceptual objectives. Channel-wise asymmetric quantization and a composite loss combining MSE and LPIPS further enhance reconstruction fidelity and subjective quality. Experiments on the Kodak, CLIC2020 and Tecnick datasets show gains of +1.15 dB PSNR, +0.065 MS-SSIM, and −0.32 LPIPS relative to the baselines variable-rate method, while improving bitrate-control accuracy by 62.5%. With approximately 15% computational overhead, the framework achieves competitive compression efficiency and enhanced perceptual quality, offering a practical solution for adaptive, high-quality image delivery.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Data Compression Techniques · Image and Video Quality Assessment · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Game Content, Wonkwang University, 460 Iksan-daero, Iksan City 5453, Jeonbuk-do, Republic of Korea
Department of Computer Software Engineering, Wonkwang University, 460 Iksan-daero, Iksan City 5453, Jeonbuk-do, Republic of Korea
Funding
Wonkwang University
References (42)
Show source-provided references
1. Deb, S.K., and Pan, W.D. (2024). Quantum Image Compression: Fundamentals, Algorithms, and Advances. Computers, 13.10.3390/computers13080185 ↗
2. The JPEG Still Picture Compression Standard10.1109/30.125072 ↗
3. Overview of the High Efficiency Video Coding (HEVC) Standard10.1109/TCSVT.2012.2221191 ↗
4. A Technical Overview of AV110.1109/JPROC.2021.3058584 ↗
5. Overview of the Versatile Video Coding (VVC) Standard and Its Applications10.1109/TCSVT.2021.3101953 ↗
6. Image and Video Compression with Neural Networks: A Review10.1109/TCSVT.2019.2910119 ↗
7. Learning End-to-End Lossy Image Compression: A Benchmark
8. Huang, C.-H., and Wu, J.-L. (2024). Unveiling the Future of Human and Machine Coding: A Survey of End-to-End Learned Image Compression. Entropy, 26.10.20944/preprints202403.1272.v1 ↗
9. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
10. Nonlinear Transform Coding10.1109/JSTSP.2020.3034501 ↗
11. Zhang, Y., Duan, Z., Huang, Y., and Zhu, F. (2025, January 11–15). Balanced Rate-Distortion Optimization in Learned Image Compression. Proceedings of the Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR52734.2025.00232 ↗
12. Learned Image Compression with Separate Hyperprior Decoders10.1109/OJCAS.2021.3125354 ↗
13. Minnen, D., Ballé, J., and Toderici, G. (2018, January 2). Joint Autoregressive and Hierarchical Priors for Learned Image Compression. Proceedings of the International Conference on Neural Information Processing Systems, Montréal, QC, Canada.
14. Learning-Driven Lossy Image Compression; A Comprehensive Survey10.1016/j.engappai.2023.106361 ↗
15. Theis, L., Shi, W., Cunningham, A., and Huszár, F. (2017, January 24–26). Lossy Image Compression with Compressive Autoencoders. Proceedings of the International Conference on Learning Representations, Toulon, France.
16. Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., and Van Gool, L. (November, January 27). Generative Adversarial Networks for Extreme Learned Image Compression. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
17. Yang, Y., and Mandt, S. (2021, January 3–7). Towards Empirical Sandwich Bounds on the Rate-Distortion Function. Proceedings of the International Conference on Learning Representations, Virtual.
18. Cui, Z., Wang, J., Gao, S., Guo, T., Feng, Y., and Bai, B. (June, January 20). Asymmetric Gained Deep Image Compression with Continuous Rate Adaptation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.
19. Kamisli, F., Racapé, F., and Choi, H. (2024, January 19–22). Variable-Rate Learned Image Compression with Multi-Objective Optimization and Quantization-Reconstruction Offsets. Proceedings of the Data Compression Conference, Snowbird, UT, USA.10.1109/DCC58796.2024.00027 ↗
20. He, D., Yang, Z., Peng, W., Ma, R., Qin, H., and Wang, Y. (2022, January 18–24). ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00563 ↗
21. Minnen, D., and Singh, S. (2020, January 25–28). Channel-wise Autoregressive Entropy Models for Learned Image Compression. Proceedings of the IEEE International Conference on Image Processing, Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9190935 ↗
22. Causal Contextual Prediction for Learned Image Compression10.1109/TCSVT.2021.3089491 ↗
23. Reversible Data Hiding in JPEG Images with Multi-Objective Optimization10.1109/TCSVT.2020.2969463 ↗
24. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–22). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Proceedings of the Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
25. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the In Advances in Neural Information Processing Systems, Long Beach, CA USA.
26. End-to-End Learning-Based Image Compression with a Decoupled Framework10.1109/TCSVT.2023.3313974 ↗
27. Peng, W.-H., and Hang, H.-M. (2020, January 1–4). Recent Advances in End-to-End Learned Image and Video Compression. Proceedings of the IEEE International Conference on Visual Communications and Image Processing, Macau, China.10.1109/VCIP49819.2020.9301753 ↗
28. Cheng, Z., Sun, H., Takeuchi, M., and Katto, J. (2020, January 13–19). Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR42600.2020.00796 ↗
29. Liu, J., Sun, H., and Katto, J. (2023, January 17–24). Learned Image Compression with Mixed Transformer-CNN Architectures. Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01383 ↗
30. Zhu, Y., Yang, Y., and Cohen, T. (2022, January 25–29). Transformer-based Transform Coding. Proceedings of the International Conference on Learning Representations, Virtual.
31. Ballé, J., Minnen, D., Singh, S., Hwang, S.J., and Johnston, N. (May, January 30). Variational Image Compression with a Scale Hyperprior. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
32. Lu, M., Guo, P., Shi, H., Cao, C., and Ma, Z. (2022, January 22–25). Transformer-based Image Compression. Proceedings of the Data Compression Conference, Snowbird, UT, USA.10.1109/DCC52660.2022.00080 ↗
33. Guo, T., Wang, J., Cui, Z., Feng, Y., Ge, Y., and Bai, B. (2020, January 13–19). Variable Rate Image Compression with Content Adaptive Optimization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW50498.2020.00069 ↗
34. Koyuncu, A.B., Gao, H., Boev, A., Gaikov, G., Alshina, E., and Steinbach, E. (2022, January 23). Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19800-7_26 ↗
35. Learnable Non-Uniform Quantization with Sampling-Based Optimization for Variable-Rate Learned Image Compression10.1109/TCSVT.2025.3546765 ↗
36. iWave: CNN-Based Wavelet-Like Transform for Image Compression10.1109/TMM.2019.2957990 ↗
37. Bengio, Y., Léonard, N., and Courville, A. (2013). Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv.
38. Asuni, N., and Giachetti, A. (2014, January 22–23). TESTIMAGES: A Large-Scale Archive for Testing Visual Devices and Basic Image Processing Algorithms. Proceedings of the STAG: Smart Tools and Apps for Graphics, Genova, Italy.
39. Image Quality Assessment: Unifying Structure and Texture Similarity
40. Bjøntegaard, G. (2001, January 2–4). Calculation of Average PSNR Differences between RD-curves. Proceedings of the ITU-T VCEG-M33 2001, 13th VCEG Meeting, Austin, TX, USA.
41. Bégaint, J., Racapé, F., Feltman, S., and Pushparaja, A. (2020). CompressAI: A PyTorch library and evaluation platform for end-to-end compression research. arXiv.
42. Video Enhancement with Task-Oriented Flow10.1007/s11263-018-01144-2 ↗