Advances in the Neural Network Quantization: A Comprehensive Review
Source abstract
Abstract
Artificial intelligence technologies based on deep convolutional neural networks and large language models have made significant breakthroughs in many tasks, such as image recognition, target detection, semantic segmentation, and natural language processing, but also face a conflict between the high computational capacity of the algorithms and limited deployment resources. Quantization, which converts floating-point neural networks into low-bit-width integer networks, is an important and essential technique for efficient deployment and cost reduction in edge computing. This paper analyzes various existing quantization methods, showcases the deployment accuracy of advanced techniques, and discusses the future challenges and trends in this domain.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Brain Tumor Detection and Classification · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software, Northwestern Polytechnical University, Xi’an 710072, China
Xi’an Microelectronics Technology Institute, Xi’an 710065, China
References (63)
Show source-provided references
1. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., and Dong, L. (2022, January 18–24). Swin transformer v2: Scaling up capacity and resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01170 ↗
2. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
3. Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L.M., and Shum, H.-Y. (2023). DINO: DETR with improved denoising anchor boxes for end-to-end object detection. In International Conference on Learning Representations. arXiv.
4. Zong, Z., Song, G., and Liu, Y. (2023, January 2–6). Detrs with collaborative hybrid assignments training. Proceedings of the International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00621 ↗
5. Unmanned system swarm intelligence and its research progresses
6. Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., and Qiao, Y. (2023). Vision transformer adapter for dense predictions. In International Conference on Learning Representations. arXiv.
7. Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, L., Wang, X., Huang, T., Wang, X., and Cao, Y. (2023, January 17–24). EVA: Exploring the limits of masked visual representation learning at scale. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01855 ↗
8. Su, W., Zhu, X., Tao, C., Lu, L., Li, B., Huang, G., Qiao, Y., Wang, X., Zhou, J., and Dai, J. (2023, January 17–24). Towards all-in-one pre-training via maximizing multi-modal mutual information. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01525 ↗
9. Wang, W., Dai, J., Chen, Z., Huang, Z., Li, Z., Zhu, X., Hu, X., Lu, T., Lu, L., and Li, H. (2023, January 17–24). Internimage: Exploring large-scale vision foundation models with deformable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01385 ↗
10. The present situation and developing trends of space-based intelligent computing technology
11. Benchmark analysis of representative deep neural network architectures10.1109/ACCESS.2018.2877890 ↗
12. Hong, J., Duan, J., Zhang, C., Li, Z., Xie, C., Lieberman, K., Diffenderfer, J., Bartoldson, B., Jaiswal, A., and Xu, K. (2024). Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs under Compression. Computing Research Repository. arXiv.
13. Universally quantized neural compression
14. Banner, R., Nahshan, Y., and Soudry, D. (2018). Post-training 4-bit quantization of convolution networks for rapid-deployment. arXiv.
15. Bulat, A., Martinez, B., and Tzimiropoulos, G. (2021). High-capacity expert binary networks. International Conference on Learning Representations. arXiv.
16. Liang, T., Glossner, J., Wang, L., Shi, S., and Zhang, X. (2021). Pruning and quantization for deep neural network acceleration: A survey. arXiv.10.1016/j.neucom.2021.07.045 ↗
17. Garg, S., Jain, A., Lou, J., and Nahmias, M. (2021). Confounding tradeoffs for neural network quantization. arXiv.
18. Garg, S., Lou, J., Jain, A., Guo, Z., Shastri, B.J., and Nahmias, M. (2021). Dynamic precision analog computing for neural networks. arXiv.
19. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. North American Chapter of the Association for Computational Linguistics. arXiv, 4171–4186.
20. Language Models are Few-Shot Learners
21. Zhu, X., Li, J., Liu, Y., Ma, C., and Wang, W. (2023). A Survey on Model Compression for Large Language Models. CoRR arXiv.
22. MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
23. Xu, Z., and Cristianini, N. (2022). QBERT: Generalist Model for Processing Questions. Advances in Intelligent Data Analysis XXI, Springer.10.1007/978-3-031-30047-9_37 ↗
24. Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. (2023). GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. arXiv.
25. Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., and Zou, Y. (2016). Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv.
26. Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M.W., and Keutzer, K. (2021). A Survey of Quantization Methods for Efficient Neural Network Inference. arXiv.10.1201/9781003162810-13 ↗
27. Nagel, M., Fournarakis, M., Amjad, R.A., Bondarenko, Y., Van Baalen, M., and Blankevoort, T. (2021). A White Paper on Neural Network Quantization. arXiv.
28. Li, Y., Dong, X., and Wang, W. (2020). Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks. arXiv.
29. Post-training quantization for vision transformer
30. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18–22). Quantization and training of neural networks for efficient integerarithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00286 ↗
31. Q-vit: Accurate and fully quantized low-bit vision transformer
32. Yao, Z., Dong, Z., Zheng, Z., Gholami, A., Yu, J., Tan, E., Wang, L., Huang, Q., Wang, Y., and Mahoney, M. (2020). Hawqv3: Dyadic neural network quantization. arXiv.
33. McKinstry, J.L., Esser, S.K., Appuswamy, R., Bablani, D., Arthur, J.V., Yildiz, I.B., and Modha, D.S. (2018). Discovering low-precision networks close to full-precision networks for efficient embedded inference. arXiv.10.1109/EMC2-NIPS53020.2019.00009 ↗
34. Quantizing deep convolutional net-works for efficient inference: A whitepaper
35. Wu, H., Judd, P., Zhang, X., Isaev, M., and Micikevicius, P. (2020). Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv.
36. Migacz, S. (2017, May 08). 8-Bit Inference with TensorRT. GPU Technology Conference 2, 7. Available online: https://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf.
37. Chen, T., Moreau, T., Jiang, Z., Zheng, L., and Yan, E. (2018, January 8–10). TVM: An automated end-to-end optimizing compiler for deep learning. Proceedings of the 13th fUSENIXg Symposium on Operating Systems Design and Implementation (fOSDIg 18), Carlsbad, CA, USA.
38. Choukroun, Y., Kravchik, E., Yang, F., and Kisilev, P. (2019, January 27–28). Low-bit quantization of neural networks for efficient inference. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.10.1109/ICCVW.2019.00363 ↗
39. Shin, S., Hwang, K., and Sung, W. (2016, January 20–25). Fixed-point performance analysis of recurrent neural networks. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7471821 ↗
40. Sung, W., Shin, S., and Hwang, K. (2015). Resiliency of deep neural networks under quantization. arXiv.
41. Zhao, R., Hu, Y.W., and Dotzel, J. (2019). Improving neural network quantization without retraining using outlier channel splitting. arXiv.
42. Park, E., Ahn, J., and Yoo, S. (2017, January 26). Weighted-entropy-based quantization for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.761 ↗
43. Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks10.32604/cmes.2023.027085 ↗
44. Choi, J., Wang, Z., Venkataramani, S., Chuang, P.I.-J., Srinivasan, V., and Gopalakrishnan, K. (2018). Pact: Parameterized clipping activation for quantized neural networks. arXiv.
45. Gong, R., Liu, X., Jiang, S., Li, T., Hu, P., Lin, J., Yu, F., and Yan, J. (November, January 27). Differentiable soft quantization: Bridging full-precision and low-bit neural networks. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
46. Jin, Q., Yang, L., and Liao, Z. (2019). Towards efficient training for neural network quantization. arXiv.
47. Liu, J., Cai, J., and Zhuang, B. (2021). Sharpness-aware Quantization for Deep Neural Networks. arXiv.
48. Zhuang, B., Liu, L., Tan, M., Shen, C., and Reid, I. (2020, January 13–19). Training Quantized Neural Networks With a Full-Precision Auxiliary Module. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00156 ↗
49. Attention Round for post-training quantization10.1016/j.neucom.2023.127012 ↗
50. Wu, B., Wang, Y., Zhang, P., Tian, Y., Vajda, P., and Keutzer, K. (2018). Mixed precision quantization of convnets via differentiable neural architecture search. arXiv.
51. Wang, K., Liu, Z., Lin, Y., Lin, J., and Han, S. (2019, January 15–20). Haq: Hardware-aware automated quantization with mixed precision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00881 ↗
52. Dong, Z., Yao, Z., Gholami, A., Mahoney, M., and Keutzer, K. (November, January 27). Hawq: Hessian aware quantization of neural networks with mixed-precision. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
53. Dong, P., Li, L., Wei, Z., Niu, X., Tian, Z., and Pan, H. (2023, January 2–6). Emq: Evolving training-free proxies for automated mixed precision quantization. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01566 ↗
54. Tang, C., Ouyang, K., Chai, Z., Bai, Y., Meng, Y., Wang, Z., and Zhu, W. (November, January 29). SEAM: Searching Transferable Mixed-Precision Quantization Policy through Large Margin Regularization. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
55. Dong, Z., Yao, Z., Cai, Y., Arfeen, D., Gholami, A., Mahoney, M.W., and Keutzer, K. (2020). Hawq-v2: Hessian aware trace-weighted quantization of neural networks. Advances in Neural Information Processing Systems, JMLR.10.1109/ICCV.2019.00038 ↗
56. Tang, C., Ouyang, K., Wang, Z., Zhu, Y., Ji, W., Wang, Y., and Zhu, W. (2022, January 23–27). Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance. Proceedings of the Computer Vision—ECCV 2022, Tel Aviv, Israel.10.1007/978-3-031-20083-0_16 ↗
57. Sheng, T., Feng, C., Zhuo, S., Zhang, X., Shen, L., and Aleksic, M. (2018, January 25). A quantization-friendly separable convolution for mobilenets. Proceedings of the 2018 1st Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2), Williamsburg, VA, USA.10.1109/EMC2.2018.00011 ↗
58. Quantization of 8-bit deep neural networks based on mean square error
59. Wang, T., Wang, K., Cai, H., Lin, J., Liu, Z., Wang, H., Lin, Y., and Han, S. (2020, January 13–19). APQ: Joint Search for Network Architecture, Pruning and Quantization Policy. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00215 ↗
60. Wang, P., Yang, A., Men, R., Lin, J., Bai, S., and Li, Z. (2022, January 17–23). Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA.
61. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S.V.N., and Garnett, R. (2017). Attention is all you need. Advances in Neural Information Processing Systems 30, Curran Associates, Inc.
62. Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). Qlora: Efficient finetuning of quantized llms. arXiv.
63. Outlier suppression: Pushing the limit of low-bit transformer language models