Efficient Fine-Tuning of Large Language Models via a Low-Rank Gradient Estimator
Source abstract
Abstract
In this paper, we present a Low-Rank Gradient Estimator (LoGE) to accelerate the finetune-time computation of transformers, especially large language models (LLMs). Unlike Parameter-Efficient Fine-Tuning (PEFT) methods, which primarily aim to minimize the number of fine-tuning parameters, LoGE also significantly reduces the computational load of activation gradient calculations by decomposing pre-trained weights and utilizing low-rank matrices during the backward pass. Our approach includes an effective solution for identifying sensitive and important latent subspaces in large models before training with downstream datasets. As LoGE does not alter the network structure, it can be conveniently integrated into existing models. We validated LoGE’s efficacy through comprehensive experiments across various models on various tasks. For the widely used LLaMA model equipped with LoRA, LoGE achieves up to a 1.3× speedup while maintaining graceful accuracy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Tensor decomposition and applications · Speech Recognition and Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
The College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, China
Hangzhou Yihe Hui Sheng, Room 1006-2, 10th Floor, Building 1, No. 180 Kecheng Street, Hangzhou 310000, China
Funding
National Key Research and Development Program of China · Experimental Technology Reaseach Project of Zhejiang University
References (48)
Show source-provided references
1. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
2. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
3. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv.
4. Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.d.l., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv.
5. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv.
6. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv.
7. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y.T., Li, Y., and Lundberg, S. (2023). Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv.
8. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., and Metzler, D. (2022). Emergent abilities of large language models. arXiv.
9. Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019, January 9–15). Parameter-efficient transfer learning for NLP. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA.
10. Guo, D., Rush, A.M., and Kim, Y. (2020). Parameter-efficient transfer learning with diff pruning. arXiv.10.18653/v1/2021.acl-long.378 ↗
11. Pfeiffer, J., Kamath, A., Rücklé, A., Cho, K., and Gurevych, I. (2020). AdapterFusion: Non-destructive task composition for transfer learning. arXiv.10.18653/v1/2021.eacl-main.39 ↗
12. Zaken, E.B., Ravfogel, S., and Goldberg, Y. (2021). Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv.
13. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022, January 25–29). LoRA: Low-Rank Adaptation of Large Language Models. Proceedings of the International Conference on Learning Representations, Virtual Event.
14. Zong, C., Xia, F., Li, W., and Navigli, R. (2021, January 1–6). Prefix-Tuning: Optimizing Continuous Prompts for Generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event.
15. Qin, G., and Eisner, J. (2021). Learning how to ask: Querying LMs with mixtures of soft prompts. arXiv.10.18653/v1/2021.naacl-main.410 ↗
16. Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., and Zhao, T. (2023, January 1–5). Adaptive budget allocation for parameter-efficient fine-tuning. Proceedings of the The Eleventh International Conference on Learning Representations, Kigali, Rwanda.
17. Zhang, L., Zhang, L., Shi, S., Chu, X., and Li, B. (2023). Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning. arXiv.10.18653/v1/2024.findings-emnlp.994 ↗
18. Xu, R., Luo, F., Zhang, Z., Tan, C., Chang, B., Huang, S., and Huang, F. (2021). Raise a child in large language model: Towards effective and generalizable fine-tuning. arXiv.10.18653/v1/2021.emnlp-main.749 ↗
19. He, H., Cai, J., Zhang, J., Tao, D., and Zhuang, B. (2023, January 1–6). Sensitivity-aware visual parameter-efficient fine-tuning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01086 ↗
20. Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., and Dong, Z. (2023). A survey of large language models. arXiv.
21. Zhou, A., Wang, K., Lu, Z., Shi, W., Luo, S., Qin, Z., Lu, S., Jia, A., Song, L., and Zhan, M. (2023). Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification. arXiv.
22. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv.
23. Qlora: Efficient finetuning of quantized llms
24. Xu, Y., Xie, L., Gu, X., Chen, X., Chang, H., Zhang, H., Chen, Z., Zhang, X., and Tian, Q. (2023). Qa-lora: Quantization-aware low-rank adaptation of large language models. arXiv.
25. He, Y., Liu, J., Wu, W., Zhou, H., and Zhuang, B. (2023). Efficientdm: Efficient quantization-aware fine-tuning of low-bit diffusion models. arXiv.
26. Li, Y., Liang, Y., and Risteski, A. (2016, January 19–24). Recovery guarantee of weighted low-rank approximation via alternating minimization. Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA.
27. A singular value thresholding algorithm for matrix completion10.1137/080738970 ↗
28. Li, Y., Ma, T., and Zhang, H. (2018, January 6–9). Algorithmic regularization in over-parameterized matrix sensing and neural networks with quadratic activations. Proceedings of the Conference On Learning Theory, PMLR, Stockholm, Sweden.
29. A literature survey of low-rank tensor approximation techniques10.1002/gamm.201310004 ↗
30. Oymak, S., Fabian, Z., Li, M., and Soltanolkotabi, M. (2019). Generalization guarantees for neural networks via harnessing the low-rank structure of the jacobian. arXiv.
31. Lialin, V., Muckatira, S., Shivagunde, N., and Rumshisky, A. (2023, January 10–16). ReLoRA: High-Rank Training Through Low-Rank Updates. Proceedings of the Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@ NeurIPS 2023), New Orleans, LA, USA.
32. Zhao, J., Zhang, Z., Chen, B., Wang, Z., Anandkumar, A., and Tian, Y. (2024). Galore: Memory-efficient llm training by gradient low-rank projection. arXiv.
33. Frantar, E., and Alistarh, D. (2023, January 23–29). Sparsegpt: Massive language models can be accurately pruned in one-shot. Proceedings of the International Conference on Machine Learning, PMLR, Honolulu, HI, USA.
34. Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., and Lin, X.V. (2022). Opt: Open pre-trained transformer language models. arXiv.
35. Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N.A., Khashabi, D., and Hajishirzi, H. (2022). Self-instruct: Aligning language model with self generated instructions. arXiv.10.18653/v1/2023.acl-long.754 ↗
36. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T.B. (2023, March 13). Stanford Alpaca: An Instruction-Following LLaMA Model. Available online: https://github.com/tatsu-lab/stanford_alpaca.
37. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. (2021, January 4). Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations, Vienna, Austria.
38. Openassistant conversations-democratizing large language model alignment
39. Yu, L., Jiang, W., Shi, H., Yu, J., Liu, Z., Zhang, Y., Kwok, J.T., Li, Z., Weller, A., and Liu, W. (2023). Metamath: Bootstrap your own mathematical questions for large language models. arXiv.
40. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., and Nakano, R. (2021). Training Verifiers to Solve Math Word Problems. arXiv.
41. Zheng, T., Zhang, G., Shen, T., Liu, X., Lin, B.Y., Fu, J., Chen, W., and Yue, X. (2024). OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement. arXiv.10.18653/v1/2024.findings-acl.762 ↗
42. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H.P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., and Brockman, G. (2021). Evaluating Large Language Models Trained on Code. arXiv, Available online: http://arxiv.org/abs/2107.03374.
43. Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., Lin, Q., and Jiang, D. (2024, January 7–11). WizardLM: Empowering large pre-trained language models to follow complex instructions. Proceedings of the The Twelfth International Conference on Learning Representations, Vienna, Austria.
44. Judging llm-as-a-judge with mt-bench and chatbot arena
45. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.R. (2019, January 6–9). GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA.
46. Kalajdzievski, D. (2023). A rank stabilization scaling factor for fine-tuning with lora. arXiv.
47. Meng, F., Wang, Z., and Zhang, M. (2024). Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv.
48. McCarley, J., Chakravarti, R., and Sil, A. (2019). Structured pruning of a bert-based question answering model. arXiv.