Synergizing Intelligence and Privacy: A Review of Integrating Internet of Things, Large Language Models, and Federated Learning in Advanced Networked Systems
Source abstract
Abstract
Bringing together the Internet of Things (IoT), LLMs, and Federated Learning (FL) offers exciting possibilities, creating a synergy to build smarter, privacy-preserving distributed systems. This review explores the merging of these technologies, particularly within edge computing environments. We examine current architectures and practical methods enabling this fusion, such as efficient low-rank adaptation (LoRA) for fine-tuning large models and memory-efficient Split Federated Learning (SFL) for collaborative edge training. However, this integration faces significant hurdles: the resource limitations of IoT devices, unreliable network communication, data heterogeneity, diverse security threats, fairness considerations, and regulatory demands. While other surveys cover pairwise combinations, this review distinctively analyzes the three-way synergy, highlighting how IoT, LLMs, and FL working in concert unlock capabilities unattainable otherwise. Our analysis compares various strategies proposed to tackle these issues (e.g., federated vs. centralized, SFL vs. standard FL, DP vs. cryptographic privacy), outlining their practical trade-offs. We showcase real-world progress and potential applications in domains like Industrial IoT and smart cities, considering both opportunities and limitations. Finally, this review identifies critical open questions and promising future research paths, including ultra-lightweight models, robust algorithms for heterogeneity, machine unlearning, standardized benchmarks, novel FL paradigms, and next-generation security. Addressing these areas is essential for responsibly harnessing this powerful technological blend.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Privacy-Preserving Technologies in Data · Cryptography and Data Security · Privacy, Security, and Data Protection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical and Data Engineering (SEDE), University of Technology Sydney (UTS), Sydney, NSW 2007, Australia
Data61, Commonwealth Scientific and Industrial Research Organization, Marsfield, Sydney, NSW 2122, Australia
Global Big Data Technologies Centre (GBDTC), University of Technology Sydney (UTS), Sydney, NSW 2007, Australia
References (100+)
Show source-provided references
1. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., and Lin, H. Language Models are Few-Shot Learners. Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS 2020).
2. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv.
3. Chen, X., Wu, W., Li, Z., Li, L., and Ji, F. (2025). LLM-Empowered IoT for 6G Networks: Architecture, Challenges, and Solutions. arXiv.
4. Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling Laws for Neural Language Models. arXiv.
5. Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.S., Cheng, M., Glaese, M., Balle, B., and Kasirzadeh, A. (2021). Ethical and social risks of harm from Language Models. arXiv.
6. A survey on mobile edge computing: The communication perspective10.1109/COMST.2017.2745201 ↗
7. The Internet of Things under Federated Learning: A Review of the Latest Advances and Applications
8. Villalobos, P., Sevilla, J., Heim, L., Besiroglu, T., Hobbhahn, M., and Ho, A. (2022). Will We Run Out of Data? An Analysis of the Limits of Scaling Datasets in Machine Learning. arXiv.
9. McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B.A. (2017, January 20–22). Communication-efficient learning of deep networks from decentralized data. Proceedings of the Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA. PMLR.
10. Cheng, Y., Zhang, W., Zhang, Z., Zhang, C., Wang, S., and Mao, S. (2024). Towards Federated Large Language Models: Motivations, Methods, and Future Directions. IEEE Commun. Surv. Tutorials, 1.10.1109/COMST.2024.3503680 ↗
11. Leverage Variational Graph Representation for Model Poisoning on Federated Learning10.1109/TNNLS.2024.3394252 ↗
12. Wevolver (2025, April 22). Chapter 5: The Future of Edge AI. Available online: https://www.wevolver.com/article/2025-edge-ai-technology-report/the-future-of-edge-ai.
13. Qu, Y., Ding, M., Sun, N., Thilakarathna, K., Zhu, T., and Niyato, D. (2024). The frontier of data erasure: Machine unlearning for large language models. arXiv.10.1109/MC.2024.3405397 ↗
14. Federated Learning For IoT: Applications, Trends, Taxonomy, Challenges, Current Solutions, and Future Directions10.1109/OJCOMS.2024.3506214 ↗
15. Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness10.1109/OJCOMS.2024.3456549 ↗
16. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv.
17. Lin, Z., Hu, X., Zhang, Y., Chen, Z., Fang, Z., Chen, X., Li, A., Vepakomma, P., and Gao, Y. (2024). Splitlora: A split parameter-efficient fine-tuning framework for large language models. arXiv.
18. Split learning over wireless networks: Parallel design and resource management10.1109/JSAC.2023.3242704 ↗
19. Chen, H.Y., Tu, C.H., Li, Z., Shen, H.W., and Chao, W.L. (2022). On the importance and applicability of pre-training for federated learning. arXiv.
20. Federated learning for internet of things: Recent advances, taxonomy, and open challenges10.1109/COMST.2021.3090430 ↗
21. Federated learning for internet of things: A comprehensive survey10.1109/COMST.2021.3075439 ↗
22. Liu, M., Ho, S., Wang, M., Gao, L., Jin, Y., and Zhang, H. (2021). Federated learning meets natural language processing: A survey. arXiv.
23. Zhuang, W., Chen, C., and Lyu, L. (2023). When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv.
24. Federated learning for computationally constrained heterogeneous devices: A survey10.1145/3596907 ↗
25. Unleashing the power of edge-cloud generative AI in mobile networks: A survey of AIGC services10.1109/COMST.2024.3353265 ↗
26. Enabling massive IoT toward 6G: A comprehensive survey10.1109/JIOT.2021.3063686 ↗
27. The industrial internet of things (IIoT): An analysis framework10.1016/j.compind.2018.04.015 ↗
28. Empowering things with intelligence: A survey of the progress, challenges, and opportunities in artificial intelligence of things10.1109/JIOT.2020.3039359 ↗
29. Stusek, M., Zeman, K., Masek, P., Sedova, J., and Hosek, J. (2019, January 28–30). IoT protocols for low-power massive IoT: A communication perspective. Proceedings of the 2019 11th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Dublin, Ireland.10.1109/ICUMT48472.2019.8970868 ↗
30. Artificial intelligence in Internet of things10.1049/trit.2018.1008 ↗
31. Context-sensitive access in industrial internet of things (IIoT) healthcare applications10.1109/TII.2018.2808190 ↗
32. Communal acts of IoT consumers: A potential threat to security and privacy10.1109/TCE.2018.2880338 ↗
33. Gong, X. (2020, January 6–9). Delay-optimal distributed edge computing in wireless edge networks. Proceedings of the IEEE INFOCOM 2020—IEEE Conference on Computer Communications, Toronto, ON, Canada.10.1109/INFOCOM41043.2020.9155272 ↗
34. Ashish, V. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17), Long Beach, CA, USA.
35. Lee, J., and Toutanova, K. (2018). Pre-training of deep bidirectional transformers for language understanding. arXiv.
36. Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. Training Language Models to Follow Instructions with Human Feedback. Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS 2022).
37. Carlini, N., Nasr, M., Choquette-Choo, C.A., Jagielski, M., Gao, I., Koh, P.W.W., Ippolito, D., Tramer, F., and Schmidt, L. (2023). Are aligned neural networks adversarially aligned?. arXiv.
38. Wu, N., Yuan, X., Wang, S., Hu, H., and Xue, M. (2024, January 13–17). Cardinality Counting in “Alcatraz”: A Privacy-aware Federated Learning Approach. Proceedings of the ACM Web Conference 2024, Singapore.10.1145/3589334.3645655 ↗
39. OFDMA-F²L: Federated Learning With Flexible Aggregation Over an OFDMA Air Interface10.1109/TWC.2023.3334691 ↗
40. FL-IDS: Federated learning-based intrusion detection system using edge devices for transportation IoT10.1109/ACCESS.2024.3386631 ↗
41. Federated vehicular transformers and their federations: Privacy-preserving computing and cooperation for autonomous driving10.1109/TIV.2022.3197815 ↗
42. Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges10.1109/COMST.2023.3315746 ↗
43. Decentral and incentivized federated learning frameworks: A systematic literature review10.1109/JIOT.2022.3231363 ↗
44. Federated learning: Challenges, methods, and future directions
45. Dwork, C., McSherry, F., Nissim, K., and Smith, A. (2006, January 4–7). Calibrating noise to sensitivity in private data analysis. Proceedings of the Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA. Proceedings 3.10.1007/11681878_14 ↗
46. Preserving the privacy of latent information for graph-structured data10.1109/TIFS.2023.3302508 ↗
47. Ragab, M., Ashary, E.B., Alghamdi, B.M., Aboalela, R., Alsaadi, N., Maghrabi, L.A., and Allehaibi, K.H. (2025). Advanced artificial intelligence with federated learning framework for privacy-preserving cyberthreat detection in IoT-assisted sustainable smart cities. Sci. Rep., 15.10.1038/s41598-025-88843-2 ↗
48. Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., and Zhang, L. (2016, January 24–28). Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.10.1145/2976749.2978318 ↗
49. Ahmadi, K., Behnia, R., Ebrahimi, R., Kermani, M.M., Birrell, J., Pacheco, J., and Yavuz, A.A. (2025). An Interactive Framework for Implementing Privacy-Preserving Federated Learning: Experiments on Large Language Models. arXiv.
50. Basu, P., Roy, T.S., Naidu, R., Muftuoglu, Z., Singh, S., and Mireshghallah, F. (2021). Benchmarking differential privacy and federated learning for bert models. arXiv.
51. Hu, S., Yuan, X., Ni, W., Wang, X., Hossain, E., and Vincent Poor, H. (2025). Differentially Private Wireless Federated Learning With Integrated Sensing and Communication. IEEE Trans. Wirel. Commun., 1.10.1109/TWC.2025.3555212 ↗
52. Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning10.1109/TIFS.2023.3258255 ↗
53. Paillier, P. (1999). Public-key cryptosystems based on composite degree residuosity classes. Advances in Cryptology—EUROCRYPT ’99, Springer.10.1007/3-540-48910-X_16 ↗
54. How to share a secret10.1145/359168.359176 ↗
55. Yao, A.C. (1982, January 3–5). Protocols for secure computations. Proceedings of the 23rd Annual Symposium on Foundations of Computer Science (SFCS 1982), Chicago, IL, USA.10.1109/SFCS.1982.38 ↗
56. Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H.B., Patel, S., Ramage, D., Segal, A., and Seth, K. (November, January 30). Practical secure aggregation for privacy-preserving machine learning. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA.10.1145/3133956.3133982 ↗
57. Che, T., Liu, J., Zhou, Y., Ren, J., Zhou, J., Sheng, V.S., Dai, H., and Dou, D. (2023). Federated learning of large language models with parameter-efficient prompt tuning and adaptive optimization. arXiv.10.18653/v1/2023.emnlp-main.488 ↗
58. Privacy and robustness in federated learning: Attacks and defenses10.1109/TNNLS.2022.3216981 ↗
59. The rise and potential of large language model based agents: A survey10.1007/s11432-024-4222-0 ↗
60. Malaviya, S., Shukla, M., and Lodha, S. (2023;, January 22–25). Reducing communication overhead in federated learning for pre-trained language models using parameter-efficient finetuning. Proceedings of the Conference on Lifelong Learning Agents, Montréal, QC, Canada.
61. Jiang, J., Jiang, H., Ma, Y., Liu, X., and Fan, C. (2024). Low-parameter federated learning with large language models. arXiv.10.1007/978-981-97-7707-5_28 ↗
62. Qlora: Efficient finetuning of quantized llms
63. Mao, Y., Huang, K., Guan, C., Bao, G., Mo, F., and Xu, J. (2024). Dora: Enhancing parameter-efficient fine-tuning with dynamic rank distribution. arXiv.10.18653/v1/2024.acl-long.626 ↗
64. Nasiri, H., and Garraghan, P. (2025). EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition. arXiv.
65. Gu, Y., Dong, L., Wei, F., and Huang, M. (2023). MiniLLM: Knowledge distillation of large language models. arXiv.
66. Naik, K. (2025, April 22). LLM Fine Tuning with LoRA. Available online: https://medium.com/@kednaik/llm-fine-tuning-with-lora-8e06f2227183.
67. Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (2023, January 23–29). SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot. Proceedings of the 40th International Conference on Machine Learning (ICML), Honolulu, HI, USA. Proceedings of Machine Learning Research.
68. Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (2023, January 23–29). SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. Proceedings of the 40th International Conference on Machine Learning (ICML), Honolulu, HI, USA. Proceedings of Machine Learning Research.
69. FedBERT: When federated learning meets pre-training10.1145/3510033 ↗
70. Differentially private low-rank adaptation of large language model using federated learning10.1145/3682068 ↗
71. Zhang, Z., Yang, Y., Dai, Y., Wang, Q., Yu, Y., Qu, L., and Xu, Z. (2023, January 9–14). FedPETuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models. Proceedings of the Annual Meeting of the Association of Computational Linguistics, Toronto, ON, Canada.10.18653/v1/2023.findings-acl.632 ↗
72. Ghiasvand, S., Alizadeh, M., and Pedarsani, R. (2025). Decentralized Low-Rank Fine-Tuning of Large Language Models. arXiv.
73. Deng, Y., Ren, J., Tang, C., Lyu, F., Liu, Y., and Zhang, Y. (2023, January 17–20). A hierarchical knowledge transfer framework for heterogeneous federated learning. Proceedings of the IEEE INFOCOM 2023—IEEE Conference on Computer Communications, New York, NY, USA.10.1109/INFOCOM53939.2023.10228954 ↗
74. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., and Lin, H. Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach. Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS 2020).
75. Collins, L., Wu, S., Oh, S., and Sim, K.C. (2023). PROFIT: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning. arXiv.
76. Yang, F.E., Wang, C.Y., and Wang, Y.C.F. (2023, January 1–6). Efficient model personalization in federated learning via client-specific prompt generation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01755 ↗
77. Yi, L., Yu, H., Wang, G., Liu, X., and Li, X. (2023). pFedLoRA: Model-heterogeneous personalized federated learning with LoRA tuning. arXiv.
78. Cho, Y.J., Liu, L., Xu, Z., Fahrezi, A., and Joshi, G. (2024). Heterogeneous lora for federated fine-tuning of on-device foundation models. arXiv.10.18653/v1/2024.emnlp-main.717 ↗
79. Su, S., Li, B., and Xue, X. (2025). Fedra: A random allocation strategy for federated tuning to unleash the power of heterogeneous clients. Computer Vision—ECCV 2024, Springer.10.1007/978-3-031-73195-2_20 ↗
80. Promptfl: Let federated participants cooperatively learn prompts instead of models–federated learning in age of foundation model10.1109/TMC.2023.3302410 ↗
81. Zhao, H., Du, W., Li, F., Li, P., and Liu, G. (2023, January 4–10). Fedprompt: Communication-efficient and privacy-preserving prompt tuning in federated learning. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.10.1109/ICASSP49357.2023.10095356 ↗
82. Chen, Y., Chen, Z., Wu, P., and Yu, H. (2022). FedOBD: Opportunistic block dropout for efficiently training large-scale neural networks through federated learning. arXiv.10.24963/ijcai.2023/394 ↗
83. Sun, G., Mendieta, M., Luo, J., Wu, S., and Chen, C. (2023, January 1–6). Fedperfix: Towards partial model personalization of vision transformers in federated learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00460 ↗
84. Chen, D., Yao, L., Gao, D., Ding, B., and Li, Y. (2023, January 23–29). Efficient personalized federated learning via sparse model-adaptation. Proceedings of the International Conference on Machine Learning (PMLR), Honolulu, HI, USA.
85. Cho, Y.J., Manoel, A., Joshi, G., Sim, R., and Dimitriadis, D. (2022). Heterogeneous ensemble knowledge transfer for training large models in federated learning. arXiv.10.24963/ijcai.2022/399 ↗
86. Sui, D., Chen, Y., Zhao, J., Jia, Y., Xie, Y., and Sun, W. (2020, January 16–20). Feded: Federated learning via ensemble distillation for medical relation extraction. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.165 ↗
87. Mei, H., Cai, D., Wu, Y., Wang, S., and Xu, M. (2024). A Survey of Backpropagation-free Training For LLMS. TechRxiv.10.36227/techrxiv.171172909.97532161/v1 ↗
88. Xu, M., Wu, Y., Cai, D., Li, X., and Wang, S. (2023). Federated Fine-tuning of Billion-sized Language Models Across Mobile Devices. arXiv.
89. Qin, Z., Chen, D., Qian, B., Ding, B., Li, Y., and Deng, S. (2023). Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. arXiv.
90. Sun, J., Xu, Z., Yin, H., Yang, D., Xu, D., Chen, Y., and Roth, H.R. (2023). Fedbpt: Efficient federated black-box prompt tuning for large language models. arXiv.
91. Pau, D.P., and Aymone, F.M. (2023, January 23–25). Suitability of forward-forward and pepita learning to mlcommons-tiny benchmarks. Proceedings of the 2023 IEEE International Conference on Omni-layer Intelligent Systems (COINS), Berlin, Germany.10.1109/COINS57856.2023.10189239 ↗
92. He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., and Qiu, H. (2020). FedML: A Research Library and Benchmark for Federated Machine Learning. arXiv.
93. Beutel, D.J., Topal, T., Mathur, A., Qiu, X., Fernandez-Marques, J., Gao, Y., Sani, L., Li, K.H., Parcollet, T., and de Gusmão, P.P.B. (2020). Flower: A Friendly Federated Learning Research Framework. arXiv.
94. A survey on IoT intrusion detection: Federated learning, game theory, social psychology, and explainable AI as future directions10.1109/JIOT.2022.3203249 ↗
95. Fan, T., Kang, Y., Ma, G., Chen, W., Wei, W., Fan, L., and Yang, Q. (2023). FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models. arXiv.
96. Kuang, W., Qian, B., Li, Z., Chen, D., Gao, D., Pan, X., Xie, Y., Li, Y., Ding, B., and Zhou, J. (2024, January 25–29). Federatedscope-llm: A comprehensive package for fine-tuning large language models in federated learning. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain.10.1145/3637528.3671573 ↗
97. Zhang, J., Vahidian, S., Kuo, M., Li, C., Zhang, R., Yu, T., Wang, G., and Chen, Y. (2024, January 14–19). Towards Building The Federatedgpt: Federated Instruction Tuning. Proceedings of the ICASSP, Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447454 ↗
98. Lin, B.Y., He, C., Zeng, Z., Wang, H., Huang, Y., Dupuy, C., Gupta, R., Soltanolkotabi, M., Ren, X., and Avestimehr, S. (2021). FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks. arXiv.10.18653/v1/2022.findings-naacl.13 ↗
99. Ye, R., Wang, W., Chai, J., Li, D., Li, Z., Xu, Y., Du, Y., Wang, Y., and Chen, S. (2024, January 25–29). Openfedllm: Training large language models on decentralized private data via federated learning. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain.10.1145/3637528.3671582 ↗
100. Chakshu, N.K., and Nithiarasu, P. (2024). Orbital learning: A novel, actively orchestrated decentralised learning for healthcare. Sci. Rep., 14.10.1038/s41598-024-60915-9 ↗