Secure and Efficient Federated Gradient Boosting Decision Trees
Source abstract
Abstract
In recent years, federated GBDTs have gradually replaced traditional GBDTs, and become the focus of academic research. They are used to solve the task of structured data mining. Aiming at the problems of information leakage, insufficient model accuracy and high communication cost in the existing schemes of horizontal federated GBDTs, this paper proposes an algorithm of gradient boosting decision trees based on horizontal federated learning, that is, secure and efficient FL for GBDTs (SeFB). The algorithm uses locality sensitive hashing (LSH) to build a tree by collecting similar information of instances without exposing the original data of participants. In the stage of updating the tree, the algorithm aggregates the local gradients of all data participants and calculates the global leaf weights, so as to improve the accuracy of the model and reduce the communication cost. Finally, the experimental analysis shows that the algorithm can protect the privacy of the original data, and the communication cost is low. At the same time, the performance of the unbalanced binary data set is evaluated. The results show that SeFB algorithm compared with the existing schemes of horizontal federated GBDTs, the accuracy is improved by 2.53% on average.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Privacy-Preserving Technologies in Data · Cryptography and Data Security · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronics and Information Engineering, Liaoning University of Technology, Jinzhou 121000, China
Funding
Liaoning Applied Basic Research Program
References (48)
Show source-provided references
1. Konečný, J., McMahan, H.B., Yu, F.X., Richtárik, P., Suresh, A.T., and Bacon, D. (2016). Federated learning: Strategies for improving communication efficiency. arXiv.
2. Gao, D., Ju, C., Wei, X., Liu, Y., Chen, T., and Yang, Q. (2019). Hhhfl: Hierarchical heterogeneous horizontal federated learning for electroencephalography. arXiv.
3. Liu, Y., Kang, Y., Zhang, X., Li, L., Cheng, Y., Chen, T., Hong, M., and Yan, Q. (2019). A communication efficient vertical federated learning framework. arXiv.
4. Sharma, S., Xing, C., Liu, Y., and Kang, Y. (2019, January 9–12). Secure and efficient federated transfer learning. Proceedings of the 2019 IEEE international conference on big data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9006280 ↗
5. Li, X., Huang, K., Yang, W., Wang, S., and Zhang, Z. (2019). On the convergence of fedavg on non-iid data. arXiv.
6. Advances and open problems in federated learning10.1561/2200000083 ↗
7. Robust and communication-efficient federated learning from non-iid data10.1109/TNNLS.2019.2944481 ↗
8. Wang, H., Kaplan, Z., Niu, D., and Li, B. (2020, January 6–9). Optimizing federated learning on non-iid data with reinforcement learning. Proceedings of the IEEE INFOCOM 2020-IEEE Conference on Computer Communications, Toronto, ON, Canada.10.1109/INFOCOM41043.2020.9155494 ↗
9. Li, Q., Diao, Y., Chen, Q., and He, B. (2022, January 9). Federated learning on non-iid data silos: An experimental study. Proceedings of the 2022 IEEE the 38th International Conference on Data Engineering (ICDE), Kuala Lumpur, Malaysia.10.1109/ICDE53745.2022.00077 ↗
10. Ternary compression for communication-efficient federated learning10.1109/TNNLS.2020.3041185 ↗
11. Haddadpour, F., Kamani, M.M., Mokhtari, A., and Mahdavi, M. (2021, January 13–15). Federated learning with compression: Unified analysis and sharp guarantees. Proceedings of the International Conference on Artificial Intelligence and Statistics, San Diego, CA, USA.
12. An efficiency-boosting client selection scheme for federated learning with fairness guarantee
13. Kairouz, P., McMahan, B., Song, S., Thakkar, O., Thakurta, A., and Xu, Z. (2021, January 18–24). Practical and private (deep) learning without sampling or shuffling. Proceedings of the International Conference on Machine Learning, Online.
14. Anonymous and privacy-preserving federated learning with industrial big data10.1109/TII.2021.3052183 ↗
15. Privacy-preserving federated learning framework based on chained secure multiparty computing10.1109/JIOT.2020.3022911 ↗
16. DP-FL: A novel differentially private federated learning framework for the unbalanced data10.1007/s11280-020-00780-4 ↗
17. PEFL: Deep privacy-encoding-based federated learning framework for smart agriculture10.1109/MM.2021.3112476 ↗
18. Elsayed, S., Thyssens, D., Rashed, A., Jomaa, H.S., and Schmidt-Thieme, L. (2021). Do we really need deep learning models for time series forecasting?. arXiv.
19. Revisiting deep learning models for tabular data
20. Tabular data: Deep learning is not all you need10.1016/j.inffus.2021.11.011 ↗
21. Grinsztajn, L., Oyallon, E., and Varoquaux, G. (2022). Why do tree-based models still outperform deep learning on tabular data?. arXiv.
22. Chen, T., and Guestrin, C. (2016, January 13–17). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
23. Lightgbm: A highly efficient gradient boosting decision tree
24. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., and Gulin, A. (2018). CatBoost: Unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst., 31.
25. A gradient boosting decision tree based GPS signal reception classification algorithm10.1016/j.asoc.2019.105942 ↗
26. Research on travel time prediction model of freeway based on gradient boosting decision tree
27. Automated detection of driver fatigue based on EEG signals using gradient boosting decision tree model10.1007/s11571-018-9485-1 ↗
28. Secureboost: A lossless federated learning framework10.1109/MIS.2021.3082561 ↗
29. Deforth, K., Desgroseilliers, M., Gama, N., Georgieva, M., Jetchev, D., and Vuille, M. (2021). XORBoost: Tree boosting in the multiparty computation setting. Proc. Privacy Enhancing Technol., 66–85.10.56553/popets-2022-0099 ↗
30. Feng, Z., Xiong, H., Song, C., Yang, S., Zhao, B., Wang, L., and Huan, J. (2019, January 9–12). Securegbm: Secure multi-party gradient boosting. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9006000 ↗
31. Meng, X., and Feigenbaum, J. (2020). Privacy-preserving xgboost inference. arXiv.
32. Law, A., Leung, C., Poddar, R., Popa, R.A., Shi, C., Sima, O., and Zheng, W. (2020, January 9). Secure collaborative training and inference for xgboost. Proceedings of the 2020 workshop on privacy-preserving machine learning in practice, Virtual Event.10.1145/3411501.3419420 ↗
33. Zhao, L., Ni, L., Hu, S., Chen, Y., Zhou, P., Xiao, F., and Wu, L. (2018, January 15–19). Inprivate digging: Enabling tree-based distributed data mining with differential privacy. Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications, Honolulu, HA, USA.10.1109/INFOCOM.2018.8486352 ↗
34. eFL-Boost: Efficient Federated Learning for Gradient Boosting Decision Trees10.1109/ACCESS.2022.3169502 ↗
35. Tian, Z., Zhang, R., Hou, X., Liu, J., and Ren, K. (2020). FederBoost: Private federated learning for GBDT. arXiv.
36. Practical federated gradient boosting decision trees
37. Similarity search in high dimensions via hashing
38. Wang, B., Yu, S., Lou, W., and Hou, Y.T. (May, January 27). Privacy-preserving multi-keyword fuzzy search over encrypted data in the cloud. Proceedings of the IEEE INFOCOM 2014-IEEE conference on computer communications, Toronto, ON, Canada.10.1109/INFOCOM.2014.6848153 ↗
39. A distributed locality-sensitive hashing-based approach for cloud service recommendation from multi-source data10.1109/JSAC.2017.2760458 ↗
40. Datar, M., Immorlica, N., Indyk, P., and Mirrokni, V.S. (2004, January 9–11). Locality-sensitive hashing scheme based on p-stable distributions. Proceedings of the twentieth annual symposium on Computational geometry, Brooklyn, NY, USA.10.1145/997817.997857 ↗
41. Du, W., Han, Y.S., and Chen, S. (2004, January 22–24). Privacy-preserving multivariate statistical analysis: Linear regression and classification. Proceedings of the 2004 SIAM international conference on data mining, Lake Buena Vista, FL, USA.10.1137/1.9781611972740.21 ↗
42. A secure federated transfer learning framework10.1109/MIS.2020.2988525 ↗
43. Yao, A.C. (1982, January 3–5). Protocols for secure computations. Proceedings of the 23rd annual symposium on foundations of computer science (sfcs 1982), Chicago, IL, USA.10.1109/SFCS.1982.38 ↗
44. Kaggle (2022, September 02). Credit Card Fraud Detection. Available online: https://www.kaggle.com/mlg-ulb/creditcardfraud.
45. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients10.1016/j.eswa.2007.12.020 ↗
46. Kohavi, R., and Becker, B. (1996). UCI Machine Learning Repository, University of California, School of Information and Computer Science.
47. Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., and Chandra, V. (2018). Federated learning with non-iid data. arXiv.
48. Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., and Khazaeni, Y. (2019, January 10–15). Bayesian nonparametric federated learning of neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.