journal article·2024-12-20Open access

Consistent Vertical Federated Deep Learning Using Task-Driven Features to Construct Integrated IoT Services

Soyeon Oh, Minsoo Lee

Applied Sciences· Volume 14· Issue 24

Abstract

By training a multivariate deep learning model distributed across existing IoT services using vertical federated learning, expanded services could be constructed cost-effectively while preserving the independent data architecture of each service. Previously, we proposed a design approach for vertical federated learning considering IoT domain characteristics. Also, our previous method, designed leveraging our approach, achieved improved performance, especially in IoT domains, compared to other representative vertical federated learning mechanisms. However, our previous method was difficult to apply in real-world scenarios because its mechanism consisted of several options. In this paper, we propose a new vertical federated learning method, TT-VFDL-ST (Task-driven Transferred Vertical Federated Deep Learning using Self-Transfer partial training), a consistent single mechanism even in various real-world scenarios. The proposed method is also designed based on our previous design approach. However, the difference is that it leverages a newly proposed self-transfer partial training mechanism. The self-transfer partial training mechanism improved the MSE and accuracy of TT-VFDL-ST by 0.00262 and 12.08% on average compared to existing mechanisms. In addition, MSE and accuracy improved by up to 0.00290 and 5.08% compared to various options of our previous method. By applying the self-transfer partial training mechanism, TT-VFDL-ST could be used as a key solution to construct real-world-integrated IoT services.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Privacy-Preserving Technologies in Data · Blockchain Technology Applications and Security · Internet Traffic Analysis and Secure E-voting

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science and Engineering, Ewha Womans University, Seoul 03760, Republic of Korea

Funding

National Research Foundation of Korea (NRF)

References (63)

Show source-provided references
1. IoT Service Classification and Clustering for Integration of IoT Service Platforms10.1007/s11227-018-2288-7 ↗
2. A Model-Driven Engineering Approach for the Service Integration of IoT Systems10.1007/s10586-020-03150-x ↗
3. A Novel Distributed Authentication of Blockchain Technology Integration in IoT Services10.1109/ACCESS.2024.3349955 ↗
4. Smart Integrated IoT Healthcare System for Cancer Care10.1007/s11276-018-01932-1 ↗
5. Liu, Y., Kang, Y., Zou, T., Pu, Y., He, Y., Ye, X., Ouyang, Y., Zhang, Y.-Q., and Yang, Q. (2023). Vertical Federated Learning: Concepts, Advances and Challenges. arXiv.
6. Li, Q., Thapa, C., Ong, L., Zheng, Y., Ma, H., Camtepe, S.A., Fu, A., and Gao, Y. (2023). Vertical Federated Learning: Taxonomies, Threats, and Prospects. arXiv.
7. Oh, S., and Lee, M. (2024). Task-Driven Transferred Vertical Federated Deep Learning for Multivariate Internet of Things Time-Series Analysis. Appl. Sci., 14.10.3390/app14114606 ↗
8. Practical Vertical Federated Learning with Unsupervised Representation Learning10.1109/TBDATA.2022.3180117 ↗
9. Adaptive Vertical Federated Learning via Feature Map Transferring in Mobile Edge Computing10.1007/s00607-022-01117-x ↗
10. Implementing Vertical Federated Learning Using Autoencoders: Practical Application, Gener-alizability, and Utility Study10.2196/26598 ↗
11. Khan, A., Ten Thij, M., and Wilbik, A. (2022). Communication-Efficient Vertical Federated Learning. Algorithms, 15.10.3390/a15080273 ↗
12. Distributed Learning of Deep Neural Network over Multiple Agents10.1016/j.jnca.2018.05.003 ↗
13. Vepakomma, P., Gupta, O., Swedish, T., and Raskar, R. (2018). Split Learning for Health: Distributed Deep Learning without Sharing Raw Patient Data. arXiv.
14. Duan, Q., Hu, S., Deng, R., and Lu, Z. (2022). Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State-of-the-Art and Future Directions. Sensors, 22.10.3390/s22165983 ↗
15. Ji, J., Yan, D., and Mu, Z. (2022, January 15–17). Personnel Status Detection Model Suitable for Vertical Federated Learning Structure. Proceedings of the 2022 The 6th International Conference on Machine Learning and Soft Computing, Haikou, China.10.1145/3523150.3523166 ↗
16. Hu, Y., Niu, D., Yang, J., and Zhou, S. (2019, January 4–8). FDML: A Collaborative Machine Learning Framework for Distributed Features. Proceedings of the Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330765 ↗
17. Vertical Federated DNN Training10.1016/j.phycom.2021.101465 ↗
18. A Spatiotemporal Deep Learning Approach for Citywide Short-Term Crash Risk Prediction with Multi-Source Data10.1016/j.aap.2018.10.015 ↗
19. Hybrid Model of Convolutional LSTM and CNN to Predict Particulate Matter
20. Peng, H., Li, H., Zhang, Y., Wang, S., Gu, K., and Ren, M. (2022). Multi-Sensor Vibration Signal Based Three-Stage Fault Prediction for Rotating Mechanical Equipment. Entropy, 24.10.3390/e24020164 ↗
21. Cui, R., Zhu, A., Zhang, S., and Hua, G. (2018, January 20–24). Multi-Source Learning for Skeleton -Based Action Recognition Using Deep LSTM Net-works. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.10.1109/ICPR.2018.8545247 ↗
22. Siraj, M.S., and Ahad, M.A.R. (2020, January 26–29). A Hybrid Deep Learning Framework Using CNN and GRU-Based RNN for Recognition of Pair-wise Similar Activities. Proceedings of the 2020 Joint 9th International Conference on Informatics, Electronics & Vision (ICIEV) and 2020 4th International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Kitakyushu, Japan.10.1109/ICIEVicIVPR48672.2020.9306630 ↗
23. Mekruksavanich, S., and Jitpattanakul, A. (2021). LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes. Sensors, 21.10.3390/s21051636 ↗
24. McMahan, H.B., Moore, E., Ramage, D., Hampson, S., and Arcas, B.A. (2023). Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv.
25. Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools10.1145/3363554 ↗
26. Zhao, M., Yin, Y., Mao, Y., Liu, Q., Chen, L., and Gao, Y. (2024, January 13–16). SparDL: Distributed Deep Learning Training with Efficient Sparse Communication. Proceedings of the 2024 IEEE 40th International Conference on Data Engineering (ICDE), Utrecht, The Netherlands.10.1109/ICDE60146.2024.00142 ↗
27. Hierarchical Federated Learning for Hybrid Data Partitioning Across Multitype Sensors10.1109/JIOT.2021.3051382 ↗
28. Das, A., and Patterson, S. (2021, January 6–11). Multi-Tier Federated Learning for Vertically Partitioned Data. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9415026 ↗
29. Wang, Q., and Yang, K. (2024). Privacy-Preserving Data Fusion for Traffic State Estimation: A Vertical Federated Learning Approach. arXiv.10.1016/j.trc.2024.104743 ↗
30. FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks10.1007/s11042-023-15184-5 ↗
31. Castiglia, T., Wang, S., and Patterson, S. (2023). Flexible Vertical Federated Learning with Heterogeneous Parties. arXiv.
32. Castiglia, T., Das, A., Wang, S., and Patterson, S. (2023). Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data. arXiv.
33. SplitFed: When Federated Learning Meets Split Learning
34. Han, D.-J., Bhatti, H.I., Lee, J., and Moon, J. (2021, January 18–24). Accelerating Federated Learning with Split Learning on Locally Generated Losses. Proceedings of the ICML 2021 Workshop on Federated Learning for User Privacy and Data Confidentiality, Shanghai, China.
35. Guo, W., Zhuang, F., Zhang, X., Tong, Y., and Dong, J. (2024). A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications. Front. Comput. Sci., 18.10.1007/s11704-024-40065-x ↗
36. Alvi, M., Cardell-Oliver, R., and French, T. (2022, January 9–10). Utilizing Autoencoders to Improve Transfer Learning When Sensor Data Is Sparse. Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Boston, MA, USA.10.1145/3563357.3567407 ↗
37. Autoencoder-Based Representation Learning and Its Application in Intelligent Fault Di-agnosis: A Review10.1016/j.measurement.2021.110460 ↗
38. Transfer Learning Based on Improved Stacked Autoencoder for Bearing Fault Diagnosis10.1016/j.knosys.2022.109846 ↗
39. Han, T., Hao, K., Ding, Y., and Tang, X. (2017, January 17–20). A New Multilayer LSTM Method of Reconstruction for Compressed Sensing in Acquiring Human Pressure Data. Proceedings of the 2017 11th Asian Control Conference (ASCC), Gold Coast, Australia.10.1109/ASCC.2017.8287482 ↗
40. ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition10.1109/ACCESS.2022.3140373 ↗
41. Trirat, P., Shin, Y., Kang, J., Nam, Y., Na, J., Bae, M., Kim, J., Kim, B., and Lee, J.-G. (2024). Universal Time-Series Representation Learn-ing: A Survey. arXiv.
42. Transfer Learning in Environmental Remote Sensing10.1016/j.rse.2023.113924 ↗
43. Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., and Muller, P.-A. (2018, January 10–13). Transfer Learning for Time Series Classification. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8621990 ↗
44. A Survey on Time-Series Pre-Trained Models10.1109/TKDE.2024.3475809 ↗
45. Yuan, X., and Qiao, Y. (2024). Diffusion-TS: Interpretable Diffusion for General Time Series Generation. arXiv.
46. Alzantot, M., Chakraborty, S., and Srivastava, M. (2017, January 13–17). SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation. Proceedings of the 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, USA.10.1109/PERCOMW.2017.7917555 ↗
47. Narasimhan, S.S., Agarwal, S., Akcin, O., Sanghavi, S., and Chinchali, S. (2024). Time Weaver: A Conditional Time Series Generation Model. arXiv.
48. STO-CVAE: State Transition-Oriented Conditional Variational Autoencoder for Data Augmentation in Disability Classification10.1007/s40747-024-01370-x ↗
49. Dahmen, J., and Cook, D. (2019). SynSys: A Synthetic Data Generation System for Healthcare Applications. Sensors, 19.10.3390/s19051181 ↗
50. Dhekane, S.G., and Ploetz, T. (2024). Transfer Learning in Human Activity Recognition: A Survey. arXiv.
51. Hwang, S., and Kim, H.-E. (2016). Self-Transfer Learning for Fully Weakly Supervised Object Localization. arXiv.10.1007/978-3-319-46723-8_28 ↗
52. CAFE: Catastrophic Data Leakage in Vertical Federated Learning
53. Fu, C., Zhang, X., Ji, S., Chen, J., Wu, J., Guo, S., Zhou, J., Liu, A.X., and Wang, T. (2022, January 10–12). Label Inference Attacks Against Vertical Fed-erated Learning. Proceedings of the 31st USENIX Security Symposium (USENIX Security 22), Boston, MA, USA.
54. Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
55. Park, J., and Lim, H. (2022). Privacy-Preserving Federated Learning Using Homomorphic Encryption. Appl. Sci., 12.10.3390/app12020734 ↗
56. Xu, R., Baracaldo, N., Zhou, Y., Anwar, A., Joshi, J., and Ludwig, H. (2021, January 5). FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data. Proceedings of the Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, Virtual Event.10.1145/3474369.3486872 ↗
57. Privacy-Preserving Data Integration and Sharing in Multi-Party IoT Environments: An Entity Em-bedding Perspective10.1016/j.inffus.2024.102380 ↗
58. Smart Contract-Based Security Architecture for Collaborative Services in Municipal Smart Cities10.1016/j.sysarc.2022.102802 ↗
59. Vito, S. (2024, October 29). Air Quality. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/360/air+quality.
60. Candanedo, L. (2024, October 29). Appliances Energy Prediction. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction.
61. Burgus, J. (2024, October 29). Gas Sensor Array Temperature Modulation. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/487/gas+sensor+array+temperature+modulation.
62. (2024, October 29). Time Series Classification Website—Epilepsy. Available online: https://www.timeseriesclassification.com/description.php?Dataset=Epilepsy.
63. (2024, October 29). Time Series Classification Website—PEMS-SF. Available online: https://www.timeseriesclassification.com/description.php?Dataset=PEMS-SF.