Task-Driven Transferred Vertical Federated Deep Learning for Multivariate Internet of Things Time-Series Analysis
Source abstract
Abstract
As big data technologies for IoT services develop, cross-service distributed learning techniques of multivariate deep learning models on IoT time-series data collected from various sources are becoming important. Vertical federated deep learning (VFDL) is used for cross-service distributed learning for multivariate IoT time-series deep learning models. Existing VFDL methods with reasonable performance require a large communication amount. On the other hand, existing communication-efficient VFDL methods have relatively low performance. We propose TT-VFDL-SIM, which can achieve improved performance over centralized training or existing VFDL methods in a communication-efficient manner. TT-VFDL-SIM derives partial tasks from the target task and applies transfer learning to them. In our task-driven transfer approach for the design of TT-VFDL-SIM, the SIM Partial Training mechanism contributes to performance improvement by introducing similar feature spaces in various ways. TT-VFDL-SIM was more communication-efficient than existing VFDL methods and achieved an average of 0.00153 improved MSE and 7.98% improved accuracy than centralized training or existing VFDL methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Time Series Analysis and Forecasting · Traffic Prediction and Management Techniques · Privacy-Preserving Technologies in Data
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Ewha Womans University, Seoul 03760, Republic of Korea
Funding
National Research Foundation of Korea (NRF)
References (46)
Show source-provided references
1. Li, Q., Thapa, C., Ong, L., Zheng, Y., Ma, H., Camtepe, S.A., Fu, A., and Gao, Y. (2023). Vertical Federated Learning: Taxonomies, Threats, and Prospects. arXiv.
2. Distributed Learning of Deep Neural Network over Multiple Agents10.1016/j.jnca.2018.05.003 ↗
3. Wu, Z., Li, Q., and He, B. (2022). Practical Vertical Federated Learning with Unsupervised Representation Learning. IEEE Trans. Big Data, Early access.
4. Wang, Q., and Yang, K. (2024). Privacy-Preserving Data Fusion for Traffic State Estimation: A Vertical Federated Learning Approach. arXiv.10.1016/j.trc.2024.104743 ↗
5. A Novel Federated Learning Approach with Knowledge Transfer for Credit Scoring10.1016/j.dss.2023.114084 ↗
6. Gradient-Based Defense Methods for Data Leakage in Vertical Federated Learning10.1016/j.cose.2024.103744 ↗
7. FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks10.1007/s11042-023-15184-5 ↗
8. Adaptive Vertical Federated Learning via Feature Map Transferring in Mobile Edge Computing10.1007/s00607-022-01117-x ↗
9. Implementing Vertical Federated Learning Using Autoencoders: Practical Application, Gener-alizability, and Utility Study10.2196/26598 ↗
10. Khan, A., Ten Thij, M., and Wilbik, A. (2022). Communication-Efficient Vertical Federated Learning. Algorithms, 15.10.3390/a15080273 ↗
11. Vepakomma, P., Gupta, O., Swedish, T., and Raskar, R. (2018). Split Learning for Health: Distributed Deep Learning without Sharing Raw Patient Data. arXiv.
12. Duan, Q., Hu, S., Deng, R., and Lu, Z. (2022). Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State-of-the-Art and Future Directions. Sensors, 22.10.3390/s22165983 ↗
13. Ji, J., Yan, D., and Mu, Z. (2022, January 15–17). Personnel Status Detection Model Suitable for Vertical Federated Learning Structure. Proceedings of the 2022 The 6th International Conference on Machine Learning and Soft Computing, Haikou, China.10.1145/3523150.3523166 ↗
14. Hu, Y., Niu, D., Yang, J., and Zhou, S. (2019, January 4–8). FDML: A Collaborative Machine Learning Framework for Distributed Features. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330765 ↗
15. Vertical Federated DNN Training10.1016/j.phycom.2021.101465 ↗
16. Han, D.-J., Bhatti, H.I., Lee, J., and Moon, J. (2021, January 18–24). Accelerating Federated Learning with Split Learning on Locally Generated Losses. Proceedings of the ICML 2021 Workshop on Federated Learning for User Privacy and Data Confidentiality, Virtual Only.
17. A Spatiotemporal Deep Learning Approach for Citywide Short-Term Crash Risk Prediction with Multi-Source Data10.1016/j.aap.2018.10.015 ↗
18. Hybrid Model of Convolutional LSTM and CNN to Predict Particulate Matter10.18178/IJIEE.2019.9.1.701 ↗
19. Peng, H., Li, H., Zhang, Y., Wang, S., Gu, K., and Ren, M. (2022). Multi-Sensor Vibration Signal Based Three-Stage Fault Prediction for Rotating Mechanical Equipment. Entropy, 24.10.3390/e24020164 ↗
20. Cui, R., Zhu, A., Zhang, S., and Hua, G. (2018, January 20–24). Multi-Source Learning for Skeleton -Based Action Recognition Using Deep LSTM Networks. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.10.1109/ICPR.2018.8545247 ↗
21. Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks10.1007/978-3-319-08010-9_33 ↗
22. Mekruksavanich, S., and Jitpattanakul, A. (2021). LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes. Sensors, 21.10.3390/s21051636 ↗
23. McMahan, H.B., Moore, E., Ramage, D., Hampson, S., and Arcas, B.A. (2023). y Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv.
24. Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P., and Yang, K. (2012, January 3–8). Large Scale Distributed Deep Networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
25. Chilimbi, T., Suzue, Y., Apacible, J., and Kalyanaraman, K. (2014, January 6–8). Project Adam: Building an Efficient and Scalable Deep Learning Training System. Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, Broomfield, CO, USA.
26. Castiglia, T., Das, A., Wang, S., and Patterson, S. (2023). Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data. arXiv.
27. SplitFed: When Federated Learning Meets Split Learning10.1609/aaai.v36i8.20825 ↗
28. Hierarchical Federated Learning for Hybrid Data Partitioning Across Multitype Sensors10.1109/JIOT.2021.3051382 ↗
29. Das, A., and Patterson, S. (2021, January 6–11). Multi-Tier Federated Learning for Vertically Partitioned Data. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9415026 ↗
30. A Survey of Transfer Learning10.1186/s40537-016-0043-6 ↗
31. Han, T., Hao, K., Ding, Y., and Tang, X. (2017, January 17–20). A New Multilayer LSTM Method of Reconstruction for Compressed Sensing in Acquiring Human Pressure Data. Proceedings of the 2017 11th Asian Control Conference (ASCC), Gold Coast, QLD, Australia.10.1109/ASCC.2017.8287482 ↗
32. A New Deep Transfer Learning Based on Sparse Auto-Encoder for Fault Diagnosis10.1109/TSMC.2017.2754287 ↗
33. Transfer Learning Based on Improved Stacked Autoencoder for Bearing Fault Diagnosis10.1016/j.knosys.2022.109846 ↗
34. Alvi, M., Cardell-Oliver, R., and French, T. (2022, January 9–10). Utilizing Autoencoders to Improve Transfer Learning When Sensor Data Is Sparse. Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Boston, MA, USA.10.1145/3563357.3567407 ↗
35. ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition10.1109/ACCESS.2022.3140373 ↗
36. (2024, May 22). Home-UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/.
37. (2024, May 22). Time Series Classification Website. Available online: https://www.timeseriesclassification.com/dataset.php.
38. (2023, November 19). Vito, Saverio. Air Quality. UCI Machine Learning Repository. Available online: https://doi.org/10.24432/C59K5F.
39. (2023, November 19). Candanedo, Luis. Appliances Energy Prediction. UCI Machine Learning Repository. Available online: https://doi.org/10.24432/C5VC8G.
40. (2023, November 19). Burgus, Javier. Gas Sensor Array Temperature Modulation. UCI Machine Learning Repository. Available online: https://doi.org/10.24432/C5S302.
41. (2023, November 19). Time Series Classification Website—Epilepsy. Available online: https://www.timeseriesclassification.com/description.php?Dataset=Epilepsy.
42. (2023, November 19). Time Series Classification Website—PEMS-SF. Available online: https://www.timeseriesclassification.com/description.php?Dataset=PEMS-SF.
43. (2023, November 19). API Documentation|TensorFlow v2.10.1. Available online: https://www.tensorflow.org/versions/r2.10/api_docs.
44. A Data Augmentation Method for Vertical Federated Learning
45. Wu, Z., Li, Q., and He, B. (2023). A Coupled Design of Exploiting Record Similarity for Practical Vertical Federated Learning. arXiv.
46. Yang, Y., Ye, X., and Sakurai, T. (2022, January 18–21). Multi-View Federated Learning with Data Collaboration. Proceedings of the 2022 14th Inter-national Conference on Machine Learning and Computing (ICMLC), Guangzhou, China.10.1145/3529836.3529904 ↗