Distributed Inference Models and Algorithms for Heterogeneous Edge Systems Using Deep Learning
Source abstract
Abstract
Computations performed by using convolutional layers in deep learning require significant resources; thus, their scope of applicability is limited. When deep neural network models are employed in an edge-computing system, the limited computational power and storage resources of edge devices can degrade inference performance, require a considerable amount of computation time, and result in increased energy consumption. To address these issues, this study presents a convolutional-layer partitioning model, based on the fused tile partitioning (FTP) algorithm, for enhancing the distributed inference capabilities of edge devices. First, a resource-adaptive workload-partitioning optimization model is designed to promote load balancing across heterogeneous edge systems. Next, the FTP algorithm is improved, leading to a new layer-fused partitioning method that is used to solve the optimization model. The results of simulation experiments show that the proposed convolutional-layer partitioning method effectively improves the inference performance of edge devices. When five edge devices are used, the speed of the proposed method becomes 1.65–3.48 times those of existing algorithms.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Neural Networks and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
References (29)
Show source-provided references
1. Edge intelligence: Paving the last mile of artificial intelligence with edge computing10.1109/JPROC.2019.2918951 ↗
2. Efficient processing of deep neural networks: A tutorial and survey10.1109/JPROC.2017.2761740 ↗
3. Model compression and hardware acceleration for neural networks: A comprehensive survey10.1109/JPROC.2020.2976475 ↗
4. Yin, M., Sui, Y., Liao, S., and Yuan, B. (2021, January 20–25). Towards efficient tensor decomposition-based dnn model compression with optimization framework. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01053 ↗
5. Deep Neural Network Channel Pruning Compression Method for Filter Elasticity
6. Han, S., Mao, H., and Dally, W.J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv.
7. Parallel blockwise knowledge distillation for deep neural network compression10.1109/TPDS.2020.3047003 ↗
8. Hinton, G. (2015). Distilling the Knowledge in a Neural Network. arXiv.
9. Han, S., Shen, H., Philipose, M., Agarwal, S., Wolman, A., and Krishnamurthy, A. (2016, January 26–30). Mcdnn: An approximation-based execution framework for deep stream processing under resource constraints. Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services, Singapore.10.1145/2906388.2906396 ↗
10. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge10.1145/3093337.3037698 ↗
11. Teerapittayanon, S., McDanel, B., and Kung, H.T. (2017, January 5–8). Distributed deep neural networks over the cloud, the edge and end devices. Proceedings of the 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA.10.1109/ICDCS.2017.226 ↗
12. Mao, J., Chen, X., Nixon, K.W., Krieger, C., and Chen, Y. (2017, January 27–31). Modnn: Local distributed mobile computing system for deep neural network. Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE), Lausanne, Switzerland.10.23919/DATE.2017.7927211 ↗
13. Mao, J., Yang, Z., Wen, W., Wu, C., Song, L., Nixon, K.W., Chen, X., Li, H., and Chen, Y. (2017, January 13–16). Mednn: A distributed mobile system with enhanced partition and deployment for large-scale dnns. Proceedings of the 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), Irvine, CA, USA.10.1109/ICCAD.2017.8203852 ↗
14. Deepthings: Distributed adaptive deep learning inference on resource-constrained iot edge clusters10.1109/TCAD.2018.2858384 ↗
15. Coedge: Cooperative dnn inference with adaptive workload partitioning over heterogeneous edge devices10.1109/TNET.2020.3042320 ↗
16. Joint architecture design and workload partitioning for dnn inference on industrial iot clusters10.1145/3551638 ↗
17. Zhou, L., Samavatian, M.H., Bacha, A., Majumdar, S., and Teodorescu, R. (2019, January 7–9). Adaptive parallel execution of deep neural networks on heterogeneous edge devices. Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, Washington, DC, USA.10.1145/3318216.3363312 ↗
18. KeepEdge: A Knowledge Distillation Empowered Edge Intelligence Framework for Visual Assisted Positioning in UAV Delivery10.1109/TMC.2022.3157957 ↗
19. Chen, C., Jiang, B., Liu, S., Li, C., Wu, C., and Yin, R. (2023, January 4–8). Efficient Federated Learning using Random Pruning in Resource-Constrained Edge Intelligence Networks. Proceedings of the GLOBECOM 2023—2023 IEEE Global Communications Conference, Kuala Lumpur, Malaysia.10.1109/GLOBECOM54140.2023.10437051 ↗
20. AdaMEC: Towards a Context-adaptive and Dynamically Combinable DNN Deployment Framework for Mobile Edge Computing10.1145/3630098 ↗
21. Ren, W., Qu, Y., Qin, Z., Dong, C., Zhou, F., Zhang, L., and Wu, Q. (2024, January 21–24). Efficient Pipeline Collaborative DNN Inference in Resource-Constrained UAV Swarm. Proceedings of the 2024 IEEE Wireless Communications and Networking Conference (WCNC), Dubai, United Arab Emirates.10.1109/WCNC57260.2024.10570535 ↗
22. Hou, X., Guan, Y., Han, T., and Zhang, N. (June, January 30). DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices. Proceedings of the 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Lyon, France.10.1109/IPDPS53621.2022.00110 ↗
23. Han, B., Dai, P., Li, K., Zhao, K., and Lei, X. (2024, January 12–14). SDPMP: Inference Acceleration of CNN Models in Heterogeneous Edge Environment. Proceedings of the 2024 7th World Conference on Computing and Communication Technologies (WCCCT), Chengdu, China.10.1109/WCCCT60665.2024.10541524 ↗
24. Hu, C., and Li, B. (2022, January 2–5). Distributed Inference with Deep Learning Models across Heterogeneous Edge Devices. Proceedings of the IEEE INFOCOM 2022—IEEE Conference on Computer Communications, Virtual.10.1109/INFOCOM48880.2022.9796896 ↗
25. Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2016). Pruning convolutional neural networks for resource efficient inference. arXiv.
26. Dantzig, G.B. (2016). Linear programming and extensions. Linear Programming and Extensions, Princeton University Press.
27. Lyken17 (2023, December 11). Pytorch-OpCounter. Available online: https://github.com/Lyken17/pytorch-OpCounter.
28. Shenzhen Beikun Cloud Computing Co., Ltd. (2023, September 11). Bei Kunyun Supercomputing Platform. Available online: https://www.bkunyun.com/.
29. Bossard, L., Guillaumin, M., and Van Gool, L. (2014, January 6–12). Food-101–mining discriminative components with random forests. Proceedings of the Computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland. Proceedings, Part VI 13.10.1007/978-3-319-10599-4_29 ↗