A Deep Learning Framework Performance Evaluation to Use YOLO in Nvidia Jetson Platform
Source abstract
Abstract
Deep learning-based object detection technology can efficiently infer results by utilizing graphics processing units (GPU). However, when using general deep learning frameworks in embedded systems and mobile devices, processing functionality is limited. This allows deep learning frameworks such as TensorFlow-Lite (TF-Lite) and TensorRT (TRT) to be optimized for different hardware. Therefore, this paper introduces a performance inference method that fuses the Jetson monitoring tool with TensorFlow and TRT source code on the Nvidia Jetson AGX Xavier platform. In addition, central processing unit (CPU) utilization, GPU utilization, object accuracy, latency, and power consumption of the deep learning framework were compared and analyzed. The model is You Look Only Once Version4 (YOLOv4), and the dataset uses Common Objects in Context (COCO) and PASCAL Visual Object Classes (VOC). We confirmed that using TensorFlow results in high latency. We also confirmed that TensorFlow-TensorRT (TF-TRT) and TRT using Tensor Cores provide the most efficiency. However, it was confirmed that TF-Lite showed the lowest performance because it utilizes a GPU limited to mobile devices. Through this paper, we think that when developing deep learning-related object detection technology on the Nvidia Jetson platform or desktop environment, services and research can be efficiently conducted through measurement results.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Artificial Intelligence Applications · COVID-19 diagnosis using AI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Anyang University, Anyang-si 14058, Korea
Department of ICT Convergence Engineering, Anyang University, Anyang-si 14058, Korea
References (42)
Show source-provided references
1. Brain intelligence: Go beyond artificial intelligence10.1007/s11036-017-0932-8 ↗
2. An overview of the supervised machine learning methods10.20544/HORIZONS.B.04.1.17.P05 ↗
3. An overview of deep learning in the field of dentistry10.5624/isd.2019.49.1.1 ↗
4. Zou, Z., Shi, Z., Guo, Y., and Ye, J. (2019). Object detection in 20 years: A survey. arXiv.
5. Buber, E., and Banu, D.I.R.I. (2018, January 25–27). Performance analysis and CPU vs GPU comparison for deep learning. Proceedings of the 2018 6th International Conference on Control Engineering & Information Technology (CEIT), Istanbul, Turkey.10.1109/CEIT.2018.8751930 ↗
6. Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., and Shelhamer, E. (2014). cudnn: Efficient primitives for deep learning. arXiv.
7. Object detection with deep learning: A review10.1109/TNNLS.2018.2876865 ↗
8. Pham, M.T., Courtrai, L., Friguet, C., Lefèvre, S., and Baussard, A. (2020). YOLO-Fine: One-stage detector of small objects under various backgrounds in remote sensing images. Remote Sens., 12.10.3390/rs12152501 ↗
9. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., and Sun, J. (2017). Light-head r-cnn: In defense of two-stage object detector. arXiv.
10. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
11. (2022, February 05). Embedded Systems with Jetson. Available online: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/.
12. Shuai, X., Shen, Y., Tang, Y., Shi, S., Ji, L., and Xing, G. (2021, January 18–21). Millieye: A lightweight mmwave radar and camera fusion system for robust object detection. Proceedings of the International Conference on Internet-of-Things Design and Implementation, Charlottesville, VA, USA.10.1145/3450268.3453532 ↗
13. Xun, D.T.W., Lim, Y.L., and Srigrarom, S. (2021, January 20–22). Drone detection using YOLOv3 with transfer learning on NVIDIA Jetson TX2. Proceedings of the 2021 Second International Symposium on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA-SYMP), Bangkok, Thailand.
14. Run your visual-inertial odometry on NVIDIA Jetson: Benchmark tests on a micro aerial vehicle10.1109/LRA.2021.3075141 ↗
15. Deep embedded hybrid CNN-LSTM network for lane detection on NVIDIA Jetson Xavier NX10.1016/j.knosys.2021.107941 ↗
16. Valladares, S., Toscano, M., Tufiño, R., Morillo, P., and Vallejo-Huanga, D. (2021, January 22–24). Performance evaluation of the Nvidia Jetson Nano through a real-time machine learning application. Proceedings of the International Conference on Intelligent Human Systems Integration, Palermo, Italy.10.1007/978-3-030-68017-6_51 ↗
17. A Survey on optimized implementation of deep learning models on the NVIDIA Jetson platform10.1016/j.sysarc.2019.01.011 ↗
18. Bokovoy, A., Muravyev, K., and Yakovlev, K. (2019, January 4–6). Real-time vision-based depth reconstruction with nvidia jetson. Proceedings of the 2019 European Conference on Mobile Robots (ECMR), Prague, Czech Republic.10.1109/ECMR.2019.8870936 ↗
19. Nara, M., Mukesh, B.R., Padala, P., and Kinnal, B. (2019, January 23–25). Performance evaluation of deep learning frameworks on computer vision problems. Proceedings of the 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India.10.1109/ICOEI.2019.8862603 ↗
20. Shams, S., Platania, R., Lee, K., and Park, S.J. (2017, January 5–8). Evaluation of deep learning frameworks over different HPC architectures. Proceedings of the 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA.10.1109/ICDCS.2017.259 ↗
21. Süzen, A.A., Duman, B., and Şen, B. (2020, January 26–28). Benchmark analysis of jetson tx2, jetson nano and raspberry pi using deep-cnn. Proceedings of the 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), Ankara, Turkey.10.1109/HORA49412.2020.9152915 ↗
22. Baller, S.P., Jindal, A., Chadha, M., and Gerndt, M. (2021, January 4–8). DeepEdgeBench: Benchmarking Deep Neural Networks on Edge Devices. Proceedings of the 2021 IEEE International Conference on Cloud Engineering (IC2E), San Francisco, CA, USA.10.1109/IC2E52221.2021.00016 ↗
23. Verma, G., Gupta, Y., Malik, A.M., and Chapman, B. (2021, January 17–21). Performance evaluation of deep learning compilers for edge inference. Proceedings of the 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Portland, OR, USA.10.1109/IPDPSW52791.2021.00128 ↗
24. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
25. (2022, March 15). Pascal VOC Dataset Mirror. Available online: https://pjreddie.com/projects/pascal-voc-dataset-mirror.
26. (2022, March 15). YOLOv4 Darknet. Available online: https://github.com/AlexeyAB/darknet.
27. (2022, February 01). Recent trends in artificial intelligence projects. Available online: https://www.itfind.or.kr/WZIN/jugidong/1899/file2522744291233853764-189902.pdf.
28. (2022, February 17). TensorFlow-Lite Guide. Available online: https://www.tensorflow.org/lite/guide?hl=en.
29. (2022, February 17). TensorFlow-Lite Converter. Available online: https://www.tensorflow.org/lite/convert/index.
30. (2022, February 17). Nvidia TensorRT Introduction. Available online: https://developer.nvidia.com/tensorrt.
31. (2022, February 24). TensorFlow-TensorRT Documentation. Available online: https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html.
32. (2022, February 15). Yolov4 TensorFlow Source Code. Available online: https://github.com/hunglc007/tensorflow-yolov4-tflite.
33. (2022, February 15). Yolov4 TensorRT Source Code. Available online: https://github.com/jkjung-avt/tensorrt_demos.
34. (2022, February 12). Jetson Platform Monitoring Tool. Available online: https://github.com/rbonghi/jetson_stats.
35. (2022, January 10). Jetpack SDK. Available online: https://developer.nvidia.com/embedded/jetpack.
36. (2022, January 10). NVPModel Clock Configuration. Available online: https://info.nvidia.com/rs/156-OFN-742/images/Jetson_AGX_Xavier_New_Era_Autonomous_Machines.pdf.
37. (2022, February 24). Jetson Installing TensorFlow. Available online: https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html.
38. (2022, January 19). Yolov4-Native Weights Files. Available online: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights.
39. (2022, March 15). Yolov4-Tiny Weights Files. Available online: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights.
40. (2022, March 15). Convert2Yolo Source Code. Available online: https://github.com/ssaru/convert2Yolo.
41. (2022, February 07). TensorFlow-Lite GPU Delegate. Available online: https://www.tensorflow.org/lite/performance/gpu.
42. (2022, February 17). Nvidia TensorRT Install Guide. Available online: https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html.