Optimized Deep Neural Networks for Real-Time Object Classification on Embedded GPUs
Source abstract
Abstract
Convolution is the most computationally intensive task of the Convolutional Neural Network (CNN). It requires a lot of memory storage and computational power. There are different approaches to compute the solution of convolution and reduce its computational complexity. In this paper, a matrix multiplication-based convolution (ConvMM) approach is fully parallelized using concurrent resources of GPU (Graphics Processing Unit) and optimized, considerably improving the performance of the image classifiers and making them applicable to real-time embedded applications. The flow of this CUDA (Compute Unified Device Architecture)-based scheme is optimized using unified memory and hardware-dependent acceleration of matrix multiplication. Proposed flow is evaluated on two different embedded platforms: first on an Nvidia Jetson TX1 embedded board and then on a Tegra K1 GPU of an Nvidia Shield K1 Tablet. The performance of this optimized and accelerated convolutional layer is compared with its sequential and heterogeneous versions. Results show that the proposed scheme significantly improves the overall results including energy efficiency, storage requirement and inference performance. In particular, the proposed scheme on embedded GPUs is hundreds of times faster than the sequential version and delivers tens of times higher performance than the heterogeneous approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Brain Tumor Detection and Classification · Machine Learning and ELM
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Dipartimento di Automatica e Informatica (DAUIN), Politecnico di Torino, 10129 Turin, Italy
Joint Open Lab, Telecom Italia Mobile (TIM), 10129 Turin, Italy
References (29)
Show source-provided references
1. Rizvi, S.T.H., Cabodi, G., and Francini, G. (2017, January 5–7). GPU-only Unified ConvMM Layer for Neural Classifiers. Proceedings of the 2017 International Conference on Control, Decision and Information Technologies (CoDIT’17), Barcelona, Spain. in press.10.1109/CoDIT.2017.8102649 ↗
2. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–8). Imagenet classification with deep convolutional neural networks. Proceedings of the Twenty-sixth Annual Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA.
3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.E., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going Deeper with Convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
4. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
5. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., and LeCun, Y. (2016, November 12). OverFeat: Integrated Recognition, Localization and Detection Using Convolutional Networks. Available online: https://arxiv.org/abs/1312.6229.
6. Mathieu, M., Henaff, M., and Lecun, Y. (2016, November 10). Fast Training of Convolutional Networks through FFTs. Available online: http://arxiv.org/abs/1312.5851.
7. Lavin, A, and Gray, S. (2016, January 27–30). Fast Algorithms for Convolutional Neural Networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.435 ↗
8. Chellapilla, K., Puri, S., and Simard, P. (2006, January 23–26). High Performance Convolutional Neural Networks for Document Processing. Proceedings of the Tenth International Workshop on Frontiers in Handwriting Recognition, La Baule, France.
9. Cong, J., and Xiao, B. (2014, January 15–19). Minimizing Computation in Convolutional Neural Networks. Proceedings of the 24th International Conference on Artificial Neural Networks and Machine Learning—ICANN 2014, Hamburg, Germany.10.1007/978-3-319-11179-7_36 ↗
10. Wang, J., Lin, J., and Wang, Z. (2016, January 13–15). Efficient convolution architectures for convolutional neural network. Proceedings of the 8th International Conference on Wireless Communications & Signal Processing (WCSP), Yangzhou, China.10.1109/WCSP.2016.7752726 ↗
11. Jiang, W., Chen, Y., Jin, H., Luo, B., and Chi, Y. (2015, January 24–26). A Novel Fast Approach for Convolutional Networks with Small Filters Based on GPU. Proceedings of the 2015 IEEE 12th International Conferen on Embedded Software and Systems (ICESS), 2015 IEEE 17th International Conference on High Performance Computing and Communications (HPCC), 2015 IEEE 7th International Symposium on Cyberspace Safety and Security (CSS), New York, NY, USA.10.1109/HPCC-CSS-ICESS.2015.218 ↗
12. Li, X., Zhang, G., Huang, H.H., Wang, Z., and Zheng, W. (2016, January 17–19). Performance Analysis of GPU-Based Convolutional Neural Networks. Proceedings of the 45th International Conference on Parallel Processing (ICPP), Philadelphia, PA, USA.10.1109/ICPP.2016.15 ↗
13. Park, H., Kim, D., Ahn, J., and Yoo, S. (2016, January 1–7). Zero and data reuse-aware fast convolution for deep neural networks on GPU. Proceedings of the 2016 International Conference on Hardware/Software Codesign and System Synthesis (CODES + ISSS), Pittsburgh, PA, USA.10.1145/2968456.2968476 ↗
14. Li, K., Shi, H., and Hu, Q. (2016, January 13–15). Complex convolution Kernel for deep networks. Proceedings of the 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP), Yangzhou, China.10.1109/WCSP.2016.7752631 ↗
15. Rizvi, S.T.H., Cabodi, G., Patti, D., and Francini, G. (2016). GPGPU Accelerated Deep Object Classification on a Heterogeneous Mobile Platform. Electronics, 5.10.3390/electronics5040088 ↗
16. Huynh, L.N., Balan, R.K., and Lee, Y. (2016, January 30). DeepSense: A GPU-based Deep Convolutional Neural Network Framework on Commodity Mobile Devices. Proceedings of the 2016 Workshop on Wearable Systems and Applications, Singapore.10.1145/2935643.2935650 ↗
17. Oskouei, S.S.L., Golestani, H., Hashemi, M., and Ghiasi, S. (2016, January 15–19). CNNdroid: GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Android. Proceedings of the 2016 ACM on Multimedia Conference (MM’16). ACM, New York, NY, USA.
18. Tsung, P.K., Tsai, S.F., Pai, A., Lai, S.J., and Lu, C. (2016, January 7–11). High performance deep neural network on low cost mobile GPU. Proceedings of the 2016 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.10.1109/ICCE.2016.7430525 ↗
19. Wu, J., Leng, C., Wang, Y., Hu, Q., and Cheng, J. (2016, January 27–30). Quantized Convolutional Neural Networks for Mobile Devices. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.521 ↗
20. Kim, Y.-D., Park, E., Yoo, S., Choi, T., Yang, L., and Shin, D. (2016, November 14). Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications. Available online: http://arxiv.org/abs/1511.06530.
21. Naik, V.H., and Kusur, C.S. (2015, January 19–20). Analysis of performance enhancement on graphic processor based heterogeneous architecture: A CUDA and MATLAB experiment. Proceedings of the 2015 National Conference on Parallel Computing Technologies (PARCOMPTECH), Bangalore, India.10.1109/PARCOMPTECH.2015.7084519 ↗
22. A Hybrid CPU–GPU Real-Time Hyperspectral Unmixing Chain10.1109/JSTARS.2015.2485399 ↗
23. Real-Time Implementation of the Vertex Component Analysis Algorithm on GPUs10.1109/LGRS.2012.2200452 ↗
24. Hyperspectral Unmixing on GPUs and Multi-Core Processors: A Comparison10.1109/JSTARS.2013.2254470 ↗
25. Yang, Y., Feng, M., and Chakradhar, S. (2016, January 16–19). HppCnn: A High-Performance, Portable Deep-Learning Library for GPGPUs. Proceedings of the 2016 45th International Conference on Parallel Processing (ICPP), Philadelphia, PA, USA.10.1109/ICPP.2016.73 ↗
26. Shi, Y., Niranjan, U.N., Anandkumar, A., and Cecka, C. (2016, January 19–22). Tensor Contractions with Extended BLAS Kernels on CPU and GPU. Proceedings of the 2016 IEEE 23rd International Conference on High Performance Computing (HiPC), Hyderabad, India.10.1109/HiPC.2016.031 ↗
27. Vehicle Logo Recognition System Based on Convolutional Neural Networks with a Pretraining Strategy10.1109/TITS.2014.2387069 ↗
28. Very Deep Convolutional Neural Networks for Noise Robust Speech Recognition10.1109/TASLP.2016.2602884 ↗
29. Christiansen, P., Nielsen, L.N., Steen, K.A., Jørgensen, R.N., and Karstoft, H. (2016). DeepAnomaly: Combining Background Subtraction and Deep Learning for Detecting Obstacles and Anomalies in an Agricultural Field. Sensors, 16.10.3390/s16111904 ↗