Hardware Platform-Aware Binarized Neural Network Model Optimization
Source abstract
Abstract
Deep Neural Networks (DNNs) have shown superior accuracy at the expense of high memory and computation requirements. Optimizing DNN models regarding energy and hardware resource requirements is extremely important for applications with resource-constrained embedded environments. Although using binary neural networks (BNNs), one of the recent promising approaches, significantly reduces the design’s complexity, accuracy degradation is inevitable when reducing the precision of parameters and output activations. To balance between implementation cost and accuracy, in addition to proposing specialized hardware accelerators for corresponding specific network models, most recent software binary neural networks have been optimized based on generalized metrics, such as FLOPs or MAC operation requirements. However, with the wide range of hardware available today, independently evaluating software network structures is not good enough to determine the final network model for typical devices. In this paper, an architecture search algorithm based on estimating the hardware performance at the design time is proposed to achieve the best binary neural network models for hardware implementation on target platforms. With the XNOR-net used as a base architecture and target platforms, including Field Programmable Gate Array (FPGA), Graphic Processing Unit (GPU), and Resistive Random Access Memory (RRAM), the proposed algorithm shows its efficiency by giving more accurate estimation for the hardware performance at the design time than FLOPs or MAC operations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Memory and Neural Computing · Machine Learning and ELM
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Kyung Hee University, Global Campus, Yongin 17104, Korea
Department of Electronics and Information Convergence Engineering, Kyung Hee University, Global Campus, Yongin 17104, Korea
Department of Electronic Engineering, Kyung Hee University, Global Campus, Yongin 17104, Korea
References (60)
Show source-provided references
1. Deep Learning for Computer Vision: A Brief Review
2. Speech recognition using deep neural networks: A systematic review10.1109/ACCESS.2019.2896880 ↗
3. Touretzky, D. (1990). Optimal brain damage. Advances in Neural Information Processing Systems, Morgan-Kaufmann.
4. Han, S., Pool, J., Tran, J., and Dally, W.J. (2015). Learning both weights and connections for efficient neural networks. arXiv.
5. Han, S., Mao, H., and Dally, W.J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv.
6. Lin, D., Talathi, S., and Annapureddy, S. (2016, January 20–22). Fixed point quantization of deep convolutional networks. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
7. Wu, J., Leng, C., Wang, Y., Hu, Q., and Cheng, J. (2016, January 27–30). Quantized convolutional neural networks for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.521 ↗
8. Courbariaux, M., Bengio, Y., and David, J.P. (2015). Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in Neural Information Processing Systems, Curran Associates, Inc.
9. Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. (2016). Xnor-net: Imagenet classification using binary convolutional neural networks. European Conference on Computer Vision, Springer.10.1007/978-3-319-46493-0_32 ↗
10. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., and Bengio, Y. (2016). Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or -1. arXiv.
11. Tang, W., Hua, G., and Wang, L. (2017, January 4–9). How to train a compact binary neural network with high accuracy?. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10862 ↗
12. Lin, X., Zhao, C., and Pan, W. (2017). Towards accurate binary convolutional neural network. arXiv.
13. Liu, Z., Shen, Z., Savvides, M., and Cheng, K.T. (2020). Reactnet: Towards precise binary neural network with generalized activation functions. European Conference on Computer Vision, Springer.10.1007/978-3-030-58568-6_9 ↗
14. Lin, M., Ji, R., Xu, Z., Zhang, B., Wang, Y., Wu, Y., Huang, F., and Lin, C.W. (2020). Rotated binary neural network. arXiv.
15. FINN-R: An end-to-end deep-learning framework for fast exploration of quantized neural networks10.1145/3242897 ↗
16. Umuroglu, Y., Fraser, N.J., Gambardella, G., Blott, M., Leong, P., Jahre, M., and Vissers, K. (2017, January 22–24). Finn: A framework for fast, scalable binarized neural network inference. Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.10.1145/3020078.3021744 ↗
17. FP-BNN: Binarized neural network on FPGA10.1016/j.neucom.2017.09.046 ↗
18. Fu, C., Zhu, S., Su, H., Lee, C.E., and Zhao, J. (2018). Towards fast and energy-efficient binarized neural network inference on fpga. arXiv.10.1145/3289602.3293990 ↗
19. Sun, X., Yin, S., Peng, X., Liu, R., Seo, J.S., and Yu, S. (2018, January 19–23). XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks. Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany.10.23919/DATE.2018.8342235 ↗
20. XNOR-SRAM: In-memory computing SRAM macro for binary/ternary deep neural networks
21. Xcel-RAM: Accelerating binary neural networks in high-throughput SRAM compute arrays10.1109/TCSI.2019.2907488 ↗
22. Khwa, W.S., Chen, J.J., Li, J.F., Si, X., Yang, E.Y., Sun, X., Liu, R., Chen, P.Y., Li, Q., and Yu, S. (2018, January 11–15). A 65 nm 4 Kb algorithm-dependent computing-in-memory SRAM unit-macro with 2.3 ns and 55.8 TOPS/W fully parallel product-sum operation for binary DNN edge processors. Proceedings of the 2018 IEEE International Solid-State Circuits Conference-(ISSCC), San Francisco, CA, USA.10.1109/ISSCC.2018.8310401 ↗
23. Kim, H., Sim, J., Choi, Y., and Kim, L.S. (2017, January 18–22). A kernel decomposition architecture for binary-weight convolutional neural networks. Proceedings of the 54th Annual Design Automation Conference 2017, Austin, TX, USA.10.1145/3061639.3062189 ↗
24. Alwani, M., Chen, H., Ferdman, M., and Milder, P. (2016, January 15–19). Fused-layer CNN accelerators. Proceedings of the 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Taipei, Taiwan.10.1109/MICRO.2016.7783725 ↗
25. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks10.1109/JSSC.2016.2616357 ↗
26. Lu, L., and Liang, Y. (2018, January 24–29). SpWA: An efficient sparse winograd convolutional neural networks accelerator on FPGAs. Proceedings of the 55th Annual Design Automation Conference, San Francisco, CA, USA.10.1145/3195970.3196120 ↗
27. Lu, L., Liang, Y., Xiao, Q., and Yan, S. (May, January 30). Evaluating fast algorithms for convolutional neural networks on FPGAs. Proceedings of the 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), Napa, CA, USA.10.1109/FCCM.2017.64 ↗
28. Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., and Song, S. (2016, January 21–23). Going deeper with embedded fpga platform for convolutional neural network. Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.10.1145/2847263.2847265 ↗
29. Wei, X., Liang, Y., Li, X., Yu, C.H., Zhang, P., and Cong, J. (2018, January 5–8). TGPA: Tile-grained pipeline architecture for low latency CNN inference. Proceedings of the International Conference on Computer-Aided Design, San Diego, CA, USA.10.1145/3240765.3240856 ↗
30. Wei, X., Yu, C.H., Zhang, P., Chen, Y., Wang, Y., Hu, H., Liang, Y., and Cong, J. (2017, January 18–22). Automated systolic array architecture synthesis for high throughput CNN inference on FPGAs. Proceedings of the 54th Annual Design Automation Conference 2017, Austin, TX, USA.10.1145/3061639.3062207 ↗
31. Xiao, Q., Liang, Y., Lu, L., Yan, S., and Tai, Y.W. (2017, January 18–22). Exploring heterogeneous algorithms for accelerating deep convolutional neural networks on FPGAs. Proceedings of the 54th Annual Design Automation Conference 2017, Austin, TX, USA.10.1145/3061639.3062244 ↗
32. Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., and Cong, J. (2015, January 22–24). Optimizing fpga-based accelerator design for deep convolutional neural networks. Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.10.1145/2684746.2689060 ↗
33. Zoph, B., and Le, Q.V. (2016). Neural architecture search with reinforcement learning. arXiv.
34. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18–22). Learning transferable architectures for scalable image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00907 ↗
35. Stamoulis, D., Cai, E., Juan, D.C., and Marculescu, D. (2018, January 19–23). Hyperpower: Power- and memory-constrained hyper-parameter optimization for neural networks. Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany.10.23919/DATE.2018.8341973 ↗
36. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2018, January 18–23). MnasNet: Platform-Aware Neural Architecture Search for Mobile. Proceedings of the CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2019.00293 ↗
37. Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., Jia, Y., and Keutzer, K. (2019, January 15–20). Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01099 ↗
38. Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., and Sun, J. (2020). Single path one-shot neural architecture search with uniform sampling. European Conference on Computer Vision, Springer.10.1007/978-3-030-58517-4_32 ↗
39. Chu, X., Zhang, B., and Xu, R. (2021, January 11–17). Fairnas: Rethinking evaluation fairness of weight sharing neural architecture search. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.01202 ↗
40. Cai, H., Zhu, L., and Han, S. (2018). Proxylessnas: Direct neural architecture search on target task and hardware. arXiv.
41. Stamoulis, D., Ding, R., Wang, D., Lymberopoulos, D., Priyantha, B., Liu, J., and Marculescu, D. (2019). Single-path nas: Designing hardware-efficient convnets in less than 4 hours. arXiv.10.1007/978-3-030-46147-8_29 ↗
42. Wang, K., Liu, Z., Lin, Y., Lin, J., and Han, S. (2019, January 15–20). Haq: Hardware-aware automated quantization with mixed precision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00881 ↗
43. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2019, January 15–20). Mnasnet: Platform-aware neural architecture search for mobile. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00293 ↗
44. (2021, November 20). Google. Edge TPU., Available online: https://cloud.google.com.
45. Hruska, J. (2021, November 20). New Movidius Myriad X VPU Packs a Custom Neural Compute Engine. Available online: https://www.extremetech.com/computing/254772-new-movidiusmyriad-x-vpu-packs-custom-neural-compute-engine.
46. Hruska, J. (2021, November 20). Nvidia’s Jetson Xavier Stuffs Volta Performance into Tiny Form Factor. Available online: https://www.extremetech.com/computing/270681-nvidias-jetsonxavier-stuffs-volta-performance-into-tiny-form-factor.
47. Benmeziane, H., Maghraoui, K.E., Ouarnoughi, H., Niar, S., Wistuba, M., and Wang, N. (2021). A Comprehensive Survey on Hardware-Aware Neural Architecture Search. arXiv.10.24963/ijcai.2021/592 ↗
48. Smithson, S.C., Yang, G., Gross, W.J., and Meyer, B.H. (2016, January 7–10). Neural networks designing neural networks: Multi-objective hyper-parameter optimization. Proceedings of the 35th International Conference on Computer-Aided Design, Austin, TX, USA.10.1145/2966986.2967058 ↗
49. Gordon, A., Eban, E., Nachum, O., Chen, B., Wu, H., Yang, T.J., and Choi, E. (2018, January 18–22). Morphnet: Fast & simple resource-constrained structure learning of deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00171 ↗
50. Dong, J.D., Cheng, A.C., Juan, D.C., Wei, W., and Sun, M. (May, January 30). Ppp-Net: Platform-Aware Progressive Search for Pareto-Optimal Neural Architectures. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
51. Elsken, T., Metzen, J.H., and Hutter, F. (2018). Efficient multi-objective neural architecture search via lamarckian evolution. arXiv.10.1007/978-3-030-05318-5_3 ↗
52. Zhou, Y., Ebrahimi, S., Arık, S.Ö., Yu, H., Liu, H., and Diamos, G. (2018). Resource-efficient neural architect. arXiv.
53. Hsu, C.H., Chang, S.H., Liang, J.H., Chou, H.P., Liu, C.H., Chang, S.C., Pan, J.Y., Chen, Y.T., Wei, W., and Juan, D.C. (2018). Monas: Multi-objective neural architecture search using reinforcement learning. arXiv.
54. Automated and lightweight network design via random search for remote sensing image scene classification10.5194/isprs-archives-XLIII-B2-2020-1217-2020 ↗
55. Zhang, L.L., Yang, Y., Jiang, Y., Zhu, W., and Liu, Y. (2020, January 14–19). Fast hardware-aware neural architecture search. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00354 ↗
56. Gupta, S., and Akin, B. (2020). Accelerator-aware neural network design using automl. arXiv.
57. Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S. (2019). Once-for-all: Train one network and specialize it for efficient deployment. arXiv.
58. Laube, K.A., and Zell, A. (2019, January 14–19). Shufflenasnets: Efficient cnn models through modified efficient neural architecture search. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.10.1109/IJCNN.2019.8852294 ↗
59. Woo, J., Peng, X., and Yu, S. (2018, January 27–30). Design considerations of selector device in cross-point RRAM array for neuromorphic computing. Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy.10.1109/ISCAS.2018.8351735 ↗
60. Ni, L., Liu, Z., Song, W., Yang, J.J., Yu, H., Wang, K., and Wang, Y. (2017, January 10–12). An energy-efficient and high-throughput bitwise CNN on sneak-path-free digital ReRAM crossbar. Proceedings of the 2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), Monterey, CA, USA.10.1109/ISLPED.2017.8009177 ↗