journal article·2021-11-24Open access

AxP: A HW-SW Co-Design Pipeline for Energy-Efficient Approximated ConvNets via Associative Matching

Luca Mocerino, Andrea Calimera

Applied Sciences· Volume 11· Issue 23

Abstract

The reduction in energy consumption is key for deep neural networks (DNNs) to ensure usability and reliability, whether they are deployed on low-power end-nodes with limited resources or high-performance platforms that serve large pools of users. Leveraging the over-parametrization shown by many DNN models, convolutional neural networks (ConvNets) in particular, energy efficiency can be improved substantially preserving the model accuracy. The solution proposed in this work exploits the intrinsic redundancy of ConvNets to maximize the reuse of partial arithmetic results during the inference stages. Specifically, the weight-set of a given ConvNet is discretized through a clustering procedure such that the largest possible number of inner multiplications fall into predefined bins; this allows an off-line computation of the most frequent results, which in turn can be stored locally and retrieved when needed during the forward pass. Such a reuse mechanism leads to remarkable energy savings with the aid of a custom processing element (PE) that integrates an associative memory with a standard floating-point unit (FPU). Moreover, the adoption of an approximate associative rule based on a partial bit-match increases the hit rate over the pre-computed results, maximizing the energy reduction even further. Results collected on a set of ConvNets trained for computer vision and speech processing tasks reveal that the proposed associative-based hw-sw co-design achieves up to 77% in energy savings with less than 1% in accuracy loss.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Neural Network Applications · Parallel Computing and Optimization Techniques · Ferroelectric and Negative Capacitance Devices

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Control and Computer Engineering, Politecnico di Torino, 10129 Torino, Italy

References (56)

Show source-provided references
1. A survey of deep learning methods and software tools for image classification and object detection10.1134/S1054661816010065 ↗
2. A survey of active learning algorithms for supervised remote sensing image classification10.1109/JSTSP.2011.2139193 ↗
3. Convolutional neural networks for speech recognition10.1109/TASLP.2014.2339736 ↗
4. A CNN-assisted enhanced audio signal processing for speech emotion recognition
5. Lopez, M.M., and Kalita, J. (2017). Deep Learning applied to NLP. arXiv.
6. Babić, K., Martinčić-Ipšić, S., and Meštrović, A. (2020). Survey of Neural Text Representation Models. Information, 11.10.3390/info11110511 ↗
7. Jouppi, N.P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., and Borchers, A. (2017, January 24–28). In-datacenter performance analysis of a tensor processing unit. Proceedings of the 44th Annual International Symposium on Computer Architecture, Toronto, ON, Canada.10.1145/3079856.3080246 ↗
8. Lu, Z., Rallapalli, S., Chan, K., and La Porta, T. (2017, January 23–27). Modeling the resource requirements of convolutional neural networks on mobile devices. Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA.10.1145/3123266.3123389 ↗
9. Short floating-point representation for convolutional neural network inference
10. Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D.T., Jammalamadaka, N., Huang, J., and Yuen, H. (2019). A study of BFLOAT16 for deep learning training. arXiv.
11. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18–23). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00286 ↗
12. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y. (2016). Quantized neural networks: Training neural networks with low precision weights and activations. arXiv.
13. Fast and accurate inference on microcontrollers with boosted cooperative convolutional neural networks (bc-net)10.1109/TCSI.2020.3039116 ↗
14. Hashemi, S., Bahar, R.I., and Reda, S. (2015, January 2–6). DRUM: A dynamic range unbiased multiplier for approximate applications. Proceedings of the 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), Austin, TX, USA.10.1109/ICCAD.2015.7372600 ↗
15. Camus, V., Schlachter, J., Enz, C., Gautschi, M., and Gurkaynak, F.K. (2016, January 12–15). Approximate 32-bit floating-point unit design with 53% power-area product reduction. Proceedings of the ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference, Lausanne, Switzerland.10.1109/ESSCIRC.2016.7598342 ↗
16. Li, S., Park, J., and Tang, P.T.P. (2017). Enabling sparse winograd convolution by native pruning. arXiv.
17. Imani, M., Rahimi, A., and Rosing, T.S. (2016, January 14–18). Resistive configurable associative memory for approximate computing. Proceedings of the 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany.10.3850/9783981537079_0454 ↗
18. Razlighi, M.S., Imani, M., Koushanfar, F., and Rosing, T. (2017, January 27–31). Looknn: Neural network with no multiplication. Proceedings of the 2017 Design, Automation & Test in Europe Conference & Exhibition (DATE), Lausanne, Switzerland.10.23919/DATE.2017.7927280 ↗
19. Imani, M., Masich, M., Peroni, D., Wang, P., and Rosing, T. (2018, January 22–25). CANNA: Neural network acceleration using configurable approximation on GPGPU. Proceedings of the 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC), Jeju Island, Korea.10.1109/ASPDAC.2018.8297401 ↗
20. Associative memories and processors: The exact match paradigm
21. Mocerino, L., Tenace, V., and Calimera, A. (2019, January 25–29). Energy-Efficient Convolutional Neural Networks via Recurrent Data Reuse. Proceedings of the 2019 Design, Automation Test in Europe Conference Exhibition (DATE), Florence, Italy.10.23919/DATE.2019.8714880 ↗
22. Peroni, D., Imani, M., Nejatollahi, H., Dutt, N., and Rosing, T. (2019, January 2–6). ARGA: Approximate Reuse for GPGPU Acceleration. Proceedings of the 56th Annual Design Automation Conference 2019 (DAC ’19), Las Vegas, NV, USA.
23. Peroni, D., Imani, M., and Rosing, T. (2019, January 21–24). ALook: Adaptive Lookup for GPGPU Acceleration. Proceedings of the 24th Asia and South Pacific Design Automation Conference (ASPDAC ’19), Tokyo, Japan.10.1145/3287624.3287634 ↗
24. A survey of FPGA-based accelerators for convolutional neural networks10.1007/s00521-018-3761-1 ↗
25. Locke, K. (2011). Parameterizable Content-Addressable Memory, Xilinx. Xilinx Application Note XAPP1151.
26. Irfan, M., Ullah, Z., and CC Cheung, R. (2019). Zi-CAM: A power and resource efficient binary content-addressable memory on FPGAs. Electronics, 8.10.3390/electronics8050584 ↗
27. Sainath, T.N., and Parada, C. (2015, January 6–10). Convolutional neural networks for small-footprint keyword spotting. Proceedings of the Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany.10.21437/Interspeech.2015-352 ↗
28. Deep neural network approximation for custom hardware: Where we’ve been, where we’re going10.1145/3214306 ↗
29. A survey of techniques for approximate computing
30. Mocerino, L., and Calimera, A. (September, January 31). TentacleNet: A pseudo-ensemble template for accurate binary convolutional neural networks. Proceedings of the 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genoa, Italy.10.1109/AICAS48895.2020.9073982 ↗
31. Choi, Y., Choi, J., El-Khamy, M., and Lee, J. (2020, January 14–19). Data-free network quantization with adversarial knowledge distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00363 ↗
32. Bhardwaj, K., Suda, N., and Marculescu, R. (2019). Dream distillation: A data-independent model compression framework. arXiv.
33. Hybrid 8-bit floating point (HFP8) training and inference for deep neural networks
34. Johnson, J. (2018). Rethinking floating point for deep learning. arXiv.
35. Wu, J., Leng, C., Wang, Y., Hu, Q., and Cheng, J. (July, January 25). Quantized convolutional neural networks for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
36. Lai, L., Suda, N., and Chandra, V. (2018). Cmsis-nn: Efficient neural network kernels for arm cortex-m cpus. arXiv.
37. Lin, D.D., and Talathi, S.S. (2016). Overcoming challenges in fixed point training of deep convolutional networks. arXiv.
38. Ma, D., Yin, X., Niemier, M., Hu, X.S., and Jiao, X. (2020, January 7–9). AxR-NN: Approximate Computation Reuse for Energy-Efficient Convolutional Neural Networks. Proceedings of the 2020 on Great Lakes Symposium on VLSI, Virtual Event, China.10.1145/3386263.3407595 ↗
39. Jiang, H., Liu, L., Lombardi, F., and Han, J. (2019). Approximate arithmetic circuits: Design and evaluation. Approximate Circuits, Springer.10.1007/978-3-319-99322-5_4 ↗
40. Zhang, Q., Wang, T., Tian, Y., Yuan, F., and Xu, Q. (2015, January 9–13). ApproxANN: An approximate computing framework for artificial neural network. Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE), Grenoble, France.10.7873/DATE.2015.0618 ↗
41. Yang, X., Pu, J., Rister, B.B., Bhagdikar, N., Richardson, S., Kvatinsky, S., Ragan-Kelley, J., Pedram, A., and Horowitz, M. (2016). A systematic approach to blocking convolutional neural networks. arXiv.
42. Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices10.1109/JETCAS.2019.2910232 ↗
43. Imani, M., Peroni, D., Kim, Y., Rahimi, A., and Simunic, T. (2017, January 27–31). Efficient neural network acceleration on GPGPU using content addressable memory. Proceedings of the 2017 Design, Automation & Test in Europe Conference & Exhibition (DATE), Lausanne, Switzerland.10.23919/DATE.2017.7927141 ↗
44. Yin, X., Niemier, M., and Hu, X.S. (2017, January 27–31). Design and benchmarking of ferroelectric FET based TCAM. Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE), Lausanne, Switzerland.10.23919/DATE.2017.7927219 ↗
45. Jiao, X., Akhlaghi, V., Jiang, Y., and Gupta, R.K. (2018, January 19–23). Energy-efficient neural networks using approximate computation reuse. Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, Germany.10.23919/DATE.2018.8342202 ↗
46. Blalock, D., and Guttag, J. (2021). Multiplying Matrices Without Multiplying. arXiv.
47. The data model concept in statistical mapping
48. LeCun, Y., Cortes, C., and Burges, C. (2010). MNIST Handwritten Digit Database, AT&T Labs.
49. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition10.1016/j.neunet.2012.02.016 ↗
50. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
51. (2021, August 28). Google Speech Commands PyTorch. Available online: https://github.com/adiyoss/GCommandsPytorch.
52. Designing custom arithmetic data paths with FloPoCo10.1109/MDT.2011.44 ↗
53. Gupta, N., Makosiej, A., Vladimirescu, A., Amara, A., and Anghel, C. (2017, January 11–14). 1.56 GHz/0.9 V energy-efficient reconfigurable CAM/SRAM using 6T-CMOS bitcell. Proceedings of the ESSCIRC 2017—43rd IEEE European Solid State Circuits Conference, Leuven, Belgium.10.1109/ESSCIRC.2017.8094589 ↗
54. Balasubramonian, R., Kahng, A.B., Muralimanohar, N., Shafiee, A., and Srinivas, V. (2017, January 2). CACTI 7: New tools for interconnect exploration in innovative off-chip memories. Proceedings of the ACM Transactions on Architecture and Code Optimization (TACO), New York, NY, USA.10.1145/3085572 ↗
55. Parallelization of the K-Means++ Clustering Algorithm10.18280/isi.260106 ↗
56. Shahrezaei, M.H., and Tavoli, R. (2019). Parallelization of Kmeans++ using CUDA. arXiv.