journal article·2021-07-05Open access

MLUTNet: A Neural Network for Memory Based Reconfigurable Logic Device Architecture

Xuechen Zang, Shigetoshi Nakatake

Applied Sciences· Volume 11· Issue 13

Abstract

Neural networks have been widely used and implemented on various hardware platforms, but high computational costs and low similarity of network structures relative to hardware structures are often obstacles to research. In this paper, we propose a novel neural network in combination with the structural features of a recently proposed memory-based programmable logic device, compare it with the standard structure, and test it on common datasets with full and binary precision, respectively. The experimental results reveal that the new structured network can provide almost consistent full-precision performance and binary-precision performance ranging from 61.0% to 78.8% after using sparser connections and about 50% reduction in the size of the weight matrix.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Memory and Neural Computing · Advanced Neural Network Applications · Neural Networks and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

The Graduate School of Environmental Engineering, Hibikino Campus, The University of Kitakyushu, Kitakyushu 808-0135, Fukuoka, Japan

References (15)

Show source-provided references
1. Review of Deep Learning Algorithms and Architectures10.1109/ACCESS.2019.2912200 ↗
2. Alzubaidi, L., Zhang, J., Humaidi, A.J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M.A., Al-Amidie, M., and Farhan, L. (2021). Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data, 8.10.1186/s40537-021-00444-8 ↗
3. Li, F., Zhang, B., and Liu, B. (2016). Ternary Weight Networks. arXiv.
4. Binary neural networks: A survey10.1016/j.patcog.2020.107281 ↗
5. Yonekawa, H., and Nakahara, H. (June, January 29). On-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an FPGA. Proceedings of the 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Lake Buena Vista, FL, USA.10.1109/IPDPSW.2017.95 ↗
6. Yu, S., Li, Z., Chen, P.Y., Wu, H., Gao, B., Wang, D., Wu, W., and Qian, H. (2016, January 3–7). Binary neural network with 16 Mb RRAM macro chip for classification and online training. Proceedings of the 2016 IEEE International Electron Devices Meeting (IEDM), San Francisco, CA, USA.10.1109/IEDM.2016.7838429 ↗
7. Wang, S., Higami, Y., Takahashi, H., Sato, M., Katsu, M., and Sekiguchi, S. (2017, January 27–30). Testing of Interconnect Defects in Memory Based Reconfigurable Logic Device (MRLD). Proceedings of the 2017 IEEE 26th Asian Test Symposium (ATS), Taipei, Taiwan.10.1109/ATS.2017.16 ↗
8. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., and Bengio, Y. (2016). Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or −1. arXiv.
9. Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. (2016). XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. arXiv.10.1007/978-3-319-46493-0_32 ↗
10. Geoffrey Hinton, N.S., and Swersky, K. (2021, May 13). Neural Networks for Machine Learning, Lecture 6a, Overview of Mini-Batch Gradient Descent. Available online: http://www.cs.toronto.edu/~hinton/coursera/lecture6/lec6.pdf.
11. Loshchilov, I., and Hutter, F. (2017). SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv.
12. Nair, V., and Hinton, G.E. (2010, January 21–24). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, Haifa, Israel.
13. LeCun, Y., and Cortes, C. (2021, May 13). MNISThandwritten Digit Database. Available online: http://yann.lecun.com/exdb/mnist/.
14. Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., and Ha, D. (2018). Deep Learning for Classical Japanese Literature. arXiv.
15. Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv.