journal article·2024-10-29

Integrated Artificial Neural Network with Trainable Activation Function Enabled by Topological Insulator-Based Spin–Orbit Torque Devices

Puyang Huang, Xinqi Liu, Yue Xin, Yu Gu, Albert Lee, Yifan Zhang, Zhuo Xu, Peng Chen, Yu Zhang, Weijie Deng, Guoqiang Yu, Di Wu, Zhongkai Liu, Qi Yao, Yumeng Yang, Zhifeng Zhu, Xufeng Kou

ACS Nano· Volume 18· Issue 43
Publisher page

Abstract

Nonvolatile memristors offer a salient platform for artificial neural network (ANN), yet the integration of different function and algorithm blocks into one hardware system remains challenging. Here we demonstrate the brain-like synaptic (SOT-S) and neuronal (SOT-N) functions in the Bi 2 Te 3 /CrTe 2 heterostructure-based spin–orbit torque (SOT) device. The SOT-S unit exhibits highly linear and symmetrical long-term potentiation/depression process, resulting in a fast-training of the MNIST data set with the classification accuracy above 90%. Meanwhile, the Sigmoid-shape transition curve inherited in the SOT-N cell replaces the software-based activation function block, hence reducing the system complexity. On this basis, we employ a serial-connected, voltage-mode sensing ANN architecture to enhance the vector-matrix multiplication signal strength with low reading error of 0.61% while simplifying the peripheral circuitry. Furthermore, the trainable activation function of SOT-N enables the implementation of the Batch Normalization algorithm and activation operation within one clock cycle, which bring about improved on/off-chip training performance close to the ideal baseline.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Advanced Memory and Neural Computing · Neural Networks and Reservoir Computing · Random lasers and scattering media

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Information Science and Technology

ShanghaiTech University

School of Physical Science and Technology

ShanghaiTech Laboratory for Topological Physics

Suzhou Inston Technology Co., Ltd.

Beijing National Laboratory for Condensed Matter

Physics Institute of Physics, Chinese Academy of Sciences

Funding

National Key Research and Development Program of China · Science and Technology Commission of Shanghai Municipality · National Natural Science Foundation of China · Shanghai Rising-Star Program · ShanghaiTech Quantum Device and Soft Matter Nano-fabrication Labs · Shanghai Engineering Research Center of Energy Efficient and Custom AI IC · State Key Laboratory of Infrared Physics

References (65)

Show source-provided references
1. 10.1038/s41586-019-1677-210.1038/s41586-019-1677-2 ↗
2. 10.1109/5.5835610.1109/5.58356 ↗
3. 10.1038/s41467-023-37180-x10.1038/s41467-023-37180-x ↗
4. 10.1038/s41586-021-04362-w10.1038/s41586-021-04362-w ↗
5. 10.1038/s41928-020-0448-210.1038/s41928-020-0448-2 ↗
6. 10.1109/JETCAS.2019.293314810.1109/JETCAS.2019.2933148 ↗
7. Agarwal, S.; Plimpton, S. J.; Hughart, D. R.; Hsia, A. H.; Richter, I.; Cox, J. A.; James, C. D.; Marinella, M. J. In Resistive Memory Device Requirements for a Neural Algorithm Accelerator, International Joint Conference on Neural Networks (IJCNN), 24–29 July; IEEE, 2016; pp 929–938.10.1109/IJCNN.2016.7727298 ↗
8. 10.1109/TVLSI.2021.306354310.1109/TVLSI.2021.3063543 ↗
9. Yu, Z.; Leroux, N.; Neftci, E. In Training-to-Learn with Memristive Devices, 2022 International Electron Devices Meeting (IEDM), 3–7 Dec; IEEE, 2022.10.1109/IEDM45625.2022.10019455 ↗
10. 10.1016/j.neunet.2021.01.02610.1016/j.neunet.2021.01.026 ↗
11. 10.1007/s00521-012-0858-910.1007/s00521-012-0858-9 ↗
12. 10.1016/j.neucom.2014.06.01810.1016/j.neucom.2014.06.018 ↗
13. 10.1109/JSSC.2018.288091810.1109/JSSC.2018.2880918 ↗
14. 10.1109/JSSC.2019.289973010.1109/JSSC.2019.2899730 ↗
15. 10.1109/TVLSI.2018.288219410.1109/TVLSI.2018.2882194 ↗
16. Song, J.; Cho, Y.; Park, J. S.; Jang, J. W.; Lee, S.; Song, J. H.; Lee, J. G.; Kang, I. In 7.1 An 11.5TOPS/W 1024-MAC Butterfly Structure Dual-Core Sparsity-Aware Neural Processing Unit in 8nm Flagship Mobile SoC, IEEE International Solid- State Circuits Conference-(ISSCC); IEEE, 2019.10.1109/ISSCC.2019.8662476 ↗
17. 10.1109/MM.2011.8910.1109/MM.2011.89 ↗
18. 10.1038/nature1444110.1038/nature14441 ↗
19. 10.1038/s41586-018-0180-510.1038/s41586-018-0180-5 ↗
20. 10.1038/s41586-022-04992-810.1038/s41586-022-04992-8 ↗
21. 10.1038/s41928-018-0054-810.1038/s41928-018-0054-8 ↗
22. Luo, Y.; Luc, Y. C.; Yu, S. In A FeRAM Based Volatile/Non-Volatile Dual-Mode Buffer Memory for Deep Neural Network Training, Design, Automation & Test in Europe Conference & Exhibition (DATE); IEEE, 2021.10.23919/DATE51398.2021.9474180 ↗
23. 10.1038/s41928-018-0023-210.1038/s41928-018-0023-2 ↗
24. 10.1038/s41586-020-1942-410.1038/s41586-020-1942-4 ↗
25. 10.1038/s41467-018-04482-410.1038/s41467-018-04482-4 ↗
26. 10.1038/s41586-021-04196-610.1038/s41586-021-04196-6 ↗
27. Patil, A. D.; Hua, H.; Gonugondla, S.; Kang, M.; Shanbhag, N. R. In An MRAM-Based Deep In-Memory Architecture for Deep Neural Networks, IEEE International Symposium on Circuits and Systems (ISCAS), 26–29 May; IEEE, 2019.10.1109/ISCAS.2019.8702206 ↗
28. 10.1038/s41427-021-00282-310.1038/s41427-021-00282-3 ↗
29. 10.1038/s41928-020-0385-010.1038/s41928-020-0385-0 ↗
30. 10.1038/nphys196810.1038/nphys1968 ↗
31. 10.1038/s41586-018-0632-y10.1038/s41586-018-0632-y ↗
32. 10.1038/nmat382310.1038/nmat3823 ↗
33. 10.1002/adfm.20210787010.1002/adfm.202107870 ↗
34. 10.1038/s41586-020-2061-y10.1038/s41586-020-2061-y ↗
35. 10.1038/s41563-021-01138-510.1038/s41563-021-01138-5 ↗
36. 10.1038/nnano.2015.29410.1038/nnano.2015.294 ↗
37. 10.1021/accountsmr.2c0012310.1021/accountsmr.2c00123 ↗
38. Shao, Q.; Wu, H.; Pan, Q.; Zhang, P.; Pan, L.; Wong, K.; Che, X.; Wang, K. L. In Room Temperature Highly Efficient Topological Insulator/Mo/CoFeB Spin-Orbit Torque Memory with Perpendicular Magnetic Anisotropy, IEEE International Electron Devices Meeting (IEDM); IEEE, 2018.10.1109/IEDM.2018.8614499 ↗
39. 10.1002/adma.20210173010.1002/adma.202101730 ↗
40. 10.1038/s41467-020-20692-110.1038/s41467-020-20692-1 ↗
41. 10.1038/s41598-021-04159-x10.1038/s41598-021-04159-x ↗
42. 10.1145/330410510.1145/3304105 ↗
43. 10.1038/s41467-020-16108-910.1038/s41467-020-16108-9 ↗
44. 10.1109/LED.2015.241834210.1109/LED.2015.2418342 ↗
45. 10.1038/s41467-022-30432-210.1038/s41467-022-30432-2 ↗
46. 10.1038/nature2239110.1038/nature22391 ↗
47. 10.1038/s41586-018-0626-910.1038/s41586-018-0626-9 ↗
48. 10.1002/adfm.20230445410.1002/adfm.202304454 ↗
49. 10.1007/s00521-021-05699-z10.1007/s00521-021-05699-z ↗
50. 10.1038/s41586-022-04714-010.1038/s41586-022-04714-0 ↗
51. 10.1038/s41928-019-0270-x10.1038/s41928-019-0270-x ↗
52. 10.1038/s41928-020-0435-710.1038/s41928-020-0435-7 ↗
53. Sergey, I.; Christian, S. In Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, Proceedings of the 32nd International Conference on Machine Learning; PMLR, 2015; pp 448–456.
54. 10.1109/LES.2020.297505510.1109/LES.2020.2975055 ↗
55. 10.3390/ma1301016610.3390/ma13010166 ↗
56. 10.1109/JPROC.2017.266578110.1109/JPROC.2017.2665781 ↗
57. 10.1109/JPROC.2003.81170210.1109/JPROC.2003.811702 ↗
58. 10.1038/s41586-022-04768-010.1038/s41586-022-04768-0 ↗
59. Saligram, R.; Datta, S.; Raychowdhury, A. In CryoMem: A 4K-300K 1.3 GHz eDRAM Macro with Hybrid 2T-Gain-Cell in a 28nm Logic Process for Cryogenic Applications, IEEE Custom Integrated Circuits Conference (CICC); IEEE, 2021.10.1109/CICC51472.2021.9431527 ↗
60. 10.1126/science.ade765610.1126/science.ade7656 ↗
61. 10.1109/JIOT.2020.296495110.1109/JIOT.2020.2964951 ↗
62. 10.1103/PhysRevLett.127.21720310.1103/PhysRevLett.127.217203 ↗
63. 10.1038/s41467-023-40714-y10.1038/s41467-023-40714-y ↗
64. 10.1038/s41598-022-11199-410.1038/s41598-022-11199-4 ↗
65. Liu, Y.; Lee, A.; Qian, K.; Zhang, P.; He, H.; Ren, Z.; Cheung, S. K.; Li, Y.; Zhang, X.; Ma, Z. Cryogenic In-Memory Computing Using Tunable Chiral Edge States. 2022, arXiv:2209.09443. arXiv.org e-Printarchive. https://arxiv.org/abs/2209.09443. (accessed May 30, 2024).