Memristive crossbar arrays for brain-inspired computing
Source abstract
Abstract
With their working mechanisms based on ion migration, the switching dynamics and electrical behaviour of memristive devices resemble those of synapses and neurons, making these devices promising candidates for brain-inspired computing. Built into large-scale crossbar arrays to form neural networks, they perform efficient in-memory computing with massive parallelism by directly using physical laws. The dynamical interactions between artificial synapses and neurons equip the networks with both supervised and unsupervised learning capabilities. Moreover, their ability to interface with analogue signals from sensors without analogue/digital conversions reduces the processing time and energy overhead. Although numerous simulations have indicated the potential of these networks for brain-inspired computing, experimental implementation of large-scale memristive arrays is still in its infancy. This Review looks at the progress, challenges and possible solutions for efficient brain-inspired computation with memristive implementations, both as accelerators for deep learning and as building blocks for spiking neural networks.
Source: europepmc · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Photoreceptor and optogenetics research · Neural dynamics and brain function
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Moore, G. E. Cramming more components onto integrated circuits. Electron. Mag 38, 114–117 (1965).
2. von Neumann, J. First draft of a report on the EDVAC. IEEE Ann. Hist. Comput. 15, 27–75 (1993).This is an exact copy of the original typescript draft written by von Neumann in 1945, with typographical errors corrected.10.1109/85.238389 ↗
3. NVIDIA Launches the World's First Graphics Processing Unit: GeForce 256 http://www.nvidia.com/object/IO_20020111_5424.html (NVIDIA, accessed 30 July 2018).
4. Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In 44th Int. Symp. Computer Architecture (ISCA) 1–12 (ACM, 2017).
5. Kautz, W. H. Cellular logic-in-memory arrays. IEEE Trans. Comput. C 18, 719–727 (1969).
6. Wolf, S. A. et al. Spintronics: a spin-based electronics vision for the future. Science 294, 1488–1495 (2001).10.1126/science.1065389 ↗
7. Ovshinsky, S. R. Reversible electrical switching phenomena in disordered structures. Phys. Rev. Lett. 21, 1450–1453 (1968).10.1103/PhysRevLett.21.1450 ↗
8. Yang, J. J. et al. Memristive switching mechanism for metal/oxide/metal nanodevices. Nat. Nanotechnol. 3, 429–433 (2008).10.1038/nnano.2008.160 ↗
9. Mikolajick, T. et al. FeRAM technology for high density applications. Microelectron. Reliab. 41, 947–950 (2001).10.1016/S0026-2714(01)00049-X ↗
10. Chua, L. Memristor—the missing circuit element. IEEE Trans. Circuit Theory 18, 507–519 (1971).10.1109/TCT.1971.1083337 ↗
11. Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008).10.1038/nature06932 ↗
12. Pi, S. et al. Memristor crossbar arrays with 6-nm half-pitch and 2-nm critical dimension. Nat. Nanotechnol. 14, 35–39 (2019).10.1038/s41565-018-0302-0 ↗
13. Choi, B. J. et al. High-speed and low-energy nitride memristors. Adv. Funct. Mater. 26, 5290–5296 (2016).10.1002/adfm.201600680 ↗
14. Hu, M., Strachan, J. P., Li, Z. & Williams, S. R. Dot-product engine as computing memory to accelerate machine learning algorithms. In 17th Int. Symp. Quality Electronic Design (ISQED) 374–379 (IEEE, 2016).10.1109/ISQED.2016.7479230 ↗
15. Yao, P. et al. Face classification using electronic synapses. Nat. Commun. 8, 15199 (2017).10.1038/ncomms15199 ↗
16. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).10.1038/s41928-017-0002-z ↗
17. Li, C. et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat. Commun. 9, 2385 (2018).10.1038/s41467-018-04484-2 ↗
18. Li, C. et al. Long short-term memory networks in memristor crossbar arrays. Nat. Mach. Intell. 1, 49–57 (2019).10.1038/s42256-018-0001-4 ↗
19. Hu, M. et al. Memristor-based analog computation and neural network classification with a dot product engine. Adv. Mater. 30, 1705914 (2018).10.1002/adma.201705914 ↗
20. Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal–oxide memristors. Nature 521, 61–64 (2015).10.1038/nature14441 ↗
21. Sheridan, P. M. et al. Sparse coding with memristor networks. Nat. Nanotechnol. 12, 784–789 (2017).10.1038/nnano.2017.83 ↗
22. Wang, Z. et al. Capacitive neural network with neuro-transistors. Nat. Commun. 9, 3208 (2018).10.1038/s41467-018-05677-5 ↗
23. Waser, R. & Aono, M. Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).10.1038/nmat2023 ↗
24. Valov, I., Waser, R., Jameson, J. R. & Kozicki, M. N. Electrochemical metallization memories—fundamentals, applications, prospects. Nanotechnology 22, 254003 (2011).10.1088/0957-4484/22/25/254003 ↗
25. Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nat. Nanotechnol. 8, 13–24 (2013).10.1038/nnano.2012.240 ↗
26. Jeong, D. S., Kim, K. M., Kim, S., Choi, B. J. & Hwang, C. S. Memristors for energy‐efficient new computing paradigms. Adv. Electron. Mater 2, 1600090 (2016).10.1002/aelm.201600090 ↗
27. Burr, G. W. et al. Neuromorphic computing using non-volatile memory. Adv. Phys. X 2, 89–124 (2017).
28. Lee, J. & Lu, W. D. On‐demand reconfiguration of nanomaterials: when electronics meets ionics. Adv. Mater. 30, 1702770 (2018).10.1002/adma.201702770 ↗
29. Yang, Y. & Huang, R. Probing memristive switching in nanoionic devices. Nat. Electron. 1, 274–287 (2018).10.1038/s41928-018-0069-1 ↗
30. Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).10.1038/nature16961 ↗
31. Pickett, M. D., Medeiros-Ribeiro, G. & Williams, R. S. A scalable neuristor built with Mott memristors. Nat. Mater. 12, 114–117 (2013).10.1038/nmat3510 ↗
32. Tuma, T., Pantazi, A., Le Gallo, M., Sebastian, A. & Eleftheriou, E. Stochastic phase-change neurons. Nat. Nanotechnol. 11, 693–699 (2016).10.1038/nnano.2016.70 ↗
33. Wang, Z. et al. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat. Mater. 16, 101–108 (2017).10.1038/nmat4756 ↗
34. Yang, J. J. et al. High switching endurance in TaOx memristive devices. Appl. Phys. Lett. 97, 232102 (2010).10.1063/1.3524521 ↗
35. Wright, C. D., Hosseini, P. & Vazquez Diosdado, J. A. Beyond von-Neumann computing with nanoscale phase-change memory devices. Adv. Funct. Mater. 23, 2248–2254 (2013).10.1002/adfm.201202383 ↗
36. Jiang, H. et al. Sub-10 nm Ta channel responsible for superior performance of a HfO2 memristor. Sci. Rep. 6, 28525 (2016).10.1038/srep28525 ↗
37. Choi, S. et al. SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations. Nat. Mater. 17, 335–340 (2018).10.1038/s41563-017-0001-5 ↗
38. Chen, J.-Y. et al. Dynamic evolution of conducting nanofilament in resistive switching memories. Nano Lett. 13, 3671–3677 (2013).10.1021/nl4015638 ↗
39. van de Burgt, Y. et al. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. Nat. Mater. 16, 414–418 (2017).10.1038/nmat4856 ↗
40. Boniardi, M. & Ielmini, D. Physical origin of the resistance drift exponent in amorphous phase change materials. Appl. Phys. Lett. 98, 243506 (2011).10.1063/1.3599559 ↗
41. Yi, W. et al. Quantized conductance coincides with state instability and excess noise in tantalum oxide memristors. Nat. Commun. 7, 11142 (2016).10.1038/ncomms11142 ↗
42. Lastras-Montaño, M. A. & Cheng, K. T. Resistive random-access memory based on ratioed memristors. Nat. Electron. 1, 466–472 (2018).10.1038/s41928-018-0115-z ↗
43. Alibart, F., Gao, L., Hoskins, B. D. & Strukov, D. B. High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology 23, 075201 (2012).10.1088/0957-4484/23/7/075201 ↗
44. Ambrogio, S. et al. Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558, 60–67 (2018).10.1038/s41586-018-0180-5 ↗
45. Shafiee, A. et al. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. in 2016 ACM/IEEE 43rd Int. Symp. Comp. Archit. (ISCA) 14–26 (IEEE, 2016).10.1109/ISCA.2016.12 ↗
46. Jiang, H. et al. Pulse-width modulation based dot-product engine for neuromorphic computing system using memristor crossbar array. In 2018 IEEE Int. Symp. Circuits and Systems (ISCAS) https://doi.org/10.1109/ISCAS.2018.8351276 (IEEE, 2018).10.1109/ISCAS.2018.8351276 ↗
47. Burr, G. W. et al. Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element. IEEE Trans. Electron Devices 62, 3498–3507 (2015).10.1109/TED.2015.2439635 ↗
48. Woo, J. & Yu, S. Resistive memory-based analog synapse: the pursuit for linear and symmetric weight update. IEEE Nanotechnol. Mag. 12, 36–44 (2018).10.1109/MNANO.2018.2844902 ↗
49. Gokmen, T. & Vlasov, Y. Acceleration of deep neural network training with resistive cross-point devices: design considerations. Front. Neurosci. 10, 33 (2016).10.3389/fnins.2016.00333 ↗
50. Suri, M. et al. Phase change memory as synapse for ultra-dense neuromorphic systems: application to complex visual pattern extraction. In 2011 Int. Electron Devices Meeting (IEDM) 4.4.1–4.4.4 (IEEE, 2011).10.1109/IEDM.2011.6131488 ↗
51. Hsu, C.-W. et al. 3D vertical TaOx/TiO2 RRAM with over 103 self-rectifying ratio and sub-µA operating current. In 2013 Int. Electron Devices Meeting (IEDM) 10.4.1–10.4.4 (IEEE, 2013).
52. Jang, J.-W., Park, S., Burr, G. W., Hwang, H. & Jeong, Y.-H. Optimization of conductance change in Pr(1−x)CaxMnO3-based synaptic devices for neuromorphic systems. IEEE Electron Device Lett. 36, 457–459 (2015).10.1109/LED.2015.2418342 ↗
53. Merrikh-Bayat, F. et al. High-performance mixed-signal neurocomputing with nanoscale floating-gate memory cell arrays. IEEE Trans. Neural Netw. Learn. Syst. 29, 4782–4790 (2018).10.1109/TNNLS.2017.2778940 ↗
54. Li, C. et al. Three-dimensional crossbar arrays of self-rectifying Si/SiO2/Si memristors. Nat. Commun. 8, 15666 (2017).10.1038/ncomms15666 ↗
55. Woo, J., Peng, X., & Yu, S. Design considerations of selector device in cross-point RRAM array for neuromorphic computing. In 2018 IEEE Int. Symp. Circuits and Systems (ISCAS) https://doi.org/10.1109/ISCAS.2018.8351735 (IEEE, 2018).10.1109/ISCAS.2018.8351735 ↗
56. Wang, W. et al. Learning of spatiotemporal patterns in a spiking neural network with resistive switching synapses. Sci. Adv. 4, eaat4752 (2018).10.1126/sciadv.aat4752 ↗
57. Hebb, D. O. The Organization of Behavior. (Wiley, New York, 1949).
58. Sourikopoulos, I. et al. A 4-fJ/spike artificial neuron in 65 nm CMOS technology. Front. Neurosci. 11, 123 (2017).10.3389/fnins.2017.00123 ↗
59. Indiveri, G., Chicca, E. & Douglas, R. J. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neur. Netw. 17, 211–221 (2006).10.1109/TNN.2005.860850 ↗
60. Benjamin, B. V. et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102, 699–716 (2014).10.1109/JPROC.2014.2313565 ↗
61. Furber, S. B., Galluppi, F., Temple, S. & Plana, L. A. The SpiNNaker Project. Proc. IEEE 102, 652–665 (2014).10.1109/JPROC.2014.2304638 ↗
62. Merolla, P. A. et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668–673 (2014).10.1126/science.1254642 ↗
63. Davies, M. Putting the ‘learning’ in machine learning processors: an introduction to the Loihi neuromorphic research chip. Zenodo https://doi.org/10.5281/zenodo.1313406 (2018).10.5281/zenodo.1313406 ↗
64. Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010).10.1021/nl904092h ↗
65. Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011).10.1038/nmat3054 ↗
66. Yu, S., Wu, Y., Jeyasingh, R., Kuzum, D. & Wong, H.-S. P. An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation. IEEE Trans. Elect. Dev. 58, 2729–2737 (2011).10.1109/TED.2011.2147791 ↗
67. Wang, Z. Q. et al. Synaptic learning and memory functions achieved using oxygen ion migration/diffusion in an amorphous InGaZnO memristor. Adv. Funct. Mater. 22, 2759–2765 (2012).10.1002/adfm.201103148 ↗
68. La Barbera, S., Vuillaume, D. & Alibart, F. Filamentary switching: synaptic plasticity through device volatility. ACS Nano 9, 941–949 (2015).10.1021/nn506735m ↗
69. Stoliar, P. et al. A leaky‐integrate‐and‐fire neuron analog realized with a Mott insulator. Adv. Funct. Mater. 27, 1604740 (2017).10.1002/adfm.201604740 ↗
70. Al-Shedivat, M., Naous, R., Cauwenberghs, G. & Salama, K. N. Memristors empower spiking neurons with stochasticity. IEEE Trans. Emerg. Sel. Topics Circuits Syst. 5, 242–253 (2015).10.1109/JETCAS.2015.2435512 ↗
71. Mehonic, A. & Kenyon, A. J. Emulating the electrical activity of the neuron using a silicon oxide RRAM cell. Front. Neurosci. 10, 57 (2016).10.3389/fnins.2016.00057 ↗
72. Zhang, X. et al. An artificial neuron based on a threshold switching memristor. IEEE Elect. Dev. Lett. 39, 308–311 (2018).10.1109/LED.2017.2782752 ↗
73. Pantazi, A., Woźniak, S., Tuma, T. & Eleftheriou, E. All-memristive neuromorphic computing with level-tuned neurons. Nanotechnology 27, 355205 (2016).10.1088/0957-4484/27/35/355205 ↗
74. Wang, Z. et al. Fully memristive neural networks for pattern classification with unsupervised learning. Nat. Electron. 1, 137–145 (2018).10.1038/s41928-018-0023-2 ↗
75. Pedretti, G. et al. Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity. Sci. Rep. 7, 5288 (2017).10.1038/s41598-017-05480-0 ↗
76. Kim, S. et al. Experimental demonstration of a second-order memristor and its ability to biorealistically implement synaptic plasticity. Nano Lett. 15, 2203–2211 (2015).10.1021/acs.nanolett.5b00697 ↗
77. Jerry, M., Parihar, A., Grisafe, B., Raychowdhury, A. & Datta, S. Ultra-low power probabilistic IMT neurons for stochastic sampling machine. In 2017 Symp. VLSI Technology (VLSIT) T186–T187 (IEEE, 2017).10.23919/VLSIT.2017.7998148 ↗
78. Lai, Q. et al. Analog memory capacitor based on field-configurable ion-doped polymers. Appl. Phys. Lett. 95, 213503 (2009).10.1063/1.3268433 ↗
79. Neftci, E., Das, S., Pedroni, B., Kreutz-Delgado, K. & Cauwenberghs, G. Event-driven contrastive divergence for spiking neuromorphic systems. Front. Neurosci. 7, 272 (2014).10.3389/fnins.2013.00272 ↗
80. Xia, Q. et al. Memristor−CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9, 3640–3645 (2009).10.1021/nl901874j ↗
81. Pi, S., Lin, P. & Xia, Q. Cross point arrays of 8 nm × 8 nm memristive devices fabricated with nanoimprint lithography. J. Vacuum Sci. Technol. B 31, 06FA02 (2013).10.1116/1.4827021 ↗
82. Shulaker, M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74–78 (2017).10.1038/nature22994 ↗
83. Jang, J. et al. Vertical cell array using TCAT(Terabit Cell Array Transistor) technology for ultra high density NAND flash memory. In 2009 Symp. VLSI Technology (VLSIT) 192–193 (IEEE, 2009).
84. Clarke, P. Report: TSMC to offer embedded ReRAM in 2019. eeNews http://www.eenewsanalog.com/news/report-tsmc-offer-embedded-reram-2019 (2017).
85. Yoon, J. H. et al. An artificial nociceptor based on a diffusive memristor. Nat. Commun. 9, 417 (2018).10.1038/s41467-017-02572-3 ↗
86. Gao, L. et al. Digital-to-analog and analog-to-digital conversion with metal oxide memristors for ultra-low power computing. In 2013 IEEE/ACM Int. Symp. Nanoscale Architectures (NANOARCH) 19–22 (IEEE, 2013).10.1109/NanoArch.2013.6623031 ↗
87. Pi, S., Ghadiri-Sadrabadi, M., Bardin, J. C. & Xia, Q. Nanoscale memristive radiofrequency switches. Nat. Commun. 6, 7519 (2015).10.1038/ncomms8519 ↗
88. Li, Z. et al. Experimental demonstration of a defect-tolerant nanocrossbar demultiplexer. Nanotechnology 19, 165203 (2008).10.1088/0957-4484/19/16/165203 ↗
89. Bayat, F. M. Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits. Nat. Commun. 9, 2331 (2018).10.1038/s41467-018-04482-4 ↗
90. Du, C. et al. Reservoir computing using dynamic memristors for temporal information processing. Nat. Commun. 8, 2204 (2017).10.1038/s41467-017-02337-y ↗
91. Serb, A. et al. Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses. Nat. Commun. 7, 12611 (2016).10.1038/ncomms12611 ↗
92. Midya, R. et al. Anatomy of Ag/Hafnia-based selectors with 1010 nonlinearity. Adv. Mater. 29, 1604457 (2017).10.1002/adma.201604457 ↗
93. Srinivasan, V. S. S. et al. Punchthrough-diode-based bipolar RRAM selector by Si epitaxy. IEEE Electron Dev. Lett. 33, 1396–1398 (2012).10.1109/LED.2012.2209394 ↗
94. Huang, J.-J., Tseng, Y.-M., Hsu, C.-W. & Hou, T.-H. Bipolar nonlinear Ni/TiO2/Ni selector for 1S1R crossbar array applications. IEEE Electron Dev. Lett. 32, 1427–1429 (2011).10.1109/LED.2011.2161601 ↗
95. Shin, J. et al. TiO2-based metal–insulator–metal selection device for bipolar resistive random access memory cross-point application. J. Appl. Phys. 109, 033712 (2011).10.1063/1.3544205 ↗
96. Govoreanu, B. et al. High-performance metal–insulator–metal tunnel diode selectors. IEEE Electron Dev. Lett. 35, 63–65 (2014).10.1109/LED.2013.2291911 ↗
97. Woo, J. et al. Electrical and reliability characteristics of a scaled (∼30 nm) tunnel barrier selector (W/Ta2O5/TaOx/TiO2/TiN) with excellent performance (JMAX > 107 A/cm2). In 2014 Symp. VLSI Technology (VLSIT) (IEEE, 2014); https://doi.org/10.1109/VLSIT.2014.689443110.1109/VLSIT.2014.6894431 ↗
98. Lee, W. et al. Varistor-type bidirectional switch (JMAX > 107 A/cm2, selectivity ∼104) for 3D bipolar resistive memory arrays. In 2012 Symp. VLSI Technology (VLSIT) 37–38 (IEEE, 2012).
99. Choi, B. J. et al. Trilayer tunnel selectors for memristor memory cells. Adv. Mater. 28, 356 (2016).10.1002/adma.201503604 ↗
100. Kawahara, A. et al. An 8 Mb multi-layered cross-point ReRAM macro with 443 MB/s write throughput. IEEE J. Solid-State Circuits 48, 178 (2013).10.1109/JSSC.2012.2215121 ↗