Perspective: Uniform switching of artificial synapses for large-scale neuromorphic arrays
Source abstract
Abstract
Resistive random-access memories are promising analog synaptic devices for efficient bio-inspired neuromorphic computing arrays. Here we first describe working principles for phase-change random-access memory, oxide random-access memory, and conductive-bridging random-access memory for artificial synapses. These devices could allow for dense and efficient storage of analog synapse connections between CMOS neuron circuits. We also discuss challenges and opportunities for analog synaptic devices toward the goal of realizing passive neuromorphic computing arrays. Finally, we focus on reducing spatial and temporal variations, which is critical to experimentally realize powerful and efficient neuromorphic computing systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Photoreceptor and optogenetics research · CCD and CMOS Imaging Sensors
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mechanical Engineering, Massachusetts Institute of Technology 1 , Cambridge, Massachusetts 02139-4307, USA
Research Laboratory of Electronics, Massachusetts Institute of Technology 2 , Cambridge, Massachusetts 02139-4307, USA
Department of Materials Science and Engineering, Massachusetts Institute of Technology 3 , Cambridge, Massachusetts 02139-4307, USA
Funding
National Science Foundation
References (100+)
Show source-provided references
1. ImageNet classification with deep convolutional neural networks
2. Very deep convolutional networks for large-scale image recognition
3. Going deeper with convolutions10.1109/CVPR.2015.7298594 ↗
4. Speech recognition with deep recurrent neural networks10.1109/ICASSP.2013.6638947 ↗
5. Recurrent deep neural networks for robust speech recognition
6. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups10.1109/msp.2012.2205597 ↗
7. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
8. Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning
9. Prioritized experience replay
10. Incentivizing exploration in reinforcement learning with deep predictive models
11. Mastering the game of Go without human knowledge10.1038/nature24270 ↗
12. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals10.1126/science.aao1733 ↗
13. Deep learning10.1038/nature14539 ↗
14. DiracNets: Training very deep neural networks without skip-connections
15. Efficient processing of deep neural networks: A tutorial and survey10.1109/JPROC.2017.2761740 ↗
16. See http://www.nvidia.com/object/accelerate-inference.html for Deep Learning Inference Accelerators | NVIDIA Tesla | NVIDIA; accessed 17 January 2018.
17. VLSI architecture for high-performance, low-cost, on-chip learning
18. 1991 · 27110.1007/978-1-4615-3994-0_14 ↗
19. DianNao: A small-footprint high-throughput accelerator for ubiquitous machine-learning10.1145/2644865.2541967 ↗
20. ShiDianNao
21. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks10.1109/jssc.2016.2616357 ↗
22. In-datacenter performance analysis of a tensor processing unit
23. DaDianNao: A machine-learning supercomputer
24. FPGA acceleration of recurrent neural network based language model
25. CNP: An FPGA-based processor for convolutional networks
26. Optimizing FPGA-based accelerator design for deep convolutional neural networks
27. A dynamically configurable coprocessor for convolutional neural networks
28. In-datacenter performance analysis of a tensor processing unit
29. Harmonica: A framework of heterogeneous computing systems with memristor-based neuromorphic computing accelerators10.1109/tcsi.2016.2529279 ↗
30. RENO
31. Training deep convolutional neural networks with resistive cross-point devices10.3389/fnins.2017.00538 ↗
32. Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain10.1002/cne.21974 ↗
33. 2002
34. From the neuron doctrine to neural networks10.1038/nrn3962 ↗
35. Time course of TEA(+)-induced anomalous rectification in squid giant axons10.1085/jgp.50.2.491 ↗
36. Anomolous rectification in the squid giant axon injected with tetraethylammonium chloride10.1085/jgp.48.5.859 ↗
37. Prior activity influences the velocity of impulses in frog and cat optic nerve fibers10.1016/0006-8993(84)90867-9 ↗
38. A quantitative description of membrane current and its application to conduction and excitation in nerve10.1113/jphysiol.1952.sp004764 ↗
39. Demonstration of two stable potential states in the squid giant axon under tetraethylammonium chloride10.1085/jgp.40.6.859 ↗
40. Replacement of the protoplasm of a giant nerve fibre with artificial solutions10.1038/190885a0 ↗
41. Neuromorphic electronic systems10.1109/5.58356 ↗
42. Nanoscale memristor device as synapse in neuromorphic systems10.1021/nl904092h ↗
43. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing10.1038/nmat4756 ↗
44. NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in situ learning
45. Neuro-inspired computing with emerging nonvolatile memorys10.1109/jproc.2018.2790840 ↗
46. Die lernmatrix10.1007/bf00293853 ↗
47. A generic systolic array building block for neural networks with on-chip learning10.1109/72.217181 ↗
48. The perceptron: A probabilistic model for information storage and organization in the brain10.1037/h0042519 ↗
49. Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex10.1523/jneurosci.02-01-00032.1982 ↗
50. The ‘wake-sleep’ algorithm for unsupervised neural networks10.1126/science.7761831 ↗
51. Training products of experts by minimizing contrastive divergence10.1162/089976602760128018 ↗
52. Learning representations by back-propagating errors10.1038/323533a0 ↗
53. Taylor expansion of the accumulated rounding error10.1007/bf01931367 ↗
54. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
55. Backpropagation applied to handwritten zip code recognition10.1162/neco.1989.1.4.541 ↗
56. Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: Comparative performance analysis (accuracy, speed, and power)
57. Training and operation of an integrated neuromorphic network based on metal-oxide memristors10.1038/nature14441 ↗
58. Scaling-up resistive synaptic arrays for neuro-inspired architecture: Challenges and prospect
59. Training itself: Mixed-signal training acceleration for memristor-based neural network
60. Parallel programming of resistive cross-point array for synaptic plasticity10.1016/j.procs.2014.11.094 ↗
61. Deep learning in neural networks: An overview10.1016/j.neunet.2014.09.003 ↗
62. Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element
63. On-chip sparse learning acceleration with CMOS and resistive synaptic devices10.1109/tnano.2015.2478861 ↗
64. Memristor-based multilayer neural networks with online gradient descent training10.1109/tnnls.2014.2383395 ↗
65. Design of resistive synaptic array for implementing on-chip sparse learning10.1109/tmscs.2016.2598742 ↗
66. Fast, energy-efficient, robust, and reproducible mixed-signal neuromorphic classifier based on embedded NOR flash memory technology
67. Temperature-insensitive analog vector-by-matrix multiplier based on 55 nm NOR flash memory cells
68. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing10.1038/nmat4856 ↗
69. Magnetoresistive random access memory: The path to competitiveness and scalability10.1109/jproc.2008.2004313 ↗
70. Ferroelectric artificial synapses for recognition of a multishaded image10.1109/ted.2014.2331707 ↗
71. TiOx-based RRAM synapse with 64-levels of conductance and symmetric conductance change by adopting a hybrid pulse scheme for neuromorphic computing10.1109/led.2016.2622716 ↗
72. A fast, high-endurance and scalable non-volatile memory device made from asymmetric Ta2O(5-x)/TaO(2-x) bilayer structures10.1038/nmat3070 ↗
73. Sub-nanosecond switching of a tantalum oxide memristor10.1088/0957-4484/22/48/485203 ↗
74. Memristor crossbar arrays with 6-nm half-pitch and 2-nm critical dimension10.1038/s41565-018-0302-0 ↗
75. SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations10.1038/s41563-017-0001-5 ↗
76. Electrochemical metallization memories—Fundamentals, applications, prospects10.1088/0957-4484/22/25/254003 ↗
77. Resistive switching : From fundamentals of nanoionic redox processes to memristive device applications
78. Metal-oxide RRAM10.1109/JPROC.2012.2190369 ↗
79. An overview of materials issues in resistive random access memory10.1016/j.jmat.2015.07.009 ↗
80. TaOx-based resistive switching memories: Prospective and challenges10.1186/1556-276X-8-418 ↗
81. Nonvolatile memory materials for neuromorphic intelligent machines10.1002/adma.201704729 ↗
82. Emerging memories: Resistive switching mechanisms and current status10.1088/0034-4885/75/7/076502 ↗
83. Towards the development of flexible non-volatile memories10.1002/adma.201301361 ↗
84. Variability of resistive switching memories and its impact on crossbar array performance
85. A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit10.1038/srep42429 ↗
86. 3-D memristor crossbars for analog and neuromorphic computing applications10.1109/ted.2016.2630925 ↗
87. Design of ternary neural network with 3-D vertical RRAM array10.1109/ted.2017.2697361 ↗
88. Four-layer 3D vertical RRAM integrated with FinFET as a versatile computing unit for brain-inspired cognitive information processing
89. Experimental demonstration of a second-order memristor and its ability to biorealistically implement synaptic plasticity10.1021/acs.nanolett.5b00697 ↗
90. Acceleration of deep neural network training with resistive cross-point devices: Design considerations10.3389/fnins.2016.00333 ↗
91. Reversible electrical switching phenomena in disordered structures10.1103/physrevlett.21.1450 ↗
92. Physics of the switching kinetics in resistive memories10.1002/adfm.201500825 ↗
93. Materials science and engineering of phase change random access memory10.1080/02670836.2017.1341723 ↗
94. 1986
95. Study on the resistance drift in amorphous Ge2Sb2Te5 according to defect annihilation and stress relaxation10.1149/2.001204esl ↗
96. Neuromorphic computing using non-volatile memory10.1080/23746149.2016.1259585 ↗
97. Equivalent-accuracy accelerated neural-network training using analogue memory10.1038/s41586-018-0180-5 ↗
98. Atomic migration in molten and crystalline Ge2Sb2Te5 under high electric field10.1063/1.3184584 ↗
99. Microstructure-dependent DC set switching behaviors of Ge–Sb–Te-based phase-change random access memory devices accessed by in situ TEM10.1038/am.2015.49 ↗
100. Phase change memory technology10.1116/1.3301579 ↗