Memory devices and applications for in-memory computing
Source abstract
Abstract
Traditional von Neumann computing systems involve separate processing and memory units. However, data movement is costly in terms of time and energy and this problem is aggravated by the recent explosive growth in highly data-centric applications related to artificial intelligence. This calls for a radical departure from the traditional systems and one such non-von Neumann computational approach is in-memory computing. Hereby certain computational tasks are performed in place in the memory itself by exploiting the physical attributes of the memory devices. Both charge-based and resistance-based memory devices are being explored for in-memory computing. In this Review, we provide a broad overview of the key computational primitives enabled by these memory devices as well as their applications spanning scientific computing, signal processing, optimization, machine learning, deep learning and stochastic computing.
Source: europepmc · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Neuroscience and Neural Engineering
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Mutlu, O., Ghose, S., Gómez-Luna, J. & Ausavarungnirun, R. Processing data where it makes sense: Enabling in-memory computation. Microprocess. Microsyst. 67, 28–41 (2019).10.1016/j.micpro.2019.01.009 ↗
2. Horowitz, M. Computing’s energy problem (and what we can do about it). In Proc. International Solid-state Circuits Conference (ISSCC) 10–14 (IEEE, 2014).10.1109/ISSCC.2014.6757323 ↗
3. Keckler, S. W., Dally, W. J., Khailany, B., Garland, M. & Glasco, D. GPUs and the future of parallel computing. IEEE Micro 31, 7–17 (2011).10.1109/MM.2011.89 ↗
4. Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In Proc. International Symposium on Computer Architecture (ISCA) 1–12 (IEEE, 2017).
5. Sze, V., Chen, Y.-H., Yang, T.-J. & Emer, J. S. Efficient processing of deep neural networks: A tutorial and survey. Proc. IEEE 105, 2295–2329 (2017).10.1109/JPROC.2017.2761740 ↗
6. Patterson, D. et al. A case for intelligent RAM. IEEE Micro 17, 34–44 (1997).10.1109/40.592312 ↗
7. Farooq, M. et al. 3D copper TSV integration, testing and reliability. In Proc. International Electron Devices Meeting 7–1 (IEEE, 2011).10.1109/IEDM.2011.6131504 ↗
8. Pawlowski, J. T. Hybrid memory cube (HMC). In Proceedings of the Hot Chips Symposium (HCS) 1–24 (IEEE, 2011).10.1109/HOTCHIPS.2011.7477494 ↗
9. Kim, J. & Kim, Y. HBM: Memory solution for bandwidth-hungry processors. In Proc. Hot Chips Symposium (HCS) 1–24 (IEEE, 2014).
10. Shulaker, M. M. et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature 547, 74 (2017).10.1038/nature22994 ↗
11. Di Ventra, M. & Pershin, Y. V. The parallel approach. Nat. Phys. 9, 200 (2013).10.1038/nphys2566 ↗
12. Indiveri, G. & Liu, S.-C. Memory and information processing in neuromorphic systems. Proc. The IEEE 103, 1379–1397 (2015).10.1109/JPROC.2015.2444094 ↗
13. Zhirnov, V. V. & Marinella, M. J. in Emerging Nanoelectronic Devices (eds Chen, A.) Ch. 3 (Wiley Online Library, 2015).
14. Wong, H.-S. P. & Salahuddin, S. Memory leads the way to better computing. Nat. Nanotechnol. 10, 191 (2015).10.1038/nnano.2015.29 ↗
15. Chua, L. Resistance switching memories are memristors. Appl. Phys. A Mater. Sci. Process. 102, 765–783 (2011).10.1007/s00339-011-6264-9 ↗
16. Li, S. et al. DRISA: A DRAM-based reconfigurable in-situ accelerator. In Proc. International Symposium on Microarchitecture (MICRO) 288–301 (IEEE, 2017).10.1145/3123939.3123977 ↗
17. Seshadri, V. et al. Ambit: In-memory accelerator for bulk bitwise operations using commodity DRAM technology. In Proc. International Symposium on Microarchitecture 273–287 (IEEE, 2017).10.1145/3123939.3124544 ↗
18. Jeloka, S., Akesh, N. B., Sylvester, D. & Blaauw, D. A 28 nm configurable memory (TCAM/BCAM/SRAM) using push-rule 6T bit cell enabling logic-in-memory. IEEE J. Solid-State Circuits 51, 1009–1021 (2016).10.1109/JSSC.2016.2515510 ↗
19. Aga, S. et al. Compute caches. In Proc. International Symposium on High Performance Computer Architecture (HPCA) 481–492 (IEEE, 2017).10.1109/HPCA.2017.21 ↗
20. Wang, J. et al. A compute SRAM with bit-serial integer/floating-point operations for programmable in-memory vector acceleration. In Proc. International Solid- State Circuits Conference (ISSCC) 224–226 (IEEE, 2019).10.1109/ISSCC.2019.8662419 ↗
21. Jiang, Z., Yin, S., Seok, M. & Seo, J. XNOR-SRAM: In-memory computing SRAM macro for binary/ternary deep neural networks. In Proc. Symposium on VLSI Technology 173–174 (IEEE, 2018).10.1109/VLSIT.2018.8510687 ↗
22. Biswas, A. & Chandrakasan, A. P. CONV-SRAM: an energy-efficient SRAM with in-memory dot-product computation for low-power convolutional neural networks. IEEE J. Solid-State Circuits 54, 217–230 (2019).10.1109/JSSC.2018.2880918 ↗
23. Valavi, H., Ramadge, P. J., Nestler, E. & Verma, N. A 64-tile 2.4-Mb in-memory-computing CNN accelerator employing charge-domain compute. IEEE J. Solid-State Circuits 54, 1789–1799 (2019).10.1109/JSSC.2019.2899730 ↗
24. Verma, N. et al. In-memory computing: Advances and prospects. IEEE J. Solid-State Circuits 11, 43–55 (2019).10.1109/MSSC.2019.2922889 ↗
25. Gonugondla, S. K., Kang, M. & Shanbhag, N. R. A variation-tolerant in-memory machine learning classifier via on-chip training. IEEE J. Solid-State Circuits 53, 3163–3173 (2018).10.1109/JSSC.2018.2867275 ↗
26. Bankman, D., Yang, L., Moons, B., Verhelst, M. & Murmann, B. An always-on 3.8 μ J/86% CIFAR-10 mixed-signal binary CNN processor with all memory on chip in 28-nm CMOS. IEEE J. Solid-State Circuits 54, 158–172 (2019).10.1109/JSSC.2018.2869150 ↗
27. Diorio, C., Hasler, P., Minch, A. & Mead, C. A. A single-transistor silicon synapse. IEEE Transactions on Electron Devices 43, 1972–1980 (1996).10.1109/16.543035 ↗
28. Merrikh-Bayat, F. et al. High-performance mixed-signal neurocomputing with nanoscale floating-gate memory cell arrays. IEEE Trans. Neural Netw. Learn. Syst. 29, 4782–4790 (2018).10.1109/TNNLS.2017.2778940 ↗
29. Wang, P. et al. Three-dimensional NAND flash for vector-matrix multiplication. EEE Trans. Very Large Scale Integr. VLSI Syst. 27, 988–991 (2019).10.1109/TVLSI.2018.2882194 ↗
30. Burr, G. W. et al. Access devices for 3D crosspoint memory. J. Vac. Sci. Technol. B Nanotechnol. Microelectron. 32, 040802 (2014).10.1116/1.4889999 ↗
31. Hickmott, T. Low-frequency negative resistance in thin anodic oxide films. J. Appl. Phys. 33, 2669–2682 (1962).10.1063/1.1702530 ↗
32. Beck, A., Bednorz, J., Gerber, C., Rossel, C. & Widmer, D. Reproducible switching effect in thin oxide films for memory applications. Applied Physics Letters 77, 139–141 (2000).10.1063/1.126902 ↗
33. Waser, R. & Aono, M. Nanoionics-based resistive switching memories. Nat. Mater. 6, 833–840 (2007).10.1038/nmat2023 ↗
34. Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80 (2008).10.1038/nature06932 ↗
35. Ovshinsky, S. R. Reversible electrical switching phenomena in disordered structures. Phys. Rev. Lett. 21, 1450 (1968).10.1103/PhysRevLett.21.1450 ↗
36. Wong, H.-S. P. et al. Phase change memory. Proc. IEEE 98, 2201–2227 (2010).10.1109/JPROC.2010.2070050 ↗
37. Burr, G. W. et al. Recent progress in phase-change memory technology. IEEE J. Emerg. Sel. Top. Circuits Syst. 6, 146–162 (2016).10.1109/JETCAS.2016.2547718 ↗
38. Khvalkovskiy, A. et al. Basic principles of STT-MRAM cell operation in memory arrays. J. Phys. D Appl. Phys. 46, 074001 (2013).10.1088/0022-3727/46/7/074001 ↗
39. Kent, A. D. & Worledge, D. C. A new spin on magnetic memories. Nat. Nanotechnol. 10, 187 (2015).10.1038/nnano.2015.24 ↗
40. Vourkas, I. & Sirakoulis, G. C. Emerging memristor-based logic circuit design approaches: A review. IEEE Circuits and Systems Magazine 16, 15–30 (2016).10.1109/MCAS.2016.2583673 ↗
41. Borghetti, J. et al. Memristive switches enable stateful logic operations via material implication. Nature 464, 873 (2010).10.1038/nature08940 ↗
42. Linn, E., Rosezin, R., Tappertzhofen, S., Böttger, U. & Waser, R. Beyond von neumann-logic operations in passive crossbar arrays alongside memory operations. Nanotechnology 23, 305205 (2012).10.1088/0957-4484/23/30/305205 ↗
43. Jeong, D. S., Kim, K. M., Kim, S., Choi, B. J. & Hwang, C. S. Memristors for energy-efficient new computing paradigms. Adv. Electron. Mater. 2, 1600090 (2016).10.1002/aelm.201600090 ↗
44. Kvatinsky, S. et al. MAGIC-memristor-aided logic IEEE Trans. Circuits Syst. II Express Briefs 61, 895–899 (2014).10.1109/TCSII.2014.2357292 ↗
45. Mahmoudi, H., Windbacher, T., Sverdlov, V. & Selberherr, S. Implication logic gates using spin-transfer-torque-operated magnetic tunnel junctions for intrinsic logic-in-memory. Solid State Electron. 84, 191–197 (2013).10.1016/j.sse.2013.02.017 ↗
46. Kim, K. M. et al. Single-cell stateful logic using a dual-bit memristor. Phys. Status Solidi Rapid Res. Lett. 13, 1800629 (2019).10.1002/pssr.201800629 ↗
47. Xu, N., Fang, L., Kim, K. M. & Hwang, C. S. Time-efficient stateful dual-bit-memristor logic. Phys. Status Solidi Rapid Res. Lett. 13, 1900033 (2019).10.1002/pssr.201900033 ↗
48. Li, S. et al. Pinatubo: A processing-in-memory architecture for bulk bitwise operations in emerging non-volatile memories. In Proc. The Design Automation Conference (DAC) 173 (ACM, 2016).10.1145/2897937.2898064 ↗
49. Xie, L. et al. Scouting logic: A novel memristor-based logic design for resistive computing. In Proc. The IEEE Symposium on VLSI (ISVLSI) 176–181 (IEEE, 2017).10.1109/ISVLSI.2017.39 ↗
50. Maan, A. K., Jayadevi, D. A. & James, A. P. A survey of memristive threshold logic circuits. IEEE Trans. Neural Netw. Learn. Syst. 28, 1734–1746 (2016).10.1109/TNNLS.2016.2547842 ↗
51. Burr, G. W. et al. Neuromorphic computing using non-volatile memory. Adv Phys X 2, 89–124 (2017).
52. Ielmini, D. & Wong, H.-S. P. In-memory computing with resistive switching devices. Nat. Electron. 1, 333 (2018).10.1038/s41928-018-0092-2 ↗
53. Wang, Z. et al. Resistive switching materials for information processing. Nat. Rev. Mater. https://doi.org/10.1038/s41578-019-0159-3 (2020).10.1038/s41578-019-0159-3 ↗
54. Wright, C. D., Hosseini, P. & Diosdado, J. A. V. Beyond von-neumann computing with nanoscale phase-change memory devices. Adv. Funct. Mater. 23, 2248–2254 (2013).10.1002/adfm.201202383 ↗
55. Sebastian, A. et al. Brain-inspired computing using phase-change memory devices. J. Appl. Phys. 124, 111101 (2018).10.1063/1.5042413 ↗
56. Godse, A. P. & Godse, D. A. Computer Organization and Architecture (Technical Publications, 2008).
57. Bojnordi, M. N. & Ipek, E. Memristive boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning. In Proc. The International Symposium on High Performance Computer Architecture (HPCA) 1–13 (IEEE, 2016).10.1109/HPCA.2016.7446049 ↗
58. Shafiee, A. et al. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. Comput. Archit. News 44, 14–26 (2016).10.1145/3007787.3001139 ↗
59. Chi, P. et al. PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory. In Proc. 43rd Annual International Symposium on Computer Architecture (ISCA) News 27–39 (IEEE, 2016).10.1145/3007787.3001140 ↗
60. Song, L., Qian, X., Li, H. & Chen, Y. PIPELAYER: A pipelined ReRAM-based accelerator for deep learning. In Proc. The International Symposium on High Performance Computer Architecture (HPCA), 541–552 (IEEE, 2017).10.1109/HPCA.2017.55 ↗
61. Zidan, M. A. et al. A general memristor-based partial differential equation solver. Nat. Electron. 1, 411 (2018).10.1038/s41928-018-0100-6 ↗
62. Higham, N. J. Accuracy and Stability of Numerical Algorithms, Vol. 80 (Society for Industrial and Applied Mathematics, 2002).10.1137/1.9780898718027 ↗
63. Bekas, C., Curioni, A. & Fedulova, I. Low cost high performance uncertainty quantification. In Proc. 2nd Workshop on High Performance Computational Finance 1–8 (ACM, 2009).10.1145/1645413.1645421 ↗
64. Le Gallo, M. et al. Mixed-precision in-memory computing. Nat. Electron. 1, 246–253 (2018).10.1038/s41928-018-0054-8 ↗
65. Liu, S., Wang, Y., Fardad, M. & Varshney, P. K. A memristor-based optimization framework for artificial intelligence applications. IEEE Circuits and Systems Magazine 18, 29–44 (2018).10.1109/MCAS.2017.2785421 ↗
66. Sun, Z. et al. Solving matrix equations in one step with cross-point resistive arrays. Proc. Natl. Acad. Sci. USA 116, 4123–4128 (2019).10.1073/pnas.1815682116 ↗
67. Sturges, R. H. Analog matrix inversion (robot kinematics). IEEE Journal on Robotics and Automation 4, 157–162 (1988).10.1109/56.2079 ↗
68. Feinberg, B., Wang, S. & Ipek, E. Making memristive neural network accelerators reliable. In Proc. The International Symposium on High Performance Computer Architecture (HPCA) 52–65 (IEEE, 2018).10.1109/HPCA.2018.00015 ↗
69. Feinberg, B., Vengalam, U. K. R., Whitehair, N., Wang, S. & Ipek, E. Enabling scientific computing on memristive accelerators. In Proc. International Symposium on Computer Architecture (ISCA) 367–382 (IEEE, 2018).10.1109/ISCA.2018.00039 ↗
70. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018).10.1038/s41928-017-0002-z ↗
71. Le Gallo, M., Sebastian, A., Cherubini, G., Giefers, H. & Eleftheriou, E. Compressed sensing with approximate message passing using in-memory computing. IEEE Trans. Electron Devices 65, 4304–4312 (2018).10.1109/TED.2018.2865352 ↗
72. Cai, F. et al. Harnessing intrinsic noise in memristor hopfield neural networks for combinatorial optimization. Preprint at https://arxiv.org/abs/1903.11194 (2019).
73. Mostafa, H., Müller, L. K. & Indiveri, G. An event-based architecture for solving constraint satisfaction problems. Nat. Commun. 6, 8941 (2015).10.1038/ncomms9941 ↗
74. Parihar, A., Shukla, N., Jerry, M., Datta, S. & Raychowdhury, A. Vertex coloring of graphs via phase dynamics of coupled oscillatory networks. Sci. Rep. 7, 911 (2017).10.1038/s41598-017-00825-1 ↗
75. Kumar, S., Strachan, J. P. & Williams, R. S. Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing. Nature 548, 318 (2017).10.1038/nature23307 ↗
76. Torrejon, J. et al. Neuromorphic computing with nanoscale spintronic oscillators. Nature 547, 428–431 (2017).10.1038/nature23011 ↗
77. Seo, J. et al. On-chip sparse learning acceleration with CMOS and resistive synaptic devices. IEEE Trans. Nanotechnol. 14, 969–979 (2015).10.1109/TNANO.2015.2478861 ↗
78. Sheridan, P. M. et al. Sparse coding with memristor networks. Nat. Nanotechnol. 12, 784–789 (2017).10.1038/nnano.2017.83 ↗
79. Sheridan, P. M., Du, C. & Lu, W. D. Feature extraction using memristor networks. IEEE Trans. Neural Netw. Learn. Syst. 27, 2327–2336 (2016).10.1109/TNNLS.2015.2482220 ↗
80. Choi, S., Sheridan, P. & Lu, W. D. Data clustering using memristor networks. Sci. Rep. 5, 10492 (2015).
81. Karam, R., Puri, R., Ghosh, S. & Bhunia, S. Emerging trends in design and applications of memory-based computing and content-addressable memories. Proc. IEEE 103, 1311–1330 (2015).10.1109/JPROC.2015.2434888 ↗
82. Rahimi, A. et al. High-dimensional computing as a nanoscalable paradigm. IEEE Trans. Circuits Syst. I Regul. Pap. 64, 2508–2521 (2017).10.1109/TCSI.2017.2705051 ↗
83. Wu, T. F. et al. Hyperdimensional computing exploiting carbon nanotube FETs, resistive RAM, and their monolithic 3D integration. IEEE J. Solid-State Circuits 53, 3183–3196 (2018).10.1109/JSSC.2018.2870560 ↗
84. Graves, A. et al. Hybrid computing using a neural network with dynamic external memory. Nature 538, 471 (2016).10.1038/nature20101 ↗
85. Ni, K. et al. Ferroelectric ternary content-addressable memory for one-shot learning. Nat. Electron. 2, 521–529 (2019).10.1038/s41928-019-0321-3 ↗
86. Eryilmaz, S. B. et al. Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array. Front. Neurosci. 8, 205 (2014).10.3389/fnins.2014.00205 ↗
87. Hu, S. et al. Associative memory realized by a reconfigurable memristive Hopfield neural network. Nat. Commun. 6, 7522 (2015).
88. Kavehei, O. et al. An associative capacitive network based on nanoscale complementary resistive switches for memory-intensive computing. Nanoscale 5, 5119–5128 (2013).10.1039/c3nr00535f ↗
89. Du, C. et al. Reservoir computing using dynamic memristors for temporal information processing. Nat. Commun. 8, 2204 (2017).
90. Sebastian, A. et al. Temporal correlation detection using computational phase-change memory. Nat. Commun. 8, 1115 (2017).10.1038/s41467-017-01481-9 ↗
91. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436 (2015).10.1038/nature14539 ↗
92. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016).10.1109/CVPR.2016.90 ↗
93. LeCun, Y. Deep learning hardware: Past, present, and future. In Proc. International Solid-State Circuits Conference (ISSCC) 12–19 (IEEE, 2019).10.1109/ISSCC.2019.8662396 ↗
94. Chen, Y., Yang, T., Emer, J. & Sze, V. Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices. IEEE J. Em. Sel. Top. C 9, 292–308 (2019).
95. Dazzi, M. et al. 5 parallel prism: A topology for pipelined implementations of convolutional neural networks using computational memory. In Proc. NeurIPS MLSys Workshop (NeurIPS, 2019); http://learningsys.org/neurips19/acceptedpapers.html
96. Jia, Z., Maggioni, M., Smith, J. & Scarpazza, D. P. Dissecting the NVidia Turing T4 GPU via microbenchmarking. Preprint at https://arxiv.org/abs/1903.07486 (2019).
97. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R. & Bengio, Y. Quantized neural networks: Training neural networks with low precision weights and activations. J. Mach. Learn. Res. 18, 6869–6898 (2017).
98. Xue, C. et al. 24.1 a 1mb multibit ReRAM computing-in-memory macro with 14.6ns parallel MAC computing time for CNN based AI edge processors. In Proc. The International Solid-State Circuits Conference (ISSCC) 388–390 (IEEE, 2019).10.1109/ISSCC.2019.8662395 ↗
99. Hu, M. et al. Memristor-based analog computation and neural network classification with a dot product engine. Advanced Materials 30, 1705914 (2018).10.1002/adma.201705914 ↗
100. Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577, 641–646 (2020).10.1038/s41586-020-1942-4 ↗