Assessing Design Space for the Device-Circuit Codesign of Nonvolatile Memory-Based Compute-in-Memory Accelerators
Source abstract
Abstract
Unprecedented penetration of artificial intelligence (AI) algorithms has brought about rapid innovations in electronic hardware, including new memory devices. Nonvolatile memory (NVM) devices offer one such attractive alternative with ∼2× density and data retention after powering off. Compute-in-memory (CIM) architectures further improve energy efficiency by fusing the computation operations with AI model storage. Electronic characteristics of NVM devices, like resistance in the two resistance states, directly affect the circuit designers' decisions and result in the varying performance of NVM-CIM chips. In this mini review, we assess the bounds on device resistances for accuracy and circuit performance to suggest recommendations to device engineers for frictionless device-circuit-system interactions. Furthermore, we review challenges in reliably programming NVM devices, followed by benchmarking recent NVM-CIM chips. Our literature review and analytical modeling reveal that a high resistance ratio and low variability are favored, and the resistance in a low resistance state is bound by accuracy and circuit performance constraints.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Semiconductor materials and devices
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Corporate Research, TSMC, San Jose, California 95134, United States
Corporate Research, TSMC, Hsinchu, 300-094, Taiwan
National Tsing Hua University, Hsinchu, 300-044, Taiwan
Funding
Youth Innovation Promotion Association of the Chinese Academy of Sciences · National Natural Science Foundation of China · Chinese Academy of Sciences
References (50)
Show source-provided references
1. 10.48550/arXiv.2406.0841310.48550/arXiv.2406.08413 ↗
2. 10.48550/arXiv.2403.0902610.48550/arXiv.2403.09026 ↗
3. Chen, R.; Sisto, G.; Jourdain, A.; Hiblot, G.; Stucchi, M.; Kakarla, N.; Chehab, B.; Salahuddin, S. M.; Schleicher, F.; Veloso, A.; Hellings, G.; Weckx, P.; Milojevic, D.; Van der Plas, G.; Ryckaert, J.; Beyne, E. Design and optimization of SRAM macro and logic using backside interconnects at 2nm node. IEEE International Electron Devices Meeting (IEDM), IEEE, 2021.10.1109/IEDM19574.2021.9720528 ↗
4. 10.1038/s41928-023-00994-010.1038/s41928-023-00994-0 ↗
5. 10.1038/s41928-021-00676-910.1038/s41928-021-00676-9 ↗
6. 10.1038/s41928-020-00505-510.1038/s41928-020-00505-5 ↗
7. 10.1002/adma.20220494410.1002/adma.202204944 ↗
8. Huang, Y.C.; Liu, S.H.; Chen, H.S.; Feng, H.C.; Li, C.F.; Yang, C.Y.; Chang, W.K.; Yang, C.F.; Wu, C.Y.; Lin, Y.C.; Yang, T.T.; Chang, C.Y.; Chu, W.T.; Chuang, H.; Wang, Y.; Chih, Y.D.; Chang, T.Y. J. 15.7 A 32 Mb RRAM in a 12 nm FinFet Technology with a 0.0249 μm 2 Bit-Cell, a 3.2 GB/S Read Throughput, a 10K Cycle Write Endurance and a 10-Year Retention at 105 °C. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2024.10.1109/ISSCC49657.2024.10454367 ↗
9. Wen, T.H.; Hsu, H.H.; Khwa, W.S.; Huang, W.H.; Ke, Z.E.; Chin, Y.H.; Wen, H.J.; Chang, Y.C.; Hsu, W.T.; Lo, C.C.; Liu, R.S.; Hsieh, C.C.; Tang, K.T.; Teng, S.H.; Chou, C.C.; Chih, Y.D.; Chang, T.Y. J.; Chang, M.F. 34.8 A 22nm 16Mb Floating-Point ReRAM Compute-in-Memory Macro with 31.2 TFLOPS/W for AI Edge Devices. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2024.10.1109/ISSCC49657.2024.10454468 ↗
10. Spetalnick, S. D.; Lele, A. S.; Crafton, B.; Chang, M.; Ryu, S.; Yoon, J.H.; Hao, Z.; Ansari, A.; Khwa, W.S.; Chih, Y.D.; Chang, M.F.; Raychowdhury, A. 30.1 A 40nm VLIW Edge Accelerator with 5MB of 0.256 pJ/b RRAM and a Localization Solver for Bristle Robot Surveillance. 2024 IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2024.10.1109/ISSCC49657.2024.10454500 ↗
11. Chiu, Y.C.; Yang, C.S.; Teng, S.H.; Huang, H.Y.; Chang, F.C.; Wu, Y.; Chien, Y.A.; Hsieh, F.L.; Li, C.Y.; Lin, G.Y.; Chen, P.J.; Pan, T.H.; Lo, C.C.; Khwa, W.S.; Liu, R.S.; Hsieh, C.C.; Tang, K.T.; Lo, C.P.; Chih, Y.D.; Chang, T.Y. J.; Chang, M.F. A 22nm 4Mb STT-MRAM data-encrypted near-memory computation macro with a 192GB/s read-and-decryption bandwidth and 25.1–55.1 TOPS/W 8b MAC for AI operations. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2022.10.1109/ISSCC42614.2022.9731621 ↗
12. Cai, H.; Bian, Z.; Hou, Y.; Zhou, Y.; Cui, J.l.; Guo, Y.; Tian, X.; Liu, B.; Si, X.; Wang, Z.; Yang, J.; Shan, W. 33.4 A 28nm 2Mb STT-MRAM computing-in-memory macro with a refined bit-cell and 22.4–41.5 TOPS/W for AI inference. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2023.10.1109/ISSCC42615.2023.10067339 ↗
13. Edelstein, D.; Rizzolo, M.; Sil, D.; Dutta, A.; DeBrosse, J.; Wordeman, M.; Arceo, A.; Chu, I. C.; Demarest, J.; Edwards, E. R. J.; Evarts, E. R.; Fullam, J.; Gasasira, A.; Hu, G.; Iwatake, M.; Johnson, R.; Katragadda, V.; Levin, T.; Li, J.; Liu, Y.; Long, C.; Maffitt, T.; McDermott, S.; Mehta, S.; Mehta, V.; Metzler, D.; Morillo, J.; Nakamura, Y.; Nguyen, S.; Nieves, P.; Pai, V.; Patlolla, R.; Pujari, R.; Southwick, R.; Standaert, T.; van der Straten, O.; Wu, H.; Yang, C.C.; Houssameddine, D.; Slaughter, J. M.; Worledge, D. C. A 14 nm embedded stt-mram cmos technology. IEEE International Electron Devices Meeting (IEDM), IEEE, 2020.10.1109/IEDM13553.2020.9371922 ↗
14. Khwa, W.S.; Chiu, Y.C.; Jhang, C.J.; Huang, S.P.; Lee, C.Y.; Wen, T.H.; Chang, F.C.; Yu, S.M.; Lee, T.Y.; Chang, M.F. A 40-nm, 2M-cell, 8b-precision, hybrid SLC-MLC PCM computing-in-memory macro with 20.5–65.0 TOPS/W for tiny-Al edge devices. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2022.10.1109/ISSCC42614.2022.9731670 ↗
15. Khaddam-Aljameh, R.; Stanisavljevic, M.; Mas, J. F.; Karunaratne, G.; Braendli, M.; Liu, F.; Singh, A.; Müller, S. M.; Egger, U.; Petropoulos, A.; Antonakopoulos, T.; Brew, K.; Choi, S.; Ok, I.; Lie, F. L.; Saulnier, N.; Chan, V.; Ahsan, I.; Narayanan, V.; Nandakumar, S. R.; Gallo, M. L.; Francese, P. A.; Sebastian, A.; Eleftheriou, E. HERMES Core–A 14nm CMOS and PCM-based In-Memory Compute Core using an array of 300ps/LSB Linearized CCO-based ADCs and local digital processing. Symposium on VLSI Circuits, IEEE, 2021.10.23919/VLSICircuits52068.2021.9492362 ↗
16. 10.1038/s41928-023-01010-110.1038/s41928-023-01010-1 ↗
17. 10.48550/arXiv.2007.0007210.48550/arXiv.2007.00072 ↗
18. 10.1038/s41928-023-01053-410.1038/s41928-023-01053-4 ↗
19. 10.1109/MCAS.2021.309253310.1109/MCAS.2021.3092533 ↗
20. Xi, Y.; Gao, B.; Tang, J.; Chen, A.; Chang, M.F.; Hu, X. S.; Spiegel, J. V. D.; Qian, H.; Wu, H. In-memory learning with analog resistive switching memory: A review and perspective. Proceedings of the IEEE, IEEE, 2021.10.1109/JPROC.2020.3004543 ↗
21. 10.1038/s44287-023-00002-910.1038/s44287-023-00002-9 ↗
22. 10.1109/TED.2021.306974610.1109/TED.2021.3069746 ↗
23. 10.1109/TCSI.2022.316535210.1109/TCSI.2022.3165352 ↗
24. 10.1038/s41467-024-45670-910.1038/s41467-024-45670-9 ↗
25. 10.1038/s44287-024-00037-610.1038/s44287-024-00037-6 ↗
26. 10.1109/JXCDC.2024.338188810.1109/JXCDC.2024.3381888 ↗
27. 10.1038/s41586-023-05759-510.1038/s41586-023-05759-5 ↗
28. 10.1109/JEDS.2019.293176910.1109/JEDS.2019.2931769 ↗
29. 10.1109/LSSC.2023.333821210.1109/LSSC.2023.3338212 ↗
30. Chang, M.; Lele, A. S.; Spetalnick, S. D.; Crafton, B.; Konno, S.; Wan, Z.; Bhat, A.; Khwa, W.S.; Chih, Y.D.; Chang, M.F.; Raychowdhury, A. A 73.53 TOPS/W 14.74 TOPS heterogeneous RRAM in-memory and SRAM near-memory SoC for hybrid frame and event-based target tracking. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2023, na.10.1109/ISSCC42615.2023.10067544 ↗
31. 10.1007/978-3-031-21514-8_4110.1007/978-3-031-21514-8_41 ↗
32. 10.1109/TCAD.2021.312475710.1109/TCAD.2021.3124757 ↗
33. Shafaei, A.; Afzali-Kusha, H.; Pedram, M. Minimizing the energy-delay product of SRAM arrays using a device-circuit-architecture co-optimization framework. Proceedings of the 53rd Annual Design Automation Conference, 2016.10.1145/2897937.2898044 ↗
34. Crafton, B.; Wan, Z.; Spetalnick, S.; Yoon, J.H.; Wu, W.; Tokunaga, C.; De, V.; Raychowdhury, A. Improving compute in-memory ECC reliability with successive correction. Design Automation Conference (DAC), 2022.10.1145/3489517.3530526 ↗
35. 10.1109/LED.2023.327436210.1109/LED.2023.3274362 ↗
36. Dutta, S.; Ye, H.; Chakraborty, W.; Luo, Y.C.; Jose, M. S.; Grisafe, B. Monolithic 3D Integration of High Endurance Multi-Bit Ferroelectric FET for Accelerating Compute-In-Memory. IEEE International Electron Devices Meeting (IEDM), IEEE, San Francisco, 2020.10.1109/IEDM13553.2020.9371974 ↗
37. 10.1109/TED.2022.322752910.1109/TED.2022.3227529 ↗
38. Hsu, T.H.; Lue, H.T.; Hsu, P.K.; Yeh, T.H.; Du, P.Y.; Lee, G.R. A Vertical Split-Gate Flash Memory Featuring High-Speed Source-Side Injection Programming, Read Disturb Free, and 100K Endurance for Embedded Flash (eFlash) Scaling and Computing-In-Memory (CIM). IEEE International Electron Devices Meeting (IEDM), IEEE, San Francisco, 2020.10.1109/IEDM13553.2020.9372036 ↗
39. Chou, C.C.; Lin, Z.J.; Lai, C.A.; Su, C.I.; Tseng, P.L.; Chen, W.C.; Tsai, W.C.; Chu, W.T.; Ong, T.C.; Chuang, H.; Chih, Y.D.; Chang, T.Y. J. A 22 nm 96KX144 RRAM macro with a self-tracking reference and a low ripple charge pump to achieve a configurable read window and a wide operating voltage range. IEEE Symposium on VLSI Circuits, IEEE, 2020.10.1109/VLSICircuits18222.2020.9163014 ↗
40. 10.1109/OJSSCS.2024.343246810.1109/OJSSCS.2024.3432468 ↗
41. 10.1109/JSSC.2021.310120910.1109/JSSC.2021.3101209 ↗
42. Sheu, S.S.; Chiang, P.C.; Lin, W.P.; Lee, H.Y.; Chen, P.S.; Chen, Y.S.; Wu, T.Y.; Chen, F. T.; Su, K.L.; Kao, M.J.; Cheng, K.H.; Tsai, M.J. A 5 ns fast write multilevel nonvolatile 1 K bits RRAM memory with advance write scheme. Symposium on VLSI Circuits, 2009.
43. 10.1021/acs.nanolett.3c0026210.1021/acs.nanolett.3c00262 ↗
44. Huang, W.H.; Wen, T.H.; Hung, J.M.; Khwa, W.S.; Lo, Y.C.; Jhang, C.J.; Hsu, H.H.; Chin, Y.H.; Chen, Y.C.; Lo, C.C.; Liu, R.S.; Tang, K.T.; Hsieh, C.C.; Chih, Y.D.; Chang, T.Y.; Chang, M.F. A nonvolatile Al-edge processor with 4MB SLC-MLC hybrid-mode ReRAM compute-in-memory macro and 51.4–251TOPS/W. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2023.10.1109/ISSCC42615.2023.10067610 ↗
45. Spetalnick, S. D.; Chang, M.; Konno, S.; Crafton, B.; Lele, A. S.; Khwa, W.S.; Chih, Y.D.; Chang, M.F.; Raychowdhury, A. A 2.38 MCells/mm2 9.81–350 TOPS/W RRAM Compute-in-Memory Macro in 40nm CMOS with Hybrid Offset/I OFF Cancellation and I CELL R BLSL Drop Mitigation. IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits), IEEE, 2023.10.23919/VLSITechnologyandCir57934.2023.10185424 ↗
46. Wen, T.H.; Hung, J.M.; Hsu, H.H.; Wu, Y.; Chang, F.C.; Li, C.Y.; Chien, C.H.; Su, C.I.; Khwa, W.S.; Wu, J.J.; Lo, C.C.; Liu, R.S.; Hsieh, C.C.; Tang, K.T.; Ho, M.S.; Chih, Y.D.; Chang, T.Y. J.; Chang, M.F. A 28 nm Nonvolatile AI Edge Processor using 4 Mb Analog-Based Near-Memory-Compute ReRAM with 27.2 TOPS/W for Tiny AI Edge Devices. IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits), IEEE, 2023.10.23919/VLSITechnologyandCir57934.2023.10185326 ↗
47. Li, W.; Read, J.; Jiang, H.; Yu, S. A 40nm RRAM compute-in-memory macro with parallelism-preserving ECC for iso-accuracy voltage scaling. IEEE 48th European Solid State Circuits Conference (ESSCIRC), IEEE, 2022.10.1109/ESSCIRC55480.2022.9911464 ↗
48. Chiu, Y.C.; Khwa, W.S.; Li, C.Y.; Hsieh, F.L.; Chien, Y.A.; Lin, G.Y.; Chen, P.J.; Pan, T.H.; You, D.Q.; Chen, F.Y.; Lee, rew, Lo, C.C.; Liu, R.S.; Hsieh, C.C.; Tang, K.T.; Chih, Y.D.; Chang, T.Y.; Chang, M.F. A 22 nm 8 Mb STT-MRAM Near-Memory-Computing Macro with 8b-Precision and 46.4–160.1 TOPS/W for Edge-AI Devices. IEEE International Solid-State Circuits Conference (ISSCC), IEEE, 2023.10.1109/ISSCC42615.2023.10067563 ↗
49. Deaville, P.; Zhang, B.; Chen, L.Y.; Verma, N. A Maximally Row-Parallel MRAM In-Memory-Computing Macro Addressing Readout Circuit Sensitivity and Area. IEEE European Solid State Circuits Conference (ESSCIRC), IEEE, 2021.10.1109/ESSCIRC53450.2021.9567807 ↗
50. 10.1109/JSSC.2024.344201310.1109/JSSC.2024.3442013 ↗