journal article·2024-01-24Open access

Analog Resistive Switching Devices for Training Deep Neural Networks with the Novel Tiki-Taka Algorithm

Tommaso Stecconi, Valeria Bragaglia, Malte J. Rasch, Fabio Carta, Folkert Horst, Donato F. Falcone, Sofieke C. ten Kate, Nanbo Gong, Takashi Ando, Antonis Olziersky, Bert Offrein

Nano Letters· Volume 24· Issue 3

Abstract

High Resolution Image Download MS PowerPoint Slide A critical bottleneck for the training of large neural networks (NNs) is communication with off-chip memory. A promising mitigation effort consists of integrating crossbar arrays of analogue memories in the Back-End-Of-Line, to store the NN parameters and efficiently perform the required synaptic operations. The “ Tiki-Taka ” algorithm was developed to facilitate NN training in the presence of device nonidealities. However, so far, a resistive switching device exhibiting all the fundamental Tiki-Taka requirements, which are many programmable states, a centered symmetry point, and low programming noise, was not yet demonstrated. Here, a complementary metal-oxide semiconductor (CMOS)-compatible resistive random access memory (RRAM), showing more than 30 programmable states with low noise and a symmetry point with only 5% skew from the center, is presented for the first time. These results enable generalization of Tiki-Taka training from small fully connected networks to larger long-/short-term-memory types of NN.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Advanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Transition Metal Oxide Nanomaterials

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

IBM Research Europe - Zürich, Rüschlikon, Zürich CH 8803, Switzerland

IBM Research - Yorktown Heights, Yorktown Heights, New York 10598, United States

Funding

Schweizerischer Nationalfonds zur F?rderung der Wissenschaftlichen Forschung · H2020 Marie Sklodowska-Curie Actions · H2020 Industrial Leadership · HORIZON EUROPE Digital, Industry and Space

References (34)

Show source-provided references
1. Amodei, D.; Hernandez, D. AI and compute. OpenAI Blog (16 May 2018); https://openai.com/blog/ai-and-compute/.
2. 10.1109/JPROC.1998.65876210.1109/JPROC.1998.658762 ↗
3. 10.1038/s41565-020-0655-z10.1038/s41565-020-0655-z ↗
4. Horowitz M. · 2014
5. 10.1038/s41928-017-0006-810.1038/s41928-017-0006-8 ↗
6. 10.1038/s41928-018-0092-210.1038/s41928-018-0092-2 ↗
7. 10.1109/TVLSI.2022.322139010.1109/TVLSI.2022.3221390 ↗
8. 10.1109/TCSII.2021.307861410.1109/TCSII.2021.3078614 ↗
9. 10.3389/fnins.2016.0033310.3389/fnins.2016.00333 ↗
10. 10.1088/0268-1242/31/6/06300210.1088/0268-1242/31/6/063002 ↗
11. 10.1109/JETCAS.2016.254771810.1109/JETCAS.2016.2547718 ↗
12. 10.3389/fnins.2020.0010310.3389/fnins.2020.00103 ↗
13. 10.1016/S0893-6080(98)00116-610.1016/S0893-6080(98)00116-6 ↗
14. 10.1038/ncomms1519910.1038/ncomms15199 ↗
15. 10.1038/s41586-020-1942-410.1038/s41586-020-1942-4 ↗
16. Gong, N. Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization. Technical Digest - International Electron Devices Meeting, IEDM 2022, 2022; pp 3371–3374.10.1109/IEDM45625.2022.10019569 ↗
17. 10.1063/1.512056510.1063/1.5120565 ↗
18. Kim S.
19. 10.1038/s41467-023-41958-410.1038/s41467-023-41958-4 ↗
20. 10.1002/advs.20230301810.1002/advs.202303018 ↗
21. 10.1038/s41598-023-42214-x10.1038/s41598-023-42214-x ↗
22. 10.1002/aelm.20220044810.1002/aelm.202200448 ↗
23. 10.3389/frai.2021.69914810.3389/frai.2021.699148 ↗
24. 10.3389/fnano.2022.100826610.3389/fnano.2022.1008266 ↗
25. 10.1109/LED.2014.233431110.1109/LED.2014.2334311 ↗
26. 10.1109/TED.2015.241811410.1109/TED.2015.2418114 ↗
27. 10.1039/c2tc00481j10.1039/c2tc00481j ↗
28. 10.1088/0268-1242/12/9/00110.1088/0268-1242/12/9/001 ↗
29. 10.1039/C9MH02033K10.1039/C9MH02033K ↗
30. Rasch M. J.
31. 10.1093/comjnl/7.2.15510.1093/comjnl/7.2.155 ↗
32. Newville, M.; Stensitzki, T.; Allen, D. B.; Rawlik, M.; Ingargiola, A.; Nelson, A. LMFIT: Non-linear least-square minimization and curve-fitting for Python. Astrophysics Source Code Library, 2016, ascl–1606. https://ui.adsabs.harvard.edu/abs/2016ascl.soft06014N/abstract.
33. 10.1109/MSP.2012.221147710.1109/MSP.2012.2211477 ↗
34. Bengio, Y.; Ducharme, R.; Vincent, P. A Neural Probabilistic Language Model. Advances in neural information processing systems, 2000, 13. https://proceedings.neurips.cc/paper_files/paper/2000/file/728f206c2a01bf572b5940d7d9a8fa4c-Paper.pdf.