Hydrogen‐Stabilized Self‐Rectifying Memristor Arrays for Reliable Multilevel Synapses in Transformer‐Based Keyword Spotting
Source abstract
Abstract
ABSTRACT This study proposes a strategy to simultaneously improve conductance uniformity and data retention characteristics by introducing the incremental step pulse with verify algorithm (ISPVA) technique and hydrogen (H 2 ) annealing into a non‐filamentary TiN/Ti/HfO 2 /TiO x /TiN resistive switching memory device. The high Schottky barrier formed at the Ti/HfO 2 interface induces asymmetric electron injection and limits reverse current flow, resulting in a rectifying ratio of approximately 1442. This self‐rectifying characteristic provides an intrinsic advantage in suppressing sneak currents in crossbar arrays. The ISPVA technique improves the linearity and uniformity of conductance modulation, enabling the implementation of up to 6‐bit multilevel states within a few‐µA current range. In addition, H 2 annealing stabilized conduction by forming hydrogen bonds with oxygen vacancies in the oxide layer and suppressing oxygen ion–vacancy recombination. As a result, data retention over 10 4 s and endurance exceeding 10 4 cycles were achieved even under a low energy consumption of 36.3 pJ. Furthermore, the experimentally obtained long‐term potentiation and depression characteristics were implemented in a Transformer‐based keyword spotting (KWS) model, achieving a recognition accuracy of 92.5%. These results suggest that the proposed device enables controlled analog conductance modulation with improved stability, showing its potential for Transformer‐based neuromorphic computing applications.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Magnetic properties of thin films
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Intelligent Semiconductor Engineering University of Seoul Seoul Republic of Korea
Division of Electronics and Electrical Engineering Dongguk University Seoul Republic of Korea
Department of Electrical and Computer Engineering University of Seoul Seoul Republic of Korea
Department of Intelligent Semiconductor Engineering School of Advanced Fusion Studies University of Seoul Seoul Republic of Korea
Funding
National Research Foundation of Korea
References (84)
Show source-provided references
1. Attention Is All You Need
2. 10.1016/j.nlp.2023.10004810.1016/j.nlp.2023.100048 ↗
3. 10.1002/brx2.5710.1002/brx2.57 ↗
4. 10.1007/s44196‐024‐00448‐110.1007/s44196‐024‐00448‐1 ↗
5. 10.1049/cit2.1221210.1049/cit2.12212 ↗
6. K.Ding M.Zong J.Li andB.Li “Letr: A Lightweight and Efficient Transformer for Keyword Spotting ” inICASSP 2022‐2022 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP)(IEEE 2022) 7987–7991 https://doi.org/10.1109/ICASSP43922.2022.9747295.10.1109/ICASSP43922.2022.9747295 ↗
7. 10.1038/nnano.2015.2910.1038/nnano.2015.29 ↗
8. 10.1002/adfm.20230812910.1002/adfm.202308129 ↗
9. 10.1109/MC.2017.910.1109/MC.2017.9 ↗
10. 10.1038/s41928‐018‐0103‐310.1038/s41928‐018‐0103‐3 ↗
11. 10.1038/s42254‐020‐0208‐210.1038/s42254‐020‐0208‐2 ↗
12. 10.1002/adma.20230106310.1002/adma.202301063 ↗
13. 10.1002/est2.7027210.1002/est2.70272 ↗
14. 10.1002/adma.20220686410.1002/adma.202206864 ↗
15. 10.1109/TETC.2022.321483310.1109/TETC.2022.3214833 ↗
16. 10.1007/s10853‐018‐2134‐610.1007/s10853‐018‐2134‐6 ↗
17. 10.1002/aisy.20210001710.1002/aisy.202100017 ↗
18. X.Yang B.Yan H.Li andY.Chen “ReTransformer: ReRAM‐based Processing‐in‐Memory Architecture for Transformer Acceleration ” inProceedings of the 39th International Conference on Computer‐Aided Design(IEEE 2020) https://doi.org/10.1145/3400302.3415640.10.1145/3400302.3415640 ↗
19. 10.1109/TVLSI.2023.334565110.1109/TVLSI.2023.3345651 ↗
20. 10.1021/acs.chemrev.4c0084510.1021/acs.chemrev.4c00845 ↗
21. 10.1038/s41598‐024‐75021‐z10.1038/s41598‐024‐75021‐z ↗
22. 10.1109/TVLSI.2023.334565110.1109/TVLSI.2023.3345651 ↗
23. 10.1109/TCAD.2024.343576210.1109/TCAD.2024.3435762 ↗
24. J.Cai M. A.Kaleem R.Genov M. R.Azghadi andA.Amirsoleimani “In‐Memory Transformer Self‐Attention Mechanism Using Passive Memristor Crossbar ” in2024 IEEE International Symposium on Circuits and Systems (ISCAS)(IEEE 2024) 1–5 https://doi.org/10.1109/ISCAS58744.2024.10558182.10.1109/ISCAS58744.2024.10558182 ↗
25. 10.1109/TED.2014.231020010.1109/TED.2014.2310200 ↗
26. 10.1038/s41467‐021‐23180‐210.1038/s41467‐021‐23180‐2 ↗
27. 10.1002/adma.20230721810.1002/adma.202307218 ↗
28. 10.1016/j.sse.2015.10.00310.1016/j.sse.2015.10.003 ↗
29. 10.1109/TED.2021.308999510.1109/TED.2021.3089995 ↗
30. 10.3390/electronics1009108410.3390/electronics10091084 ↗
31. 10.1002/adfm.20250095610.1002/adfm.202500956 ↗
32. 10.1002/admt.20250065110.1002/admt.202500651 ↗
33. Bagdzevicius S. · 2021 · 235
34. 10.1021/acsami.4c0314810.1021/acsami.4c03148 ↗
35. 10.1002/adfm.20242388010.1002/adfm.202423880 ↗
36. 10.1002/pssr.20130719210.1002/pssr.201307192 ↗
37. 10.3390/nano1310166510.3390/nano13101665 ↗
38. 10.1186/s40580‐026‐00535‐310.1186/s40580‐026‐00535‐3 ↗
39. 10.1140/epjst/e2019‐900046‐x10.1140/epjst/e2019‐900046‐x ↗
40. 10.3390/electronics1122366710.3390/electronics11223667 ↗
41. 10.1007/s00339‐016‐0569‐710.1007/s00339‐016‐0569‐7 ↗
42. 10.1016/j.matdes.2019.10846510.1016/j.matdes.2019.108465 ↗
43. 10.1016/j.tsf.2011.10.07510.1016/j.tsf.2011.10.075 ↗
44. 10.1063/1.497567610.1063/1.4975676 ↗
45. 10.1016/j.matchemphys.2011.01.03610.1016/j.matchemphys.2011.01.036 ↗
46. 10.1016/j.jmrt.2021.03.08210.1016/j.jmrt.2021.03.082 ↗
47. 10.3390/mi1311181210.3390/mi13111812 ↗
48. 10.1016/j.mset.2019.01.00110.1016/j.mset.2019.01.001 ↗
49. 10.1002/smll.20210757510.1002/smll.202107575 ↗
50. 10.1007/s42247‐022‐00356‐010.1007/s42247‐022‐00356‐0 ↗
51. 10.1039/D0NA00100G10.1039/D0NA00100G ↗
52. 10.3390/electronics403058610.3390/electronics4030586 ↗
53. 10.1016/j.jmst.2021.04.02510.1016/j.jmst.2021.04.025 ↗
54. 10.1186/s11671‐019‐3216‐310.1186/s11671‐019‐3216‐3 ↗
55. 10.1109/ACCESS.2022.314236810.1109/ACCESS.2022.3142368 ↗
56. 10.1063/1.483207610.1063/1.4832076 ↗
57. 10.1002/sia.382610.1002/sia.3826 ↗
58. 10.1021/acsaelm.0c0051510.1021/acsaelm.0c00515 ↗
59. 10.1088/1361‐6463/acc87810.1088/1361‐6463/acc878 ↗
60. 10.1002/advs.20230953810.1002/advs.202309538 ↗
61. 10.1016/j.memori.2023.10004710.1016/j.memori.2023.100047 ↗
62. 10.1038/s41598‐017‐01354‐710.1038/s41598‐017‐01354‐7 ↗
63. 10.1016/j.nanoen.2025.11113710.1016/j.nanoen.2025.111137 ↗
64. A.Berg M.O'Connor andM. T.Cruz “Keyword Transformer: A Self‐Attention Model for Keyword Spotting ” inProceedings of Interspeech 2021(ISCA 2021):4249–4253 https://doi.org/10.21437/Interspeech.2021‐1286.10.21437/Interspeech.2021-1286 ↗
65. 10.3390/electronics1009106310.3390/electronics10091063 ↗
66. 10.1088/1674‐4926/42/1/01310410.1088/1674‐4926/42/1/013104 ↗
67. 10.1038/s41598‐022‐09556‐410.1038/s41598‐022‐09556‐4 ↗
68. 10.1145/3007787.300113910.1145/3007787.3001139 ↗
69. C. J.Schaefer M.Horeni P.Taheri andS.Joshi “LSTMs for Keyword Spotting With ReRAM‐Based Compute‐in‐Memory Architectures ” in2021 IEEE International Symposium on Circuits and Systems (ISCAS)(IEEE 2021) 1–5 https://doi.org/10.1109/ISCAS51556.2021.9401295.10.1109/ISCAS51556.2021.9401295 ↗
70. 10.1038/s41586‐022‐04992‐810.1038/s41586‐022‐04992‐8 ↗
71. 10.1016/j.jallcom.2024.17492610.1016/j.jallcom.2024.174926 ↗
72. 10.1021/acsami.3c0272710.1021/acsami.3c02727 ↗
73. 10.1002/admt.20250028910.1002/admt.202500289 ↗
74. 10.1021/acsami.1c0661810.1021/acsami.1c06618 ↗
75. 10.1002/aisy.20220005310.1002/aisy.202200053 ↗
76. 10.1002/admt.20230139010.1002/admt.202301390 ↗
77. 10.1039/D5MH00086F10.1039/D5MH00086F ↗
78. 10.1063/1.510007510.1063/1.5100075 ↗
79. 10.1002/admt.20240089510.1002/admt.202400895 ↗
80. 10.1016/j.jallcom.2025.17889710.1016/j.jallcom.2025.178897 ↗
81. 10.1109/TED.2024.343808710.1109/TED.2024.3438087 ↗
82. 10.1002/adfm.20241811310.1002/adfm.202418113 ↗
83. 10.1002/advs.20220565410.1002/advs.202205654 ↗
84. 10.1038/s41467‐025‐60970‐410.1038/s41467‐025‐60970‐4 ↗