Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing
Source abstract
Abstract
In-sensor computing, which integrates sensing, memory and processing functions, has shown substantial potential in artificial vision systems. However, large-scale monolithic integration of in-sensor computing based on emerging devices with complementary metal-oxide-semiconductor (CMOS) circuits remains challenging, lacking functional demonstrations at the hardware level. Here we report a fully integrated 1-kb array with 128 × 8 one-transistor one-optoelectronic memristor (OEM) cells and silicon CMOS circuits, which features configurable multi-mode functionality encompassing three different modes of electronic memristor, dynamic OEM and non-volatile OEM (NV-OEM). These modes are configured by modulating the charge density within the oxygen vacancies via synergistic optical and electrical operations, as confirmed by differential phase-contrast scanning transmission electron microscopy. Using this OEM system, three visual processing tasks are demonstrated: image sensory pre-processing with a recognition accuracy enhanced from 85.7% to 96.1% by the NV-OEM mode, more advanced object tracking with 96.1% accuracy using both dynamic OEM and NV-OEM modes and human motion recognition with a fully OEM-based in-sensor reservoir computing system achieving 91.2% accuracy. A system-level benchmark further shows that it consumes over 20 times less energy than graphics processing units. By monolithically integrating the multi-functional OEMs with Si CMOS, this work provides a cost-effective platform for diverse in-sensor computing applications.
Source: europepmc · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Photoreceptor and optogenetics research · CCD and CMOS Imaging Sensors
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (40)
Show source-provided references
1. Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).10.1038/s41565-019-0501-3 ↗
2. Wu, N. Neuromorphic vision chips. Sci. China Inf. Sci. 61, 060421 (2018).10.1007/s11432-017-9303-0 ↗
3. Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).10.1038/s41928-020-00501-9 ↗
4. Dai, S. et al. Emerging iontronic neural devices for neuromorphic sensory computing. Adv. Mater. 35, 2300329 (2023).10.1002/adma.202300329 ↗
5. Du, J. et al. A robust neuromorphic vision sensor with optical control of ferroelectric switching. Nano Energy 89, 106439 (2021).10.1016/j.nanoen.2021.106439 ↗
6. Li, G. et al. Photo-induced non-volatile VO2 phase transition for neuromorphic ultraviolet sensors. Nat. Commun. 13, 1729 (2022).10.1038/s41467-022-29456-5 ↗
7. Jang, H. et al. An atomically thin optoelectronic machine vision processor. Adv. Mater. 32, 2002431 (2020).10.1002/adma.202002431 ↗
8. Zhang, Z. et al. In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array. Nat. Commun. 13, 6590 (2022).10.1038/s41467-022-34230-8 ↗
9. Zhao, R. et al. A framework for the general design and computation of hybrid neural networks. Nat. Commun. 13, 3427 (2022).10.1038/s41467-022-30964-7 ↗
10. Wang, Y. et al. Optoelectronic synaptic devices for neuromorphic computing. Adv. Intell. Syst 3, 2000099 (2021).10.1002/aisy.202000099 ↗
11. Zhang, J., Dai, S., Zhao, Y., Zhang, J. & Huang, J. Recent progress in photonic synapses for neuromorphic systems. Adv. Intell. Syst. 2, 1900136 (2020).10.1002/aisy.201900136 ↗
12. Song, S. et al. Recent progress of optoelectronic and all-optical neuromorphic devices: a comprehensive review of device structures, materials, and applications. Adv. Intell. Syst. 3, 2000119 (2021).10.1002/aisy.202000119 ↗
13. John, R. A. et al. Reconfigurable halide perovskite nanocrystal memristors for neuromorphic computing. Nat. Commun. 13, 2074 (2022).10.1038/s41467-022-29727-1 ↗
14. Wang, T. et al. Reconfigurable neuromorphic memristor network for ultralow-power smart textile electronics. Nat. Commun. 13, 7432 (2022).10.1038/s41467-022-35160-1 ↗
15. Pi, L. et al. Broadband convolutional processing using band-alignment-tunable heterostructures. Nat. Electron. 5, 248–254 (2022).10.1038/s41928-022-00747-5 ↗
16. Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).10.1038/s41586-020-2038-x ↗
17. Zhang, Z. et al. All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat. Nanotechnol. 17, 27–32 (2022).10.1038/s41565-021-01003-1 ↗
18. Sun, L. et al. In-sensor reservoir computing for language learning via two-dimensional memristors. Sci. Adv. 7, eabg1455 (2021).10.1126/sciadv.abg1455 ↗
19. Wu, X. et al. Wearable in-sensor reservoir computing using optoelectronic polymers with through-space charge-transport characteristics for multi-task learning. Nat. Commun. 14, 468 (2023).10.1038/s41467-023-36205-9 ↗
20. Lao, J. et al. Ultralow-power machine vision with self-powered sensor reservoir. Adv. Sci. 9, 2106092 (2022).10.1002/advs.202106092 ↗
21. Zhong, Y. et al. A memristor-based analogue reservoir computing system for real-time and power-efficient signal processing. Nat. Electron. 5, 672–681 (2022).10.1038/s41928-022-00838-3 ↗
22. Liang, X. et al. Physical reservoir computing with emerging electronics. Nat. Electron. 7, 193–206 (2024).10.1038/s41928-024-01133-z ↗
23. Moon, J. et al. Temporal data classification and forecasting using a memristor-based reservoir computing system. Nat. Electron. 2, 480–487 (2019).10.1038/s41928-019-0313-3 ↗
24. Portner, K. et al. Analog nanoscale electro-optical synapses for neuromorphic computing applications. ACS. Nano. 15, 14776–14785 (2021).10.1021/acsnano.1c04654 ↗
25. Hu, L. et al. All-optically controlled memristor for optoelectronic neuromorphic computing. Adv. Funct. Mater. 31, 2005582 (2021).10.1002/adfm.202005582 ↗
26. Tan, H. et al. An optoelectronic resistive switching memory with integrated demodulating and arithmetic functions. Adv. Mater. 27, 2797–2803 (2015).10.1002/adma.201500039 ↗
27. Chen, J. Y. et al. Dynamic evolution of conducting nanofilament in resistive switching memories. Nano Lett. 13, 3671–3677 (2013).10.1021/nl4015638 ↗
28. Simanjuntak, F. M., Panda, D., Wei, K. H. & Tseng, T. Y. Status and prospects of ZnO-based resistive switching memory devices. Nanoscale Res. Lett. 11, 368 (2016).10.1186/s11671-016-1570-y ↗
29. Xu, N. et al. Characteristics and mechanism of conduction/set process in TiN/ZnO/Pt resistance switching random-access memories. Appl. Phys. Lett. 92, 232112 (2008).10.1063/1.2945278 ↗
30. Zhou, Z., Pei, Y., Zhao, J., Fu, G. & Yan, X. Visible light responsive optoelectronic memristor device based on CeOx/ZnO structure for artificial vision system. Appl. Phys. Lett. 118, 191103 (2021).10.1063/5.0047403 ↗
31. Wang, T.-Y. et al. Reconfigurable optoelectronic memristor for in-sensor computing applications. Nano Energy 89, 106291 (2021).10.1016/j.nanoen.2021.106291 ↗
32. Wang, W. et al. CMOS backend-of-line compatible memory array and logic circuitries enabled by high performance atomic layer deposited ZnO thin-film transistor. Nat. Commun. 14, 6079 (2023).10.1038/s41467-023-41868-5 ↗
33. Wang, Z. et al. Vacancy driven surface disorder catalyzes anisotropic evaporation of ZnO (0001) polar surface. Nat. Commun. 13, 5616 (2022).10.1038/s41467-022-33353-2 ↗
34. Lanza, M. et al. Recommended methods to study resistive switching devices. Adv. Electron. Mater. 5, 1800143 (2019).
35. Kuzum, D., Yu, S. & Wong, H. S. Synaptic electronics: materials, devices and applications. Nanotechnology 24, 382001 (2013).10.1088/0957-4484/24/38/382001 ↗
36. Russo, P., Xiao, M., Liang, R. & Zhou, N. Y. UV-induced multilevel current amplification memory effect in zinc oxide rods resistive switching devices. Adv. Funct. Mater. 28, 1706230 (2018).10.1002/adfm.201706230 ↗
37. Oh, I., Pyo, J. & Kim, S. Resistive switching and synaptic characteristics in ZnO/TaON-based RRAM for neuromorphic system. Nanomaterial 12, 2185 (2022).10.3390/nano12132185 ↗
38. Seo, S. et al. Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat. Commun. 9, 5106 (2018).10.1038/s41467-018-07572-5 ↗
39. Yilmaz, A., Javed, O. & Shah, M. Object tracking. ACM Comput. Surv. 38, 1–45 (2006).10.1145/1177352.1177355 ↗
40. Wang, S. et al. Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci. Rev. 8, nwaa172 (2020).10.1093/nsr/nwaa172 ↗