journal article·2024-11Open access

Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio‐inspired Electronic Eyes and Neuromorphic Image Sensors

Changsoon Choi, Gil Ju Lee, Sehui Chang, Young Min Song, Dae‐Hyeong Kim

Advanced Materials· Volume 36· Issue 48

Abstract

Abstract In robotics, particularly for autonomous navigation and human–robot collaboration, the significance of unconventional imaging techniques and efficient data processing capabilities is paramount. The unstructured environments encountered by robots, coupled with complex missions assigned to them, present numerous challenges necessitating diverse visual functionalities, and consequently, the development of multifunctional robotic vision systems has become indispensable. Meanwhile, rich diversity inherent in animal vision systems, honed over evolutionary epochs to meet their survival demands across varied habitats, serves as a profound source of inspirations. Here, recent advancements in multifunctional robotic vision systems drawing inspiration from natural ocular structures and their visual perception mechanisms are delineated. First, unique imaging functionalities of natural eyes across terrestrial, aerial, and aquatic habitats and visual signal processing mechanism of humans are explored. Then, designs and functionalities of bio‐inspired electronic eyes are explored, engineered to mimic key components and underlying optical principles of natural eyes. Furthermore, neuromorphic image sensors are discussed, emulating functional properties of synapses, neurons, and retinas and thereby enhancing accuracy and efficiency of robotic vision tasks. Next, integration examples of electronic eyes with mobile robotic/biological systems are introduced. Finally, a forward‐looking outlook on the development of bio‐inspired electronic eyes and neuromorphic image sensors is provided.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Memory and Neural Computing · Photoreceptor and optogenetics research · Neuroscience and Neural Engineering

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Center for Quantum Technology Korea Institute of Science and Technology Seoul 02792 Republic of Korea

School of Electrical and Electronics Engineering Pusan National University Busan 46241 Republic of Korea

School of Electrical Engineering and Computer Science Gwangju Institute of Science and Technology Gwangju 61005 Republic of Korea

AI Graduate School Gwangju Institute of Science and Technology Gwangju 61005 Republic of Korea

Department of Semiconductor Engineering Gwangju Institute of Science and Technology Gwangju 61005 Republic of Korea

Center for Nanoparticle Research Institute for Basic Science (IBS) Seoul 08826 Republic of Korea

School of Chemical and Biological Engineering, Institute of Chemical Processes Seoul National University Seoul 08826 Republic of Korea

Funding

National Research Foundation of Korea · Korea Institute of Science and Technology · Ministry of Science and ICT, South Korea · Institute for Basic Science

References (100+)

Show source-provided references
1. 10.1038/s41586-020-2285-x10.1038/s41586-020-2285-x ↗
2. 10.1038/s41928-021-00600-110.1038/s41928-021-00600-1 ↗
3. 10.1038/s41565-022-01160-x10.1038/s41565-022-01160-x ↗
4. 10.1038/nature0711310.1038/nature07113 ↗
5. 10.1038/s41565-023-01379-210.1038/s41565-023-01379-2 ↗
6. 10.1038/s41928-022-00819-610.1038/s41928-022-00819-6 ↗
7. 10.1126/scirobotics.adl360610.1126/scirobotics.adl3606 ↗
8. 10.1038/s41467-023-43944-210.1038/s41467-023-43944-2 ↗
9. 10.1126/scirobotics.adi866610.1126/scirobotics.adi8666 ↗
10. 10.1038/s41928-023-01055-210.1038/s41928-023-01055-2 ↗
11. 10.1002/adma.20240747610.1002/adma.202407476 ↗
12. 10.1038/s41928-020-00501-910.1038/s41928-020-00501-9 ↗
13. 10.1021/acsnano.3c1018110.1021/acsnano.3c10181 ↗
14. 10.1002/adma.20220383010.1002/adma.202203830 ↗
15. 10.1126/scirobotics.ade469810.1126/scirobotics.ade4698 ↗
16. 10.1002/inf2.1227510.1002/inf2.12275 ↗
17. 10.1109/JSTQE.2023.330877010.1109/JSTQE.2023.3308770 ↗
18. 10.1016/j.mtelec.2023.10007110.1016/j.mtelec.2023.100071 ↗
19. 10.1016/j.cois.2018.09.00510.1016/j.cois.2018.09.005 ↗
20. 10.1038/s41928-024-01124-010.1038/s41928-024-01124-0 ↗
21. 10.1126/scirobotics.adi059110.1126/scirobotics.adi0591 ↗
22. 10.1038/s41467-021-21917-710.1038/s41467-021-21917-7 ↗
23. 10.1021/acsnano.2c0787710.1021/acsnano.2c07877 ↗
24. 10.1260/17568290978949828810.1260/175682909789498288 ↗
25. 10.1109/JFLEX.2022.316216910.1109/JFLEX.2022.3162169 ↗
26. 10.3389/fnins.2021.67216110.3389/fnins.2021.672161 ↗
27. 10.1038/s41928-020-0429-510.1038/s41928-020-0429-5 ↗
28. 10.1073/pnas.151795311310.1073/pnas.1517953113 ↗
29. 10.1038/s41928-022-00789-910.1038/s41928-022-00789-9 ↗
30. R.Kanth J.Heikkonen inProc. 2023 IEEE International Conference on Emerging Trends in Engineering Sciences and Technology (ICES&T) Bahawalpur India2023.
31. 10.1109/ACCESS.2023.331611610.1109/ACCESS.2023.3316116 ↗
32. 10.1109/TCYB.2021.307131210.1109/TCYB.2021.3071312 ↗
33. 10.1002/aisy.20190013810.1002/aisy.201900138 ↗
34. 10.1126/scirobotics.abj290810.1126/scirobotics.abj2908 ↗
35. 10.1038/s41467-023-37886-y10.1038/s41467-023-37886-y ↗
36. 10.1126/sciadv.adi408310.1126/sciadv.adi4083 ↗
37. 10.1088/1361-6633/ab6a4210.1088/1361-6633/ab6a42 ↗
38. 10.1002/adfm.20200928110.1002/adfm.202009281 ↗
39. 10.1080/15599612.2024.233429310.1080/15599612.2024.2334293 ↗
40. 10.1016/j.optlastec.2017.04.02610.1016/j.optlastec.2017.04.026 ↗
41. 10.1002/admt.20210014410.1002/admt.202100144 ↗
42. 10.1002/mame.20210012110.1002/mame.202100121 ↗
43. 10.1038/lsa.2013.5410.1038/lsa.2013.54 ↗
44. 10.1021/acsomega.3c0044010.1021/acsomega.3c00440 ↗
45. 10.1038/s41565-024-01665-710.1038/s41565-024-01665-7 ↗
46. 10.1038/s41467-022-32790-310.1038/s41467-022-32790-3 ↗
47. 10.1016/j.device.2024.10035710.1016/j.device.2024.100357 ↗
48. 10.1021/acs.nanolett.1c0324010.1021/acs.nanolett.1c03240 ↗
49. 10.1021/acs.nanolett.4c0172710.1021/acs.nanolett.4c01727 ↗
50. 10.1038/s41563-023-01704-z10.1038/s41563-023-01704-z ↗
51. 10.1002/adfm.20210844010.1002/adfm.202108440 ↗
52. D.Shah B.Osiński B.Ichter S.Levine Proceedings of the 6th Conference on Robot learning. PMLR2023.
53. 10.1002/adfm.20170520210.1002/adfm.201705202 ↗
54. 10.1002/adma.20230198610.1002/adma.202301986 ↗
55. 10.2307/131111210.2307/1311112 ↗
56. 10.1039/D3TC01883K10.1039/D3TC01883K ↗
57. 10.1515/nanoph-2024-001410.1515/nanoph-2024-0014 ↗
58. 10.1126/sciadv.150039110.1126/sciadv.1500391 ↗
59. 10.1038/nrn149710.1038/nrn1497 ↗
60. 10.1038/s41928-020-00466-910.1038/s41928-020-00466-9 ↗
61. 10.1038/s41583-019-0242-110.1038/s41583-019-0242-1 ↗
62. 10.1021/acsnano.1c0675810.1021/acsnano.1c06758 ↗
63. 10.1038/s41467-018-07572-510.1038/s41467-018-07572-5 ↗
64. 10.1103/PhysRevE.89.02272110.1103/PhysRevE.89.022721 ↗
65. 10.1038/s41928-024-01208-x10.1038/s41928-024-01208-x ↗
66. 10.1038/s41377-018-0081-210.1038/s41377-018-0081-2 ↗
67. 10.1002/adma.20230909910.1002/adma.202309099 ↗
68. 10.1002/adma.20230738010.1002/adma.202307380 ↗
69. 10.1126/science.ade122010.1126/science.ade1220 ↗
70. 10.1038/lsa.2016.6810.1038/lsa.2016.68 ↗
71. 10.1002/adma.20210897910.1002/adma.202108979 ↗
72. 10.1038/s41377-023-01072-y10.1038/s41377-023-01072-y ↗
73. 10.1364/AO.50.00326810.1364/AO.50.003268 ↗
74. 10.1016/j.matt.2022.02.02010.1016/j.matt.2022.02.020 ↗
75. 10.1002/adfm.20190376210.1002/adfm.201903762 ↗
76. 10.1073/pnas.170109211410.1073/pnas.1701092114 ↗
77. 10.1038/s41467-024-50271-710.1038/s41467-024-50271-7 ↗
78. 10.1073/pnas.191215411610.1073/pnas.1912154116 ↗
79. 10.1002/adfm.20220676510.1002/adfm.202206765 ↗
80. 10.1002/adfm.20230567710.1002/adfm.202305677 ↗
81. 10.1126/sciadv.160265510.1126/sciadv.1602655 ↗
82. Park J. · Sci. Adv. · 2024 · 9
83. 10.1111/ahe.1243210.1111/ahe.12432 ↗
84. 10.1038/s41467-021-26606-z10.1038/s41467-021-26606-z ↗
85. 10.1073/pnas.121906811010.1073/pnas.1219068110 ↗
86. 10.1002/smll.20120010710.1002/smll.201200107 ↗
87. 10.1364/AO.48.00336810.1364/AO.48.003368 ↗
88. 10.1002/adfm.20150494110.1002/adfm.201504941 ↗
89. 10.1038/s42003-023-04575-x10.1038/s42003-023-04575-x ↗
90. 10.1038/s41587-023-01717-810.1038/s41587-023-01717-8 ↗
91. 10.1016/j.preteyeres.2012.12.00110.1016/j.preteyeres.2012.12.001 ↗
92. 10.1016/0042-6989(73)90160-010.1016/0042-6989(73)90160-0 ↗
93. 10.1016/j.visres.2013.02.01610.1016/j.visres.2013.02.016 ↗
94. 10.1126/sciadv.abe319610.1126/sciadv.abe3196 ↗
95. 10.1038/s41467-023-43034-310.1038/s41467-023-43034-3 ↗
96. 10.1002/cne.2347210.1002/cne.23472 ↗
97. 10.1038/s41467-022-34565-210.1038/s41467-022-34565-2 ↗
98. 10.1109/JPROC.2014.234615310.1109/JPROC.2014.2346153 ↗
99. 10.1002/adma.20220681610.1002/adma.202206816 ↗
100. 10.1038/s41928-022-00713-110.1038/s41928-022-00713-1 ↗