In-Sensor Compressed Imaging with Reconstruction-Free Recognition via Ferroelectric Photodiodes
Source abstract
Abstract
The transition from human-centric to machine-centric vision systems demands innovative imaging architectures. Compressed imaging, using key mathematical transformations to capture essential visual information during sampling, is suited for sensor-level integration to minimize data redundancy. Here, we propose an efficient machine vision strategy that connects in-sensor compressed imaging with neural networks. By manipulating ferroelectric polarization, BiFeO 3 photodiodes exhibit 113 to 274 nonvolatile photoresponse states with good stability, linearity, and an 85% yield across 150 devices. The Hadamard product between the devices and image matrices transforms the input images from the spatial to the Haar wavelet domain, enabling in-sensor compression by discarding high-frequency coefficients. The compressed data are directly fed into neural networks, bypassing image reconstruction, and achieve a simulated accuracy of 87.0% at 0.04 compression ratio and 95.8% at 0.9. This approach merges image sensing and compression within the ferroelectric sensors, integrating reconstruction-free deep processing for an efficient machine vision solution.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Sparse and Compressive Sensing Techniques · Photoacoustic and Ultrasonic Imaging · CCD and CMOS Imaging Sensors
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Beijing National Laboratory for Condensed Matter Physics
Institute of Physics, Chinese Academy of Sciences
School of Physical Sciences
University of Chinese Academy of Sciences
CAS Key Laboratory of Nanosystem and Hierarchical Fabrication
National Center for Nanoscience and Technology
Center of Materials Science and Optoelectronics Engineering
School of Physics
Liaoning University
Funding
National Key Research and Development Program of China · National Natural Science Foundation of China
References (50)
Show source-provided references
1. 10.1145/306538610.1145/3065386 ↗
2. Hsu T.-H. · 2019 · 22.5.110.1109/IEDM19573.2019.8993452 ↗
3. 10.1016/S0031-3203(01)00178-910.1016/S0031-3203(01)00178-9 ↗
4. 10.1038/s41565-024-01665-710.1038/s41565-024-01665-7 ↗
5. 10.1109/TCSI.2019.293722710.1109/TCSI.2019.2937227 ↗
6. 10.1021/acs.nanolett.3c0089910.1021/acs.nanolett.3c00899 ↗
7. 10.1002/adma.20242033310.1002/adma.202420333 ↗
8. 10.1007/s11220-018-0212-010.1007/s11220-018-0212-0 ↗
9. 10.1364/OE.25.00980210.1364/OE.25.009802 ↗
10. 10.14257/ijsia.2015.9.12.2610.14257/ijsia.2015.9.12.26 ↗
11. 10.1038/s41586-020-2038-x10.1038/s41586-020-2038-x ↗
12. 10.1038/s41467-024-55412-610.1038/s41467-024-55412-6 ↗
13. 10.1021/acs.nanolett.4c0119010.1021/acs.nanolett.4c01190 ↗
14. Dut, Z.; Fasthuber, R.; Chen, T.; Feng, X.; Chen, Y.; Ienne, P.; Li, L.; Temam, O., ShiDianNao: shifting vision processing closer to the sensor. In 2015 ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA); IEEE: 2015; pp 92–104.10.1145/2749469.2750389 ↗
15. 10.1038/s41467-024-55508-z10.1038/s41467-024-55508-z ↗
16. 10.1002/adma.20240952010.1002/adma.202409520 ↗
17. 10.1002/adfm.20221291710.1002/adfm.202212917 ↗
18. 10.1002/adma.20241411310.1002/adma.202414113 ↗
19. 10.1038/s41467-025-56233-x10.1038/s41467-025-56233-x ↗
20. 10.1177/002029401987750810.1177/0020294019877508 ↗
21. 10.1109/TIT.2005.86208310.1109/TIT.2005.862083 ↗
22. 10.1109/TIT.2006.87158210.1109/TIT.2006.871582 ↗
23. 10.1109/ACCESS.2018.279385110.1109/ACCESS.2018.2793851 ↗
24. 10.1038/s41928-024-01328-410.1038/s41928-024-01328-4 ↗
25. 10.1038/s41467-024-45312-010.1038/s41467-024-45312-0 ↗
26. 10.1002/cpa.2012410.1002/cpa.20124 ↗
27. 10.1109/JPROC.2010.204142210.1109/JPROC.2010.2041422 ↗
28. 10.1109/ACCESS.2021.306857910.1109/ACCESS.2021.3068579 ↗
29. 10.1109/JSSC.2012.221485110.1109/JSSC.2012.2214851 ↗
30. 10.1109/JSEN.2012.221914310.1109/JSEN.2012.2219143 ↗
31. 10.1109/JSEN.2015.239787410.1109/JSEN.2015.2397874 ↗
32. 10.1109/ACCESS.2020.300686110.1109/ACCESS.2020.3006861 ↗
33. 10.1038/s41467-024-47580-210.1038/s41467-024-47580-2 ↗
34. Ma, T.; Boloor, A. J.; Yang, X.; Cao, W.; Williams, P.; Sun, N.; Chakrabarti, A.; Zhang, X., LeCA: In-sensor learned compressive acquisition for efficient mashine vision on the edge. In Proceedings of the 50th Annual International Symposium on Computer Architecture (ISCA); ACM: 2023; pp 1–14.10.1145/3579371.3589089 ↗
35. 10.1007/s11760-021-01872-y10.1007/s11760-021-01872-y ↗
36. 10.1038/s41598-019-44119-010.1038/s41598-019-44119-0 ↗
37. 10.1103/PhysRevB.84.12530510.1103/PhysRevB.84.125305 ↗
38. 10.1063/1.358981410.1063/1.3589814 ↗
39. 10.1126/science.116863610.1126/science.1168636 ↗
40. 10.1002/adfm.20190623210.1002/adfm.201906232 ↗
41. 10.1103/PhysRevB.105.13410310.1103/PhysRevB.105.134103 ↗
42. 10.1038/s41467-023-37823-z10.1038/s41467-023-37823-z ↗
43. Zu, X.; Tu, M.; Zhang, T.; Hao, M., General Gross Error Identification for Maximum-error Specification and Measurement Uncertainty. In 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), 2020; pp 1743–1746.10.1109/ITAIC49862.2020.9339185 ↗
44. 10.1007/978-981-10-2663-8_5110.1007/978-981-10-2663-8_51 ↗
45. 10.1145/336583610.1145/3365836 ↗
46. 10.1109/TSP.2017.277342710.1109/TSP.2017.2773427 ↗
47. 10.1109/TIM.2021.308356110.1109/TIM.2021.3083561 ↗
48. 10.1109/TPAMI.2015.246928810.1109/TPAMI.2015.2469288 ↗
49. Chadha, A.; Abbas, A.; Andreopoulos, Y. Compressed-domain video classification with deep neural networks: “There’s way too much information to decode the matrix”, IEEE Int. Conf. Image Process., 2017; pp 1832–1836.10.1109/ICIP.2017.8296598 ↗
50. 10.1038/s41467-025-59104-710.1038/s41467-025-59104-7 ↗