Spiking Neural Networks for Object Detection Based on Integrating Neuronal Variants and Self-Attention Mechanisms
Source abstract
Abstract
Thanks to their event-driven asynchronous computing capabilities and low power consumption advantages, spiking neural networks (SNNs) show significant potential for computer vision tasks, especially in object detection. However, effective training methods and optimization mechanisms for SNNs remain underexplored. This study proposes two high accuracy SNNs for object detection, AMS_YOLO and AMSpiking_VGG, integrating neuronal variants and attention mechanisms. To enhance these proposed networks, we explore the impact of incorporating different neuronal variants.The results show that the optimization in the SNN’s structure with neuronal variants outperforms that in the attention mechanism for object detection. Compared to the state-of-the-art in the current SNNs, AMS_YOLO improved by 6.7% in accuracy on the static dataset COCO2017, and AMS_Spiking has improved by 11.4% on the dynamic dataset GEN1.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Neural dynamics and brain function · Neuroscience and Neural Engineering
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Engineering, Capital Normal University, Beijing 100048, China
School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China
Funding
National Natural Science Foundation of China · R&D Program of Beijing Municipal Education Commission · Beijing Natural Science Foundation
References (29)
Show source-provided references
1. Networks of spiking neurons: The third generation of neural network models10.1016/S0893-6080(97)00011-7 ↗
2. A review of the integrate-and-fire neuron model: I. homogeneous synaptic input10.1007/s00422-006-0068-6 ↗
3. Lapicque’s introduction of the integrate-and-fire model neuron (1907)10.1016/S0361-9230(99)00161-6 ↗
4. Gerstner, W., and Kistler, W.M. (2002). Spiking Neuron Models: Single Neurons, Populations, Plasticity, Cambridge University Press.10.1017/CBO9780511815706 ↗
5. Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., and Tian, Y. (2021, January 11–17). Incorporating learnable membrane time constant to enhance learning of spiking neural networks. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00266 ↗
6. Stdp-based spiking deep convolutional neural networks for object recognition10.1016/j.neunet.2017.12.005 ↗
7. Abhronil, S., Ye, Y., Wang, R., Liu, C., and Roy, K. (2019). Going deeper in spiking neural networks: Vgg and residual architectures. Front. Neurosci., 13.10.3389/fnins.2019.00095 ↗
8. Wu, Y., Deng, L., Li, G., Zhu, J., Xie, Y., and Shi, L. (February, January 27). Direct training for spiking neural networks: Faster, larger, better. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
9. Wu, Y., Deng, L., Li, G., Zhu, J., and Shi, L. (2018). Spatio-temporal backpropagation for training high-performance spiking neural networks. Front. Neurosci., 12.10.3389/fnins.2018.00331 ↗
10. Zheng, H., Wu, Y., Deng, L., Hu, Y., and Li, G. (2021, January 2–9). Going deeper with directly-trained larger spiking neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, virtually.10.1609/aaai.v35i12.17320 ↗
11. Temporal effective batch normalization in spiking neural networks
12. Deng, S., Li, Y., Zhang, S., and Gu, S. (2022). Temporal efficient training of spiking neural network via gradient re-weighting. arXiv.
13. Deep residual learning in spiking neural networks
14. Tuning convolutional spiking neural network with biologically plausible reward propagation10.1109/TNNLS.2021.3085966 ↗
15. Kim, S., Park, S., Na, B., and Yoon, S. (2020, January 7–12). Spiking-yolo: Spiking neural network for energy-efficient object detection. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6787 ↗
16. Cordone, L., Miramond, B., and Thierion, P. (2022, January 18–23). Object detection with spiking neural networks on automotive event data. Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy.10.1109/IJCNN55064.2022.9892618 ↗
17. Su, Q., Chou, Y., Hu, Y., Li, J., Mei, S., Zhang, Z., and Li, G. (2023, January 2–3). Deep directly-trained spiking neural networks for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00603 ↗
18. A solution to the learning dilemma for recurrent networks of spiking neurons10.1038/s41467-020-17236-y ↗
19. Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks10.1126/sciadv.abh0146 ↗
20. Sa-flownet: Event-based self-attention optical flow estimation with spiking-analogue neural networks10.1049/cvi2.12206 ↗
21. Eagan: Event-based attention generative adversarial networks for optical flow and depth estimation10.1049/cvi2.12115 ↗
22. Event-based vision: A survey10.1109/TPAMI.2020.3008413 ↗
23. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the Computer Vision—ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part V 13.10.1007/978-3-319-10602-1_48 ↗
24. Lee, C., Kosta, A.K., Zhu, A.Z., Chaney, K., Daniilidis, K., and Roy, K. (2020, January 23–28). Spike-flownet: Event-based optical flow estimation with energy-efficient hybrid neural networks. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58526-6_22 ↗
25. Woo, S., Park, J., Lee, J., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
26. Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., and Liu, W. (November, January 27). Ccnet: Criss-cross attention for semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
27. Yao, M., Gao, H., Zhao, G., Wang, D., Lin, Y., Yang, Z., and Li, G. (2021, January 11–17). Temporal-wise attention spiking neural networks for event streams classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01006 ↗
28. Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., and Lu, H. (2019, January 15–20). Dual attention network for scene segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00326 ↗
29. Pan, X., Ge, C., Lu, R., Song, S., Chen, G., Huang, Z., and Huang, G. (2022, January 18–24). On the integration of self-attention and convolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00089 ↗