A Review of Computing with Spiking Neural Networks
Source abstract
Abstract
Artificial neural networks (ANNs) have led to landmark changes in many fields, but they still differ significantly from the mechanisms of real biological neural networks and face problems such as high computing costs, excessive computing power, and so on. Spiking neural networks (SNNs) provide a new approach combined with brain-like science to improve the computational energy efficiency, computational architecture, and biological credibility of current deep learning applications. In the early stage of development, its poor performance hindered the application of SNNs in real-world scenarios. In recent years, SNNs have made great progress in computational performance and practicability compared with the earlier research results, and are continuously producing significant results. Although there are already many pieces of literature on SNNs, there is still a lack of comprehensive review on SNNs from the perspective of improving performance and practicality as well as incorporating the latest research results. Starting from this issue, this paper elaborates on SNNs along the complete usage process of SNNs including network construction, data processing, model training, development, and deployment, aiming to provide more comprehensive and practical guidance to promote the development of SNNs. Therefore, the connotation and development status of SNN computing is reviewed systematically and comprehensively from four aspects: composition structure, data set, learning algorithm, software/hardware development platform. Then the development characteristics of SNNs in intelligent computing are summarized, the current challenges of SNNs are discussed and the future development directions are also prospected. Our research shows that in the fields of machine learning and intelligent computing, SNNs have comparable network scale and performance to ANNs and the ability to challenge large datasets and a variety of tasks. The advantages of SNNs over ANNs in terms of energy efficiency and spatial-temporal data processing have been more fully exploited. And the development of programming and deployment tools has lowered the threshold for the use of SNNs. SNNs show a broad development prospect for brain-like computing.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Advanced Memory and Neural Computing · Ferroelectric and Negative Capacitance Devices · Neural dynamics and brain function
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Synergic deep learning for smart health diagnosis of COVID-19 for connected living and smart cities10.1145/3453168 ↗
2. False alert detection based on deep learning and machine learning10.4018/IJSWIS.313190 ↗
3. Comparing deep learning models for low-light natural scene image enhancement and their impact on object detection and classification: Overview, empirical evaluation, and challenges10.1016/j.image.2022.116848 ↗
4. A robust intelligent system for detecting tomato crop diseases using deep learning
5. Language models are few-shot learners
6. Neural networks and neuroscience-inspired computer vision (review)10.1016/j.cub.2014.08.026 ↗
7. Spiking neural networks: A survey10.1109/ACCESS.2022.3179968 ↗
8. A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input10.1007/s00422-006-0068-6 ↗
9. Gerstner · Spiking Neuron Models: Single Neurons Populations Plasticity · 2002 · 94
10. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification10.3389/fnins.2017.00682 ↗
11. Which model to use for cortical spiking neurons?10.1109/TNN.2004.832719 ↗
12. Incorporating learnable membrane time constant to enhance learning of spiking neural networks
13. Constructing accurate and efficient deep spiking neural networks with double-threshold and augmented schemes10.1109/TNNLS.2020.3043415 ↗
14. Signed neuron with memory: Towards simple, accurate and high-efficient ANN-SNN conversion
15. Spiking-YOLO: Spiking neural network for real-time object detection
16. The implementation and optimization of neuromorphic hardware for supporting spiking neural networks with MLP and CNN topologies10.1109/TCAD.2022.3179246 ↗
17. Gerstner · Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition · 2014 · 154
18. Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns10.1007/BF00199450 ↗
19. Advancements in algorithms and neuromorphic hardware for spiking neural networks10.1162/neco_a_01499 ↗
20. Simplified spiking neural network architecture and STDP learning algorithm applied to image classification
21. A quantitative description of membrane current and its application to conduction and excitation in nerve10.1113/jphysiol.1952.sp004764 ↗
22. Simple model of spiking neurons10.1109/TNN.2003.820440 ↗
23. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
24. A. Krizhevsky, “Learning multiple layers of features from tiny images,” Accessed: May 19, 2023. [Online]. Available: https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf
25. Gerstner · Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition · 2014 · 172
26. Time to first spike in stochastic Hodgkin-Huxley systems10.1016/j.physa.2004.11.059 ↗
27. Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns10.1016/j.neuron.2009.01.008 ↗
28. Thorpe · Computational Neuroscience: Trends in Research 1998 · 1998 · 11310.1007/978-1-4615-4831-7_19 ↗
29. Interspike-interval-based analog spike-time-dependent encoder for neuromorphic processors10.1109/TVLSI.2017.2683260 ↗
30. A survey of encoding techniques for signal processing in spiking neural networks10.1007/s11063-021-10562-2 ↗
31. Selection and optimization of temporal spike encoding methods for spiking neural networks10.1109/TNNLS.2019.2906158 ↗
32. Rate coding or direct coding: Which one is better for accurate, robust, and energy-efficient spiking neural networks?
33. Direct training for spiking neural networks: Faster, larger, better
34. Spiking neural networks for frame-based and event-based single object localization10.1016/j.neucom.2023.126805 ↗
35. Object detection with spiking neural networks on automotive event data
36. Converting static image datasets to spiking neuromorphic datasets using saccades
37. CIFAR10-DVS: An event-stream dataset for object classification10.3389/fnins.2017.00309 ↗
38. ES-ImageNet: A million event-stream classification dataset for spiking neural networks10.3389/fnins.2021.726582 ↗
39. A low power, fully event-based gesture recognition system
40. DVS benchmark datasets for object tracking, action recognition, and object recognition10.3389/fnins.2016.00405 ↗
41. DSEC: A stereo event camera dataset for driving scenarios10.1109/LRA.2021.3068942 ↗
42. Feature representations for neuromorphic audio spike streams10.3389/fnins.2018.00023 ↗
43. A review of learning in biologically plausible spiking neural networks10.1016/j.neunet.2019.09.036 ↗
44. A survey of deep active learning
45. A comprehensive survey on transfer learning10.1109/JPROC.2020.3004555 ↗
46. T-NGA: Temporal network grafting algorithm for learning to process spiking audio sensor events
47. A database for speaker-independent digit recognition
48. Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type10.1523/JNEUROSCI.18-24-10464.1998 ↗
49. Competitive hebbian learning through spike-timing-dependent synaptic plasticity10.1038/78829 ↗
50. Hebb · The Organization of Behavior: A Neuropsychological Theory · 1949 · 107
51. Synaptic equalization by anti-STDP10.1016/j.neucom.2004.01.067 ↗
52. Triplets of spikes in a model of spike timing-dependent plasticity10.1523/JNEUROSCI.1425-06.2006 ↗
53. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception10.1109/TPAMI.2019.2903179 ↗
54. STDP-based spiking deep convolutional neural networks for object recognition10.1016/j.neunet.2017.12.005 ↗
55. Multi-layer unsupervised learning in a spiking convolutional neural network
56. Li · Neurocomput. · 2021 · 463 · 25110.1016/j.neucom.2021.08.011 ↗
57. Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classification, and spike shifting10.1162/neco.2009.11-08-901 ↗
58. Supervised learning in multilayer spiking neural networks10.1162/NECO_a_00396 ↗
59. Enhanced spiking neural network with forgetting phenomenon based on electronic synaptic devices10.1016/j.neucom.2019.09.030 ↗
60. Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks10.1016/j.patcog.2019.05.015 ↗
61. Training deep spiking convolutional neural networks with STDP-based unsupervised pre-training followed by supervised fine-tuning10.3389/fnins.2018.00435 ↗
62. Deep CovDenseSNN: A hierarchical event-driven dynamic framework with spiking neurons in noisy environment10.1016/j.neunet.2019.08.034 ↗
63. EICIL: Joint excitatory inhibitory cycle iteration learning for deep spiking neural networks
64. Error-backpropagation in temporally encoded networks of spiking neurons10.1016/S0925-2312(01)00658-0 ↗
65. A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection10.1016/j.neunet.2009.04.003 ↗
66. Learning beyond finite memory in recurrent networks of spiking neurons10.1162/neco.2006.18.3.591 ↗
67. The tempotron: A neuron that learns spike timing-based decisions10.1038/nn1643 ↗
68. The chronotron: A neuron that learns to fire temporally-precise spike patterns10.1371/journal.pone.0040233 ↗
69. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks10.1109/MSP.2019.2931595 ↗
70. SLAYER: Spike layer error reassignment in time
71. Spatio-temporal backpropagation for training high-performance spiking neural networks10.3389/fnins.2018.00331 ↗
72. Enabling spike-based backpropagation for training deep neural network architectures10.3389/fnins.2020.00119 ↗
73. Direct learning-based deep spiking neural networks: A review10.3389/fnins.2023.1209795 ↗
74. ImageNet: A large-scale hierarchical image database
75. Going deeper with directly-trained larger spiking neural networks
76. Deep residual learning in spiking neural networks
77. Multi-level firing with spiking DS-ResNet: Enabling better and deeper directly-trained spiking neural networks
78. Temporal efficient training of spiking neural network via gradient re-weighting
79. Deep directly-trained spiking neural networks for object detection
80. Spikformer: When spiking neural network meets transformer
81. Spatial-temporal self-attention for asynchronous spiking neural networks
82. Going deeper in spiking neural networks: VGG and residual architectures10.3389/fnins.2019.00095 ↗
83. Spiking deep residual networks
84. Efficient processing of spatio-temporal data streams with spiking neural networks10.3389/fnins.2020.00439 ↗
85. The streaming rollout of deep networks—towards fully model—parallel execution
86. Efficient and accurate conversion of spiking neural network with burst spikes
87. Optimized potential initialization for low-latency spiking neural networks
88. The neuron simulation environment10.1162/neco.1997.9.6.1179 ↗
89. GENESIS 1.1: A hybrid-parallel molecular dynamics simulator with enhanced sampling algorithms on multiple computational platforms10.1002/jcc.24874 ↗
90. Plesser · Encyclopedia of Computational Neuroscience · 2019 · 1
91. Brian 2, an intuitive and efficient neural simulator10.7554/eLife.47314 ↗
92. Nengo: A python tool for building large-scale functional brain models10.3389/fninf.2013.00048 ↗
93. GeNN: A code generation framework for accelerated brain simulations10.1038/srep18854 ↗
94. BindsNET: A machine learning-oriented spiking neural networks library in python10.3389/fninf.2018.00089 ↗
95. SpykeTorch: Efficient simulation of convolutional spiking neural networks with at most one spike per neuron10.3389/fnins.2019.00625 ↗
96. Norse—A deep learning library for spiking neural networks
97. Training spiking neural networks using lessons from deep learning10.1109/JPROC.2023.3308088 ↗
98. SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence10.1126/sciadv.adi1480 ↗
99. A review of basic software for brain-inspired computing10.1007/s42514-022-00092-1 ↗
100. Simulation of networks of spiking neurons: A review of tools and strategies10.1007/s10827-007-0038-6 ↗