Hands-On Fundamentals of 1D Convolutional Neural Networks—A Tutorial for Beginner Users
Source abstract
Abstract
In recent years, deep learning (DL) has garnered significant attention for its successful applications across various domains in solving complex problems. This interest has spurred the development of numerous neural network architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and the more recently introduced Transformers. The choice of architecture depends on the data characteristics and the specific task at hand. In the 1D domain, one-dimensional CNNs (1D CNNs) are widely used, particularly for tasks involving the classification and recognition of 1D signals. While there are many applications of 1D CNNs in the literature, the technical details of their training are often not thoroughly explained, posing challenges for those developing new libraries in languages other than those supported by available open-source solutions. This paper offers a comprehensive, step-by-step tutorial on deriving feedforward and backpropagation equations for 1D CNNs, applicable to both regression and classification tasks. By linking neural networks with linear algebra, statistics, and optimization, this tutorial aims to clarify concepts related to 1D CNNs, making it a valuable resource for those interested in developing new libraries beyond existing ones.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Neural Networks and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Istituto di Fisica Applicata “Nello Carrara”, Consiglio Nazionale delle Ricerche, Via Madonna del Piano 10, 50019 Sesto Fiorentino, Italy
References (100+)
Show source-provided references
1. Deep learning10.1038/nature14539 ↗
2. Deep learning for visual understanding: A review10.1016/j.neucom.2015.09.116 ↗
3. Convolutional neural networks: An overview and application in rology10.1007/s13244-018-0639-9 ↗
4. Deep learning for time series classification: A review10.1007/s10618-019-00619-1 ↗
5. Object Detection with Deep Learning: A Review10.1109/TNNLS.2018.2876865 ↗
6. Emmert-Streib, F., Yang, Z., Feng, H., Tripathi, S., and Dehmer, M. (2020). An Introductory Review of Deep Learning for Prediction Models with Big Data. Front. Artif. Intell., 3.10.3389/frai.2020.00004 ↗
7. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions10.1186/s40537-021-00444-8 ↗
8. Deep learning in optical metrology: A review10.1038/s41377-022-00714-x ↗
9. A comprehensive review on ensemble deep learning: Opportunities and challenges
10. Machine Learning and Deep Learning: A Review of Methods and Applications
11. Deep learning: Systematic review, models, challenges, and research directions10.1007/s00521-023-08957-4 ↗
12. Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions10.1007/s42979-021-00815-1 ↗
13. Mankitsa, N., Maraslidi, G., Moysis, L., and Fragulis, G. (2024). A Review of Machine Learning and Deep Learning for Object Detection, Semantic Segmentation, and Human Action Recognition in Machine and Robotic Vision. Technologies, 12.10.3390/technologies12020015 ↗
14. Using transformers for multimodal emotion recognition: Taxonomies and state of the art review10.1016/j.engappai.2024.108339 ↗
15. Audiovisual speech recognition based on a deep convolutional neural network10.1016/j.dsm.2023.10.002 ↗
16. A review of deep learning techniques for speech processing10.1016/j.inffus.2023.101869 ↗
17. Emotion Speech Synthesis Method Based on Multi-Channel Time-Frequency Domain Generative Adversarial Networks (MC-TFD GANs) and Mixup10.1007/s13369-021-06090-9 ↗
18. Speech Recognition using Convolution Deep Neural Networks10.1088/1742-6596/1973/1/012166 ↗
19. Speech Expression Multimodal Emotion Recognition Based on Deep Belief Network10.1007/s10723-021-09564-0 ↗
20. Amberkar, A., Awasarmol, P., Deshmukh, G., and Dave, P. (2018, January 1–3). Speech Recognition using Recurrent Neural Networks. Proceedings of the 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT), Coimbatore, Tamil Nadu, India.10.1109/ICCTCT.2018.8551185 ↗
21. Graves, A., Mohamed, A.-r., and Hinton, G. (2013, January 26–31). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vancouver, BC, Canada.10.1109/ICASSP.2013.6638947 ↗
22. Sentiment classification with modified RoBERTa and recurrent neural networks10.1007/s11042-023-16833-5 ↗
23. Alemu, A.A., Melese, M.D., and Salau, A.O. (2023). Towards audio-based identification of Ethio-Semitic languages using recurrent neural network. Sci. Rep., 13.10.1038/s41598-023-46646-3 ↗
24. Tunstall, L., von Werra, L., and Wolf, T. (2022). Natural Language Processing with Transformers, O’Reilly Media Inc.
25. Wang, W., and Gang, J. (2018, January 6–8). Application of Convolutional Neural Network in Natural Language Processing. Proceedings of the 2018 International Conference on Information Systems and Computer Aided Education (ICISCAE), Changchun, China.10.1109/ICISCAE.2018.8666928 ↗
26. Chapter 9—Deep Neural Networks for Natural Language Processing10.1016/bs.host.2018.07.006 ↗
27. Deep learning for natural language processing: Advantages and challenges10.1093/nsr/nwx110 ↗
28. Goyal, P., Pandey, S., and Jain, K. (2018). Deep Learning for Natural Language Processing, Apress.10.1007/978-1-4842-3685-7 ↗
29. Application of Deep Belief Networks for Natural Language Understanding10.1109/TASLP.2014.2303296 ↗
30. Raza, K., Barh, D., Singh, D., and Ahmad, N. (2024). Protein structure prediction with recurrent neural network and convolutional neural network: A case study. Deep Learning Applications in Translational Bioinformatics, Academic Press. Chapter 13.
31. Zhang, Z.X., Xuan, J., Yao, C., Gao, Q., Wang, L., Jin, X., and Li, S. (2022). A deep learning approach for orphan gene identification in moso bamboo (Phyllostachys edulis) based on the CNN and Transformer model. BMC Bioinform., 23.10.1186/s12859-022-04702-1 ↗
32. Xing, C., Tian-Hao, L., Yan, Z., Chun-Chun, W., and Chi-Chi, Z. (2021). Deep-belief network for predicting potential miRNA-disease associations. Brief. Bioinform., 22.10.1093/bib/bbaa186 ↗
33. Application of deep learning methods in biological networks
34. Identifying antimicrobial peptides using word embedding with deep recurrent neural networks
35. Deep learning in bioinformatics: Introduction, application, and perspective in the big data era10.1016/j.ymeth.2019.04.008 ↗
36. Deep learning in bioinformatics
37. Convolutional neural network architectures for predicting DNA–protein binding10.1093/bioinformatics/btw255 ↗
38. Identification of meningioma tumor using recurrent neural networks
39. Han, Q., Qian, X., Xu, H., Wu, K., Meng, L., Qiu, Z., Weng, T., Zhou, B., and Gao, X. (2024). DM-CNN: Dynamic Multi-scale Convolutional Neural Network with uncertainty quantification for medical image classification. Comput. Biol. Med., 168.10.1016/j.compbiomed.2023.107758 ↗
40. Pu, Q., Xi, Z., Yin, S., Zhao, Z., and Zhao, L. (2014). Advantages of transformer and its application for medical image segmentation: A survey. BioMed. Eng. OnLine, 23.10.1186/s12938-024-01212-4 ↗
41. Zhang, Y., Wang, J., Gorriz, J.M., and Wang, S. (2023). Deep Learning and Vision Transformer for Medical Image Analysis. J. Imaging, 9.10.3390/jimaging9070147 ↗
42. Generative adversarial networks in EEG analysis An overview10.1186/s12984-023-01169-w ↗
43. Radiography image analysis using cat swarm optimized deep belief networks
44. A Review of Deep Learning on Medical Image Analysis10.1007/s11036-020-01672-7 ↗
45. Convolutional neural networks for medical image analysis State-of-the-art, comparisons, improvement and perspectives10.1016/j.neucom.2020.04.157 ↗
46. Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction10.1109/TMI.2018.2863670 ↗
47. Deep Learning Applications in Medical Image Analysis
48. On the use of convolutional neural networks for downscaling daily temperatures over southern South America in a climate change scenario10.1007/s00382-023-06912-6 ↗
49. Climate change and anthropogenic effects on the drying of a saline lake in an arid region (Namak Lake, Iran)10.1007/s00704-023-04622-x ↗
50. Xu, G., Wang, H., Ji, S., Ma, Y., and Feng, Y. (2024). MPformer: A transformer-based model for earthen ruins climate prediction. Tsinghua Science and Technology, TUP.10.26599/TST.2024.9010035 ↗
51. Hybrid optimized deep recurrent neural network for atmospheric and oceanic parameters prediction by feature fusion and data augmentation model10.1007/s10878-024-01159-1 ↗
52. Generative adversarial networks review in earthquake-related engineering fields10.1007/s10518-023-01645-7 ↗
53. Using an optimized soil and water assessment tool by deep belief networks to evaluate the impact of land use and climate change on water resources10.1002/cpe.6807 ↗
54. A transformer neural network for predicting near-surface temperature10.1002/met.2098 ↗
55. Interpretable deep learning model for building energy consumption prediction based on attention mechanism10.1016/j.enbuild.2021.111379 ↗
56. Chattopadhyay, A., Hassanzadeh, P., and Pasha, S. (2020). Predicting clustered weather patterns: A test case for applications of convolutional neural networks to spatio-temporal climate data. Sci. Rep., 10.10.1038/s41598-020-57897-9 ↗
57. Solar radiation prediction using recurrent neural network and artificial neural network: A case study with comparisons10.1016/j.renene.2020.04.042 ↗
58. Várkonyi-Kóczy, A.R. (2020). Deep Learning and Machine Learning in Hydrological Processes Climate Change and Earth Systems a Systematic Review. Engineering for Sustainable Future, Springer International Publishing.10.1007/978-3-030-36841-8 ↗
59. Deep learning for nano-photonic materials—The solution to everything!?10.1016/j.cossms.2023.101129 ↗
60. Kale, A.P., Wahul, R.M., Patange, A.D., Soman, R., and Ostachowicz, W. (2023). Development of Deep Belief Network for Tool Faults Recognition. Sensors, 23.10.3390/s23041872 ↗
61. Defect-aware transformer network for intelligent visual surface defect detection10.1016/j.aei.2023.101882 ↗
62. Rane, N. (2023). Transformers in Material Science: Roles, Challenges, and Future Scope. Chall. Future Scope, 1–21.10.2139/ssrn.4609920 ↗
63. Generative adversarial networks for labeled acceleration data augmentation for structural damage detection10.1007/s13349-022-00627-8 ↗
64. Ultrasonic guided wave based structural damage detection and localization using model assisted convolutional and recurrent neural networks10.1016/j.eswa.2020.114189 ↗
65. Recurrent convolutional neural network: A new framework for remaining useful life prediction of machinery10.1016/j.neucom.2019.10.064 ↗
66. Yang, J., Li, S., Wang, Z., Dong, H., Wang, J., and Tang, S. (2020). Using Deep Learning to Detect Defectsin Manufacturing: A Comprehensive Survey and Current Challenges. Materials, 13.10.3390/ma13245755 ↗
67. Ratchev, S. (2014). Application of Deep Belief Networks for Precision Mechanism Quality Inspection. Precision Assembly Technologies and Systems, Springer.10.1007/978-3-662-45586-9 ↗
68. Cordeiro, J.R., Raimundo, A., Postolache, O., and Sebastiao, P. (2021). Neural Architecture Search for 1D CNNs-Different Approaches Tests and Measurements. Sensors, 21.10.3390/s21237990 ↗
69. 1D convolutional neural networks and applications: A survey10.1016/j.ymssp.2020.107398 ↗
70. Deep Learning for Raman Spectroscopy: A Review10.3390/analytica3030020 ↗
71. State-of-the-Art Deep Learning Methods on Electrocardiogram Data: Systematic Review10.2196/38454 ↗
72. Overfitting One-Dimensional convolutional neural networks for Raman spectra identification10.1016/j.saa.2022.120961 ↗
73. De Silva, U., Koike-Akino, T., Ma, R., Yamashita, A., and Nakamizo, H. (2022, January 16–19). A Modular 1D-CNN Architecture for Real-time Digital Pre-distortion. Proceedings of the 2022 IEEE Topical Conference on RF/Microwave Power Amplifiers for Radio and Wireless Applications (PAWR), Las Vegas, NV, USA.10.1109/PAWR53092.2022.9719754 ↗
74. 1D-CNN based real-time fault detection system for power asset diagnostics
75. Real-time phonocardiogram anomaly detection by adaptive 1D Convolutional Neural Networks10.1016/j.neucom.2020.05.063 ↗
76. A Lightweight Channel and Time Attention Enhanced 1D CNN Model for Environmental Sound Classification10.1016/j.eswa.2024.123768 ↗
77. Allamy, S., and Koerich, A.L. (2021, January 5–7). 1D CNN Architectures for Music Genre Classification. Proceedings of the 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA.10.1109/SSCI50451.2021.9659979 ↗
78. Comparison and Analysis of SampleCNN Architectures for Audio Classification10.1109/JSTSP.2019.2909479 ↗
79. Hoshen, Y., Weiss, R.J., and Wilson, K.W. (2015, January 19–24). Speech acoustic modeling from raw multichannel waveforms. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178847 ↗
80. Jain, M., Bhardwaj, H., and Srivastav, A. (2024, January 14–15). Bayesian-Enhanced EEG Signal Analysis for Epileptic Seizure Recognition: A 1D-CNN LSTM Approach. Proceedings of the 11th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India.10.1109/ICRITO61523.2024.10522346 ↗
81. Xu, G., Ren, T., Chen, Y., and Che, W. (2020). A One-Dimensional CNN-LSTM Model for Epileptic Seizure Recognition Using EEG Signal Analysis. Front. Neurosci., 14.10.3389/fnins.2020.578126 ↗
82. A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface10.1088/1741-2552/ac4430 ↗
83. A deep learning algorithm based on 1D CNN-LSTM for automatic sleep staging10.3233/THC-212847 ↗
84. Dutta, M.D.A. (2024, January 14–16). ECG Disease Classification Using 1D CNN. Proceedings of the 2024 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI), Gwalior, India.10.1109/IATMSI60426.2024.10502954 ↗
85. Jasvitha, B.D., Kanagaraj, K., Murali, K., Singh, T., and Duraisamy, P. (2024, January 1–3). 1D CNN Framework on ECG Signals. Proceedings of the 2024 3rd International Conference for Innovation in Technology (INOCON), Bangalore, India.10.1109/INOCON60754.2024.10512284 ↗
86. Feature extraction and classification of heart sound using 1D convolutional neural networks10.1186/s13634-019-0651-3 ↗
87. Zhang, W., Yu, L., Ye, L., Zhuang, W., and Ma, F. (2018, January 22–24). ECG Signal Classification with Deep Learning for Heart Disease Identification. Proceedings of the 2018 International Conference on Big Data and Artificial Intelligence (BDAI), Beijing, China.10.1109/BDAI.2018.8546681 ↗
88. Mian, T.S. (2022). An Unsupervised Neural Network Feature Selection and 1D Convolution Neural Network Classification for Screening of Parkinsonism. Diagnostics, 12.10.3390/diagnostics12081796 ↗
89. Automatic COVID-19 disease diagnosis using 1D convolutional neural network and augmentation with human respiratory sound based on parameters: Cough, breath, and voice10.3934/publichealth.2021019 ↗
90. Using a Convolutional Neural Network to Predict Remission of Diabetes After Gastric Bypass Surgery10.2196/25612 ↗
91. ResNet50-1D-CNN: A new lightweight resNet50-One-dimensional convolution neural network transfer learning-based approach for improved intrusion detection in cyber-physical systems10.1016/j.ijcip.2024.100674 ↗
92. Qazi, E.U.H., Almorjan, A., and Zia, T. (2022). A One-Dimensional Convolutional Neural Network (1D-CNN) Based Deep Learning System for Network Intrusion Detection. Appl. Sci., 12.10.3390/app12167986 ↗
93. Meliboev, A., Alikhanov, J., and Kim, W. (2022). Performance Evaluation of Deep Learning Based Network Intrusion Detection System across Multiple Balanced and Imbalanced Datasets. Electronics, 11.10.3390/electronics11040515 ↗
94. Huo, D., Li, X., Li, L., Gao, Y., Li, X., and Yuan, J. (2022, January 4–6). The Application of 1D-CNN in Microsoft Malware Detection. Proceedings of the 2022 7th International Conference on Big Data Analytics (ICBDA), Guangzhou, China.10.1109/ICBDA55095.2022.9760349 ↗
95. Detecting Unauthorized Network Intrusion based on Network Traffic using Behavior Analysis Techniques
96. Deep Learning-Based Framework for the Detection of Cyberattack Using Feature Engineering10.1155/2021/6129210 ↗
97. Few-Shot Learning for Discovering Anomalous Behaviors in Edge Networks
98. Meliboev, A., Alikhanov, J., and Wooseong, K. (2020, January 19–21). 1D CNN based network intrusion detection with normalization on imbalanced data. Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan.
99. CNID: Research of Network Intrusion Detection Based on Convolutional Neural Network10.1155/2020/4705982 ↗
100. Wireless Network Intrusion Detection Based on Improved Convolutional Neural Network10.1109/ACCESS.2019.2917299 ↗