Deterministic Convergence Analysis for GRU Networks via Smoothing Regularization
Source abstract
Abstract
In this study, we present a deterministic convergence analysis of Gated Recurrent Unit (GRU) networks enhanced by a smoothing regularization technique. While GRU architectures effectively mitigate gradient vanishing/explodin... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Antenna Design and Optimization · Topology Optimization in Engineering · Advanced MEMS and NEMS Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (34)
Show source-provided references
1. A neural network architecture combining gated recurrent unit (GRU) and support vector machine (SVM) for intrusion detection in network traffic data
2. A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints10.1109/72.883412 ↗
3. Untersuchungen zu dynamischen neuronalen Netzen
4. Learning long-term dependencies with gradient descent is difficult10.1109/72.279181 ↗
5. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
6. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:14061078. 2014.10.3115/v1/D14-1179 ↗
7. Goodfellow · Deep learning · 2016
8. Performance evaluation of deep neural networks applied to speech recognition: rNN, LSTM and GRU10.2478/jaiscr-2019-0006 ↗
9. An efficient long short-term memory and gated recurrent unit based smart vessel trajectory prediction using automatic identification system data
10. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions10.1186/s40537-021-00444-8 ↗
11. A systematic review on overfitting control in shallow and deep neural networks10.1007/s10462-021-09975-1 ↗
12. Two strategies to avoid overfitting in feedforward networks10.1016/S0893-6080(96)00086-X ↗
13. Li H, Kadav A, Durdanovic I, Samet H, Graf HP. Pruning filters for efficient convnets. arXiv:160808710. 2016.
14. Regularization theory and neural networks architectures10.1162/neco.1995.7.2.219 ↗
15. Quasdane · Knowl Based Syst · 2024 · 284 · 11132710.1016/j.knosys.2023.111327 ↗
16. Van Laarhoven T. L2 regularization versus batch and weight normalization. arXiv:170605350. 2017.
17. Avoiding overfitting: a survey on regularization methods for convolutional neural networks10.1145/3510413 ↗
18. Dropout: a simple way to prevent neural networks from overfitting
19. R-drop: regularized dropout for neural networks
20. Neural machine translation models with attention-based dropout layer
21. Park · J Royal Statist Soc Ser B: Statist Method · 2007 · 69 · 65910.1111/j.1467-9868.2007.00607.x ↗
22. The impact of regularization on high-dimensional logistic regression
23. Novel L1 regularized extreme learning machine for soft-sensing of an industrial process10.1109/TII.2021.3065377 ↗
24. Boundedness and convergence of batch back-propagation algorithm with penalty for feedforward neural networks10.1016/j.neucom.2012.02.029 ↗
25. Computational properties and convergence analysis of BPNN for cyclic and almost cyclic learning with penalty10.1016/j.neunet.2012.04.013 ↗
26. Deterministic convergence analysis via smoothing group Lasso regularization and adaptive momentum for Sigma-Pi-Sigma neural network10.1016/j.ins.2020.12.014 ↗
27. Yu · Pattern Recognit · 2023 · 141 · 10965610.1016/j.patcog.2023.109656 ↗
28. Convergence analysis of BP neural networks via sparse response regularization10.1016/j.asoc.2017.07.059 ↗
29. Convergence analysis of online gradient method for high-order neural networks and their sparse optimization10.1109/TNNLS.2023.3319989 ↗
30. Kang · Knowl Based Syst · 2022 · 257 · 10985810.1016/j.knosys.2022.109858 ↗
31. Convergence of a gradient-based learning algorithm with penalty for ridge polynomial neural networks10.1109/ACCESS.2020.3048235 ↗
32. LSTM and GRU neural network performance comparison study: taking yelp review dataset as an example
33. Ma · Neural Netw · 2019 · 119 · 28610.1016/j.neunet.2019.08.015 ↗
34. Campi · SIAM J Cont Optimiza · 2013 · 51 · 353210.1137/110856204 ↗