Performance Enhancement of Adaptive Neural Networks Based on燣earning燫ate
Source abstract
Abstract
Deep learning is the process of determining parameters that reduce the cost function derived from the dataset. The optimization in neural networks at the time is known as the optimal parameters. To solve optimization, it initialize the parameters during the optimization process. There should be no variation in the cost function parameters at the global minimum. The momentum technique is a parameters optimization approach; however, it has difficulties stopping the parameter when the cost function value fulfills the global minimum (non-stop problem). Moreover, existing approaches use techniques; the learning rate is reduced during the iteration period. These techniques are monotonically reducing at a steady rate over time; our goal is to make the learning rate parameters. We present a method for determining the best parameters that adjust the learning rate in response to the cost function value. As a result, after the cost function has been optimized, the process of the rate Schedule is complete. This approach is shown to ensure convergence to the optimal parameters. This indicates that our strategy minimizes the cost function (or effective learning). The momentum approach is used in the proposed method. To solve the Momentum approach non-stop problem, we use the cost function of the parameter in our proposed method. As a result, this learning technique reduces the quantity of the parameter due to the impact of the cost function parameter. To verify that the learning works to test the strategy, we employed proof of convergence and empirical tests using current methods and the results are obtained using Python.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Neural Networks and Applications · Advanced Neural Network Applications · Stochastic Gradient Optimization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (34)
Show source-provided references
1. Learning representations by back-propagating errors10.1038/323533a0 ↗
2. Cassification of brain tumors using deep encoder along with regression techniques
3. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups10.1109/MSP.2012.2205597 ↗
4. Revisiting natural gradient for deep networks
5. An experimental approach to diagnose COVID-19 using optimized CNN
6. Improving neural networks by preventing co-adaptation of feature detectors
7. COVID-19 pandemic prediction and forecasting using machine learning classifiers10.32604/iasc.2022.021507 ↗
8. Iterative methods for linear and nonlinear equations
9. Iterative methods for optimization
10. Adaptive sub gradient methods for online learning and stochastic optimization
11. On the importance of initialization and momentum in deep learning
12. Adadelta: An adaptive learning rate method
13. ADAM: A method for stochastic optimization
14. On the convergence of ADAM and beyond
15. Optimization methods for large-scale machine learning10.1137/16M1080173 ↗
16. Nesterov · Introductory lectures on convex optimization: A basic course · 2003 · 87
17. Parameter optimization in convolutional neural networks using gradient descent
18. The step decay schedule: A near optimal, geometrically decaying learning rate procedure for least squares
19. Network in sequential form: Combine tree structure components into recurrent neural network10.1088/1757-899X/1017/1/012004 ↗
20. Comparison of the stochastic gradient descent based optimization techniques
21. Variants of RMSProp and adagrad with logarithmic regret bounds
22. Convergence guarantees for RMSProp and ADAM in nonconvex optimization and an empirical comparison to nesterov acceleration
23. A comparative analysis of gradient descent-based optimization algorithms on convolutional neural networks
24. Comparative analysis of stochastic optimization algorithms for image registration
25. Adaptive subgradient methods for online learning and stochastic optimization
26. Adam revisited: A weighted past gradients perspective Front
27. Keratoconus severity identification using unsupervised machine learning10.1371/journal.pone.0205998 ↗
28. Speed predication of wind using Artificial neural network
29. Tieleman · Lecture 6.5—RMSProp,coursera: Neural networks for machine learning · 2012
30. Recognition of human layered structure using gradient decent model
31. Recognition of human layered structure using gradient decent model
32. Deep learning10.1038/nature14539 ↗
33. An effective optimization method for machine learning based on ADAM10.3390/app10031073 ↗
34. Imagenet classification with deep convolutional neural networks