Hessian with Mini-Batches for Electrical Demand Prediction
Source abstract
Abstract
The steepest descent method is frequently used for neural network tuning. Mini-batches are commonly used to get better tuning of the steepest descent in the neural network. Nevertheless, steepest descent with mini-batches could be delayed in reaching a minimum. The Hessian could be quicker than the steepest descent in reaching a minimum, and it is easier to achieve this goal by using the Hessian with mini-batches. In this article, the Hessian is combined with mini-batches for neural network tuning. The discussed algorithm is applied for electrical demand prediction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Neural Networks and Applications · Blind Source Separation Techniques · Energy Load and Power Forecasting
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Sección de Estudios de Posgrado e Investigación, ESIME Azcapotzalco, Instituto Politécnico Nacional, Av. de las Granjas no. 682, Col. Santa Catarina, Ciudad de México 02250, Mexico
References (38)
Show source-provided references
1. Facial Landmark Detection via Attention-Adaptive Deep Network10.1109/ACCESS.2019.2955156 ↗
2. An efficient DDoS detection based on SU-Genetic feature selection10.1007/s10586-018-2275-z ↗
3. Dinculeana, D., and Cheng, X. (2019). Vulnerabilities and Limitations of MQTT Protocol Used between IoT Devices. Appl. Sci., 9.10.3390/app9050848 ↗
4. Behavior Modeling and Individual Recognition of Sonar Transmitter for Secure Communication in UASNs10.1109/ACCESS.2019.2923059 ↗
5. Jia, B., Hao, L., Zhang, C., and Chen, D. (2018). A Dynamic Estimation of Service Level Based on Fuzzy Logic for Robustness in the Internet of Things. Sensors, 18.10.3390/s18072190 ↗
6. Behavior Data Provenance with Retention of Reference Relations10.1109/ACCESS.2018.2876879 ↗
7. A Training-integrity Privacy-preserving Federated Learning Scheme with Trusted Execution Environment10.1016/j.ins.2020.02.037 ↗
8. A High Quality Task Assignment Mechanism in Vehicle-Based Crowdsourcing Using Predictable Mobility Based on Markov10.1109/ACCESS.2018.2876908 ↗
9. An adaptive mini-batch stochastic gradient method for AUC maximization10.1016/j.neucom.2018.08.041 ↗
10. Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting10.1109/JSTSP.2015.2505682 ↗
11. Learnable Line Segment Descriptor for Visual SLAM10.1109/ACCESS.2019.2901584 ↗
12. Mini-batch optimized full waveform inversion with geological constrained gradient filtering10.1016/j.jappgeo.2018.02.011 ↗
13. Clustering Approach Based on Mini Batch Kmeans for Intrusion Detection System Over Big Data10.1109/ACCESS.2018.2810267 ↗
14. Clustering big IoT data by metaheuristic optimized mini-batch and parallel partition-based DGC in Hadoop10.1016/j.future.2018.03.006 ↗
15. Mini-batch algorithms with Barzilai-Borwein update step10.1016/j.neucom.2018.06.002 ↗
16. Random Barzilai-Borwein step size for mini-batch algorithms10.1016/j.engappai.2018.03.017 ↗
17. Hessian semi-supervised extreme learning machine10.1016/j.neucom.2016.05.039 ↗
18. HSAE: A Hessian regularized sparse auto-encoders10.1016/j.neucom.2015.07.119 ↗
19. Hessian regularization by patch alignment framework10.1016/j.neucom.2015.07.152 ↗
20. Optimization in Quaternion Dynamic Systems: Gradient, Hessian, and Learning Algorithms10.1109/TNNLS.2015.2440473 ↗
21. Leveraging Multiscale Hessian-Based Enhancement with a Novel Exudate Inpainting Technique for Retinal Vessel Segmentation10.1109/JBHI.2015.2440091 ↗
22. Hessian based approaches for 3D lung nodule segmentation10.1016/j.eswa.2016.05.024 ↗
23. Segmentation of optic disc and blood vessels in retinal images using wavelets, mathematical morphology and Hessian-based multi-scalefiltering10.1016/j.bspc.2017.03.014 ↗
24. Non-negative and local sparse coding based on l 2 -norm and Hessian regularization10.1016/j.ins.2019.02.024 ↗
25. Fast convex optimization via inertial dynamics with Hessian driven damping10.1016/j.jde.2016.08.020 ↗
26. On optimal simplicial 3D meshes for minimizing the Hessian-based errors10.1016/j.apnum.2016.07.007 ↗
27. On efficient Hessian computation using the edge pushing algorithm in Julia10.1080/10556788.2018.1480625 ↗
28. Xu, P., Roosta, F., and Mahoney, M.W. (2019). Newton-typemethods for non-convex optimization under inexact Hessian information. Math. Program.10.1007/s10107-019-01405-z ↗
29. Hessian-Regularized Multitask Dictionary Learning for Remote Sensing Image Recognition10.1109/LGRS.2018.2881834 ↗
30. Multiview Hessian regularized logistic regression for action recognition10.1016/j.sigpro.2014.08.002 ↗
31. Wrinkle Detection Using Hessian Line Tracking10.1109/ACCESS.2015.2455871 ↗
32. Multiview Hessian Semisupervised Sparse Feature Selection for Multimedia Analysis10.1109/TCSVT.2016.2576919 ↗
33. A Novel Generalized Intensity-Hue-Saturation (GIHS) Based Pan-Sharpening Method with Variational Hessian Transferring
34. An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems10.1007/s10107-018-1329-6 ↗
35. Hessian Regularized Distance Metric Learning for People Re-Identification10.1007/s11063-019-10000-4 ↗
36. Identification of important error fields in stellarators using the Hessian matrix method10.1088/1741-4326/ab3a7c ↗
37. Efficient symmetric Hessian propagation for direct optimal control10.1016/j.jprocont.2016.11.011 ↗
38. An Approach to Formulate the Hessian Matrix for Dynamic Control of Parallel Robots10.1109/TMECH.2019.2891297 ↗