journal article·2024-10-01Open access

A Reinforcement Learning-Based Bi-Population Nutcracker Optimizer for Global Optimization

Yu Li, Yan Zhang

Biomimetics· Volume 9· Issue 10

Abstract

The nutcracker optimizer algorithm (NOA) is a metaheuristic method proposed in recent years. This algorithm simulates the behavior of nutcrackers searching and storing food in nature to solve the optimization problem. However, the traditional NOA struggles to balance global exploration and local exploitation effectively, making it prone to getting trapped in local optima when solving complex problems. To address these shortcomings, this study proposes a reinforcement learning-based bi-population nutcracker optimizer algorithm called RLNOA. In the RLNOA, a bi-population mechanism is introduced to better balance global and local optimization capabilities. At the beginning of each iteration, the raw population is divided into an exploration sub-population and an exploitation sub-population based on the fitness value of each individual. The exploration sub-population is composed of individuals with poor fitness values. An improved foraging strategy based on random opposition-based learning is designed as the update method for the exploration sub-population to enhance diversity. Meanwhile, Q-learning serves as an adaptive selector for exploitation strategies, enabling optimal adjustment of the exploitation sub-population’s behavior across various problems. The performance of the RLNOA is evaluated using the CEC-2014, CEC-2017, and CEC-2020 benchmark function sets, and it is compared against nine state-of-the-art metaheuristic algorithms. Experimental results demonstrate the superior performance of the proposed algorithm.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Metaheuristic Optimization Algorithms Research · Evolutionary Algorithms and Applications · Advanced Multi-Objective Optimization Algorithms

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Aeronautics and Astronautics, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518000, China

School of Electronics and Communication Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518000, China

References (49)

Show source-provided references
1. Hubálovsky, S., Hubálovská, M., and Matousová, I. (2024). A New Hybrid Particle Swarm Optimization-Teaching-Learning-Based Optimization for Solving Optimization Problems. Biomimetics, 9.10.3390/biomimetics9010008 ↗
2. Wang, R.T., Zhang, S.S., and Zou, G.Y. (2024). An Improved Multi-Strategy Crayfish Optimization Algorithm for Solving Numerical Optimization Problems. Biomimetics, 9.10.3390/biomimetics9060361 ↗
3. Population based metaheuristics in Spark: Towards a general framework using PSO as a case study10.1016/j.swevo.2024.101483 ↗
4. Reinforcement learning for enhanced online gradient-based parameter adaptation in metaheuristics10.1016/j.swevo.2023.101371 ↗
5. Large-scale power system multi-area economic dispatch considering valve point effects with comprehensive learning differential evolution10.1016/j.swevo.2024.101620 ↗
6. Cooperative constrained multi-objective dual-population evolutionary algorithm for optimal dispatching of wind-power integrated power system10.1016/j.swevo.2024.101525 ↗
7. An adaptive dual-population based evolutionary algorithm for industrial cut tobacco drying system10.1016/j.asoc.2023.110446 ↗
8. An improved levy chaotic particle swarm optimization algorithm for energy-efficient cluster routing scheme in industrial wireless sensor networks10.1016/j.eswa.2023.122780 ↗
9. A novel hybrid grey wolf optimizer algorithm for unmanned aerial vehicle (UAV) path planning10.1016/j.knosys.2020.105530 ↗
10. A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning10.1016/j.asoc.2020.106099 ↗
11. Parameter optimization for point clouds denoising based on no-reference quality assessment10.1016/j.measurement.2023.112592 ↗
12. An archive-based self-adaptive artificial electric field algorithm with orthogonal initialization for real-parameter optimization problems10.1016/j.asoc.2023.111109 ↗
13. Balancing exploration and exploitation in dynamic constrained multimodal multi-objective co-evolutionary algorithm10.1016/j.swevo.2024.101652 ↗
14. SFE: A Simple, Fast, and Efficient Feature Selection Algorithm for High-Dimensional Data10.1109/TEVC.2023.3238420 ↗
15. Improved dwarf mongoose optimization algorithm using novel nonlinear control and exploration strategies10.1016/j.eswa.2023.120904 ↗
16. Differential evolution with an adaptive penalty coefficient mechanism and a search history exploitation mechanism10.1016/j.eswa.2023.120530 ↗
17. A framework of global exploration and local exploitation using surrogates for expensive optimization10.1016/j.knosys.2023.111018 ↗
18. Multiple strategies based Grey Wolf Optimizer for feature selection in performance evaluation of open-ended funds10.1016/j.swevo.2024.101518 ↗
19. Snake Optimizer: A novel meta-heuristic optimization algorithm10.1016/j.knosys.2022.108320 ↗
20. White Shark Optimizer: A novel bio-inspired meta-heuristic algorithm for global optimization problems10.1016/j.knosys.2022.108457 ↗
21. WSOmark: An adaptive dual-purpose color image watermarking using white shark optimizer and Levenberg-Marquardt BPNN10.1016/j.eswa.2023.120137 ↗
22. Reptile Search Algorithm (RSA): A nature-inspired meta-heuristic optimizer10.1016/j.eswa.2021.116158 ↗
23. Crested Porcupine Optimizer: A new nature-inspired metaheuristic10.1016/j.knosys.2023.111257 ↗
24. Nutcracker optimizer: A novel nature-inspired metaheuristic algorithm for global optimization and engineering design problems10.1016/j.knosys.2022.110248 ↗
25. Quadratic interpolation and a new local search approach to improve particle swarm optimization: Solar photovoltaic parameter estimation10.1016/j.eswa.2023.121417 ↗
26. Memory, evolutionary operator, and local search based improved Grey Wolf Optimizer with linear population size reduction technique10.1016/j.knosys.2023.110297 ↗
27. An improved group teaching optimization algorithm based on local search and chaotic map for feature selection in high-dimensional data10.1016/j.eswa.2022.117493 ↗
28. A Modified Oppositional Chaotic Local Search Strategy Based Aquila Optimizer to Design an Effective Controller for Vehicle Cruise Control System10.1007/s42235-023-00336-y ↗
29. Fuzzy Community Detection Based on Elite Symbiotic Organisms Search and Node Neighborhood Information10.1109/TFUZZ.2021.3087235 ↗
30. An Elite Gene Guided Reproduction Operator for Many-Objective Optimization10.1109/TCYB.2019.2932451 ↗
31. Elite archives-driven particle swarm optimization for large scale numerical optimization and its engineering applications10.1016/j.swevo.2022.101212 ↗
32. An elite-guided hierarchical differential evolution algorithm10.1007/s10489-020-02091-7 ↗
33. Learnable Evolutionary Search Across Heterogeneous Problems via Kernelized Autoencoding10.1109/TEVC.2021.3056514 ↗
34. Solving Dynamic Multiobjective Problem via Autoencoding Evolutionary Search10.1109/TCYB.2020.3017017 ↗
35. Adaptive Distributed Differential Evolution10.1109/TCYB.2019.2944873 ↗
36. Learning-Aided Evolution for Optimization10.1109/TEVC.2022.3232776 ↗
37. Reinforcement Learning Methods for Computation Offloading: A Systematic Review10.1145/3603703 ↗
38. Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications10.1109/JAS.2023.123843 ↗
39. An inverse reinforcement learning framework with the Q-learning mechanism for the metaheuristic algorithm10.1016/j.knosys.2023.110368 ↗
40. A novel reinforcement learning-based reptile search algorithm for solving optimization problems10.1007/s00521-023-09023-9 ↗
41. Differential evolution based on reinforcement learning with fitness ranking for solving multimodal multiobjective problems10.1016/j.swevo.2019.06.010 ↗
42. Differential evolution with mixed mutation strategy based on deep reinforcement learning10.1016/j.asoc.2021.107678 ↗
43. An Improved Teaching-Learning-Based Optimization Algorithm with Reinforcement Learning Strategy for Solving Optimization Problems
44. A new Reinforcement Learning-Based Memetic Particle Swarm Optimizer10.1016/j.asoc.2016.01.006 ↗
45. Reinforcement learning-based comprehensive learning grey wolf optimizer for feature selection10.1016/j.asoc.2023.110959 ↗
46. Multi-objective constrained black-box optimization algorithm based on feasible region localization and performance-improvement exploration10.1016/j.asoc.2023.110874 ↗
47. Multiobjective Combinatorial Optimization Using a Single Deep Reinforcement Learning Model10.1109/TCYB.2023.3312476 ↗
48. Offline Reinforcement Learning with Behavior Value Regularization10.1109/TCYB.2024.3385910 ↗
49. Exponential distribution optimizer (EDO): A novel math-inspired algorithm for global optimization and engineering problems10.1007/s10462-023-10403-9 ↗