journal article·2025-11-19Open access

FBCA: Flexible Besiege and Conquer Algorithm for Multi-Layer Perceptron Optimization Problems

Shuxin Guo, Chenxu Guo, Jianhua Jiang

Biomimetics· Volume 10· Issue 11

Abstract

A Multi-Layer Perceptron (MLP), as the basic structure of neural networks, is an important component of various deep learning models such as CNNs, RNNs, and Transformers. Nevertheless, MLP training faces significant challenges, with a large number of saddle points and local minima in its non-convex optimization space, which can easily lead to gradient vanishing and premature convergence. Compared with traditional heuristic algorithms relying on a population-based parallel search, such as GA, GWO, DE, etc., the Besiege and Conquer Algorithm (BCA) employs a one-spot update strategy that provides a certain level of global optimization capability but exhibits clear limitations in search flexibility. Specifically, it lacks fast detection, fast adaptation, and fast convergence. First, the fixed sinusoidal amplitude limits the accuracy of fast detection in complex regions. Second, the combination of a random location and fixed perturbation range limits the fast adaptation of global convergence. Finally, the lack of a hierarchical adjustment under a single parameter (BCB) hinders the dynamic transition from exploration to exploitation, resulting in slow convergence. To address these limitations, this paper proposes a Flexible Besiege and Conquer Algorithm (FBCA), which improves search flexibility and convergence capability through three new mechanisms: (1) the sine-guided soft asymmetric Gaussian perturbation mechanism enhances local micro-exploration, thereby achieving a fast detection response near the global optimum; (2) the exponentially modulated spiral perturbation mechanism adopts an exponential spiral factor for fast adaptation of global convergence; and (3) the nonlinear cognitive coefficient-driven velocity update mechanism improves the convergence performance, realizing a more balanced exploration–exploitation process. In the IEEE CEC 2017 benchmark function test, FBCA ranked first in the comprehensive comparison with 12 state-of-the-art algorithms, with a win rate of 62% over BCA in 100-dimensional problems. It also achieved the best performance in six MLP optimization problems, showing excellent convergence accuracy and robustness, proving its excellent global optimization ability in complex nonlinear MLP optimization training. It demonstrates its application value and potential in optimizing neural networks and deep learning models.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Machine Learning and ELM · Advanced Technologies in Various Fields · Advanced Neural Network Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Center for Artificial Intelligence, Jilin University of Finance and Economics, Changchun 130117, China

Jilin Province Key Laboratory of Fintech, Jilin University of Finance and Economics, Changchun 130117, China

Funding

Scientific Research Project of Jilin Provincial Department of Education

References (63)

Show source-provided references
1. Feature importance measure of a multilayer perceptron based on the presingle-connection layer10.1007/s10115-023-01959-7 ↗
2. Deep neural networks in the cloud: Review, applications, challenges and research directions10.1016/j.neucom.2023.126327 ↗
3. A review of convolutional neural networks in computer vision10.1007/s10462-024-10721-6 ↗
4. Sajun, A.R., Zualkernan, I., and Sankalpa, D. (2024). A historical survey of advances in transformer architectures. Appl. Sci., 14.10.3390/app14104316 ↗
5. A comprehensive review of yolo architectures in computer vision: From yolov1 to yolov8 and yolo-nas10.3390/make5040083 ↗
6. Exploring DeepSeek: A Survey on Advances, Applications, Challenges and Future Directions10.1109/JAS.2025.125498 ↗
7. On nonconvex optimization for machine learning: Gradients, stochasticity, and saddle points10.1145/3418526 ↗
8. Distributed stochastic gradient descent: Nonconvexity, nonsmoothness, and convergence to local minima
9. MetaPerceptron: A standardized framework for metaheuristic-driven multi-layer perceptron optimization10.1016/j.csi.2025.103977 ↗
10. How effective is the Grey Wolf optimizer in training multi-layer perceptrons10.1007/s10489-014-0645-7 ↗
11. Comparison of the efficacy of particle swarm optimization and stochastic gradient descent algorithms on multi-layer perceptron model to estimate longitudinal dispersion coefficients in natural streams
12. Hameed, F., and Alkhzaimi, H. (2025). Hybrid genetic algorithm and deep learning techniques for advanced side-channel attacks. Sci. Rep., 15.10.1038/s41598-025-06375-1 ↗
13. Gurgenc, E., Altay, O., and Altay, E.V. (2024). AOSMA-MLP: A novel method for hybrid metaheuristics artificial neural networks and a new approach for prediction of geothermal reservoir temperature. Appl. Sci., 14.10.3390/app14083534 ↗
14. Lu, Y., and Zhao, H. (2025). Research on Slope Stability Prediction Based on MC-BKA-MLP Mixed Model. Appl. Sci., 15.10.3390/app15063158 ↗
15. Enhancing multilayer perceptron neural network using archive-based harris hawks optimizer to predict gold prices10.1016/j.jksuci.2023.101557 ↗
16. Implementation of hybrid particle swarm optimization-differential evolution algorithms coupled with multi-layer perceptron for suspended sediment load estimation10.1016/j.catena.2020.105024 ↗
17. Predicting evaporation with optimized artificial neural network using multi-objective salp swarm algorithm10.1007/s11356-021-16301-3 ↗
18. Yang, Z., Jiang, Y., and Yeh, W.C. (2024). Self-learning salp swarm algorithm for global optimization and its application in multi-layer perceptron model training. Sci. Rep., 14.10.1038/s41598-024-77440-4 ↗
19. A machine learning proxy based multi-objective optimization method for low-carbon hydrogen production10.1016/j.jclepro.2024.141377 ↗
20. Jiang, J., Wu, J., Luo, J., Yang, X., and Huang, Z. (2024). MOBCA: Multi-Objective Besiege and Conquer Algorithm. Biomimetics, 9.10.3390/biomimetics9060316 ↗
21. Jiang, J., Meng, X., Wu, J., Tian, J., Xu, G., and Li, W. (2025). BCA: Besiege and Conquer Algorithm. Symmetry, 17.10.3390/sym17020217 ↗
23. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces10.1023/A:1008202821328 ↗
24. Particle swarm optimization10.1109/ICNN.1995.488968 ↗
25. Artificial Neural Networks: An Overview10.58496/MJCSC/2023/015 ↗
26. Stability for a random evolution equation with Gaussian perturbation10.1016/S0022-247X(02)00163-4 ↗
27. Omar, M.B., Bingi, K., Prusty, B.R., and Ibrahim, R. (2022). Recent advances and applications of spiral dynamics optimization algorithm: A review. Fractal Fract., 6.10.3390/fractalfract6010027 ↗
28. The Whale Optimization Algorithm10.1016/j.advengsoft.2016.01.008 ↗
29. Yang, Q., Hua, L., Gao, X., Xu, D., Lu, Z., Jeon, S.W., and Zhang, J. (2022). Stochastic cognitive dominance leading particle swarm optimization for multimodal problems. Mathematics, 10.10.3390/math10050761 ↗
30. Yang, Q., Jing, Y., Gao, X., Xu, D., Lu, Z., Jeon, S.W., and Zhang, J. (2022). Predominant cognitive learning particle swarm optimization for global numerical optimization. Mathematics, 10.10.3390/math10101620 ↗
31. SCA: A sine cosine algorithm for solving optimization problems10.1016/j.knosys.2015.12.022 ↗
32. Reptile Search Algorithm (RSA): A nature-inspired meta-heuristic optimizer10.1016/j.eswa.2021.116158 ↗
33. Dung beetle optimizer: A new meta-heuristic algorithm for global optimization10.1007/s11227-022-04959-6 ↗
34. Black-winged kite algorithm: A nature-inspired meta-heuristic for solving benchmark functions and engineering problems10.1007/s10462-024-10723-4 ↗
35. Harris hawks optimization: Algorithm and applications10.1016/j.future.2019.02.028 ↗
36. The Hiking Optimization Algorithm: A novel human-based metaheuristic approach10.1016/j.knosys.2024.111880 ↗
37. Coati Optimization Algorithm: A new bio-inspired metaheuristic algorithm for solving optimization problems10.1016/j.knosys.2022.110011 ↗
38. Crested Porcupine Optimizer: A new nature-inspired metaheuristic10.1016/j.knosys.2023.111257 ↗
39. Lian, J., Hui, G., Ma, L., Zhu, T., Wu, X., Heidari, A.A., Chen, Y., and Chen, H. (2024). Parrot optimizer: Algorithm and applications to medical problems. Comput. Biol. Med., 172.10.1016/j.compbiomed.2024.108064 ↗
40. Slime mould algorithm: A new method for stochastic optimization10.1016/j.future.2020.03.055 ↗
41. The moss growth optimization (MGO): Concepts and performance
42. Cai, X., and Zhang, C. (2025). An Innovative Differentiated Creative Search Based on Collaborative Development and Population Evaluation. Biomimetics, 10.10.3390/biomimetics10050260 ↗
43. Potter, K., Hagen, H., Kerren, A., and Dannenmann, P. (2006, January 11). Methods for presenting statistical information: The box plot. Proceedings of the VLUDS, Seoul, Republic of Korea. Available online: https://api.semanticscholar.org/CorpusID:1344717.
44. Brest, J., Zumer, V., and Maucec, M.S. (2006, January 16–21). Self-adaptive differential evolution algorithm in constrained real-parameter optimization. Proceedings of the 2006 IEEE International Conference on Evolutionary Computation, Vancouver, BC, Canada.
45. Tanabe, R., and Fukunaga, A.S. (2014, January 6–11). Improving the search performance of SHADE using linear population size reduction. Proceedings of the 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China.10.1109/CEC.2014.6900380 ↗
46. Awad, N.H., Ali, M.Z., and Suganthan, P.N. (2017, January 5–8). Ensemble sinusoidal differential covariance matrix adaptation with Euclidean neighborhood for solving CEC2017 benchmark problems. Proceedings of the 2017 IEEE Congress on Evolutionary Computation (CEC), San Sebastian, Spain.10.1109/CEC.2017.7969336 ↗
47. No free lunch theorems for optimization10.1109/4235.585893 ↗
48. Guided learning strategy: A novel update mechanism for metaheuristic algorithms design and improvement10.1016/j.knosys.2024.111402 ↗
49. Dehghani, M., and Trojovskỳ, P. (2023). Osprey optimization algorithm: A new bio-inspired metaheuristic algorithm for solving engineering optimization problems. Front. Mech. Eng., 8.10.3389/fmech.2022.1126450 ↗
50. AGWO: Advanced GWO in multi-layer perception optimization10.1016/j.eswa.2021.114676 ↗
51. Knowledge extraction from radial basis function networks and multilayer perceptrons10.1109/IJCNN.1999.833464 ↗
52. Daniel, W.B., and Yeung, E. (2019). A constructive approach for one-shot training of neural networks using hypercube-based topological coverings. arXiv.
53. Fisher, R.A. (2025, October 28). Iris. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/53/iris.
54. A comparative analysis of meta-heuristic optimization algorithms for feature selection on ML-based classification of heart-related diseases10.1007/s11227-023-05132-3 ↗
55. Janosi, A., Steinbrunn, W., Pfisterer, M., and Detrano, R. (2025, October 28). Heart Disease. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/45/heart+disease.
56. Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv.
57. A stochastic approximation method10.1214/aoms/1177729586 ↗
58. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. CoRR, abs/1412.6980, Available online: https://api.semanticscholar.org/CorpusID:6628106.
59. A Dynamics Theory of RMSProp-Based Implicit Regularization in Deep Low-Rank Matrix Factorization10.1109/TNNLS.2025.3574683 ↗
60. Adagrad stepsizes: Sharp convergence over nonconvex landscapes
61. An adaptation of hybrid binary optimization algorithms for medical image feature selection in neural network for classification of breast cancer10.1016/j.neucom.2024.129018 ↗
62. Research on Financial Time Series Prediction Model Based on Multifractal Trend Cross Correlation Removal and Deep Learning10.1016/j.procs.2025.04.192 ↗
63. Advancing multimodal emotion recognition in big data through prompt engineering and deep adaptive learning10.1186/s40537-025-01264-w ↗