Population Dynamics in Genetic Programming for Dynamic Symbolic Regression
Source abstract
Abstract
This paper investigates the application of genetic programming (GP) for dynamic symbolic regression (SR), addressing the challenge of adapting machine learning models to evolving data in practical applications. Benchmark instances with changing underlying functions over time are defined to assess the performance of a genetic algorithm (GA) as a traditional evolutionary algorithm and an age-layered population structure (ALPS) as an open-ended evolutionary algorithm for dynamic symbolic regression. This study analyzes population dynamics by examining variable frequencies and impact changes over time in response to dynamic shifts in the training data. The results demonstrate the effectiveness of both the GA and ALPS in handling changing data, showcasing their ability to recover and evolve improved solutions after an initial drop in population quality following data changes. Population dynamics reveal that variable impacts respond rapidly to data changes, while variable frequencies shift gradually across generations, aligning with the indirect measure of fitness represented by variable impacts. Notably, the GA shows a strong dependence on mutation to avoid variables becoming permanently extinct, contrasting with the ALPS’s unexpected insensitivity to mutation rates owing to its reseeding mechanism for effective variable reintroduction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Evolutionary Algorithms and Applications · Metaheuristic Optimization Algorithms Research · Evolution and Genetic Dynamics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Heuristic and Evolutionary Algorithms Laboratory, University of Applied Sciences Upper Austria, 4232 Hagenberg, Austria
Institute for Symbolic Artificial Intelligence, Johannes Kepler University, 4040 Linz, Austria
Josef Ressel Center for Adaptive Optimization in Dynamic Environments, University of Applied Sciences Upper Austria, 4232 Hagenberg, Austria
Funding
Austrian Federal Ministry for Digital and Economic Affairs · National Foundation for Research, Technology and Development · Christian Doppler Research Association
References (44)
Show source-provided references
1. Random forests10.1023/A:1010933404324 ↗
2. Support-vector networks10.1007/BF00994018 ↗
3. Genetic programming as a means for programming computers by natural selection10.1007/BF00175355 ↗
4. Poli, R., Langdon, W.B., and McPhee, N.F. (2008). A Field Guide to Genetic Programming, Lulu Enterprises UK Ltd.. Available online: http://gpbib.cs.ucl.ac.uk/gp-html/poli08_fieldguide.html.
5. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
6. Holland, J.H. (1992). Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence, MIT Press.10.7551/mitpress/1090.001.0001 ↗
7. Macedo, J., Costa, E., and Marques, L. (April, January 30). Genetic programming algorithms for dynamic environments. Proceedings of the Applications of Evolutionary Computation: 19th European Conference, EvoApplications 2016, Porto, Portugal. Proceedings, Part II 19.
8. Yin, Z., Brabazon, A., O’Sullivan, C., and O’Neil, M. (2007, January 15–17). Genetic programming for dynamic environments. Proceedings of the International Multiconference on Computer Science and Information Technology, Wisła, Poland.
9. Prediction of dynamical systems by symbolic regression10.1103/PhysRevE.94.012214 ↗
10. O’Neill, M., Nicolau, M., and Brabazon, A. (2011, January 12–16). Dynamic environments can speed up evolution with genetic programming. Proceedings of the 13th Annual Conference Companion on Genetic and Evolutionary Computation, Dublin, Ireland.10.1145/2001858.2001965 ↗
11. LIII. On lines and planes of closest fit to systems of points in space10.1080/14786440109462720 ↗
12. Virgolin, M., and Pissis, S.P. (2022). Symbolic regression is np-hard. arXiv.
13. Nocedal, J., and Wright, S.J. (2006). Numerical Optimization, Springer. [2nd ed.].
14. Banzhaf, W., Nordin, P., Keller, R.E., and Francone, F.D. (1998). Genetic Programming: An Introduction: On the Automatic Evolution of Computer Programs and Its Applications, Morgan Kaufmann Publishers Inc.
15. Perkis, T. (1994, January 27–29). Stack-based genetic programming. Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, Orlando, FL, USA.
16. Grammatical evolution10.1109/4235.942529 ↗
17. McConaghy, T. (2011). Genetic Programming Theory and Practice IX, Springer.
18. Parameter identification for symbolic regression using nonlinear least squares10.1007/s10710-019-09371-3 ↗
19. Grammar-based genetic programming: A survey10.1007/s10710-010-9109-y ↗
20. Affenzeller, M., and Wagner, S. (2015). Adaptive and Natural Computing Algorithms, Proceedings of the International Conference, Coimbra, Portugal, 21–23 March 2005, Springer.
21. Hornby, G.S. (2006, January 8–12). ALPS: The age-layered population structure for reducing the problem of premature convergence. Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, WA, USA.10.1145/1143997.1144142 ↗
22. A fast and elitist multiobjective genetic algorithm: NSGA-II10.1109/4235.996017 ↗
23. SPEA2: Improving the strength Pareto evolutionary algorithm
24. Evolutionary dynamic optimization: A survey of the state of the art10.1016/j.swevo.2012.05.001 ↗
25. Benchmarking continuous dynamic optimization: Survey and generalized test suite10.1109/TCYB.2020.3011828 ↗
26. Li, C., Yang, S., Nguyen, T.T., Yu, E.L., Yao, X., Jin, Y., Beyer, H., and Suganthan, P.N. (2023, November 29). Benchmark Generator for CEC 2009 Competition on Dynamic Optimization. Technical Report. Available online: https://bura.brunel.ac.uk/bitstream/2438/5897/2/Fulltext.pdf.
27. Yang, S. (2003, January 8–12). Non-stationary problem optimization using the primal-dual genetic algorithm. Proceedings of the 2003 Congress on Evolutionary Computation, Canberra, ACT, Australia.
28. A survey of evolutionary continuous dynamic optimization over two decades—Part B10.1109/TEVC.2021.3060012 ↗
29. Tinós, R., Whitley, D., and Howe, A. (2014, January 12–16). Use of explicit memory in the dynamic traveling salesman problem. Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, Vancouver, BC, Canada.10.1145/2576768.2598247 ↗
30. Hansknecht, C., Joormann, I., and Stiller, S. (2021). Dynamic shortest paths methods for the time-dependent TSP. Algorithms, 14.10.3390/a14010021 ↗
31. Branke, J. (2012). Evolutionary Optimization in Dynamic Environments, Springer Science & Business Media.
32. Strąk, Ł., Skinderowicz, R., Boryczka, U., and Nowakowski, A. (2019). A self-adaptive discrete PSO algorithm with heterogeneous parameter values for dynamic TSP. Entropy, 21.10.20944/preprints201905.0187.v1 ↗
33. A novel evolutionary algorithm for dynamic constrained multiobjective optimization problems10.1109/TEVC.2019.2958075 ↗
34. Karder, J., Werth, B., Beham, A., Wagner, S., and Affenzeller, M. (2022, January 20–25). Analysis and Handling of Dynamic Problem Changes in Open-Ended Optimization. Proceedings of the International Conference on Computer Aided Systems Theory, Las Palmas de Gran Canaria, Spain.10.1007/978-3-031-25312-6_7 ↗
35. Brest, J., Zamuda, A., Boskovic, B., Maucec, M.S., and Zumer, V. (2009, January 18–21). Dynamic optimization using self-adaptive differential evolution. Proceedings of the 2009 IEEE Congress on Evolutionary Computation, Trondheim, Norway.10.1109/CEC.2009.4982976 ↗
36. On the elusivity of dynamic optimisation problems10.1016/j.swevo.2023.101289 ↗
37. Branke, J. (1999, January 6–9). Memory enhanced evolutionary algorithms for changing optimization problems. Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406), Washington, DC, USA.
38. Yu, E., and Suganthan, P.N. (2009, January 18–21). Evolutionary programming with ensemble of explicit memories for dynamic optimization. Proceedings of the 2009 IEEE Congress on Evolutionary Computation, Trondheim, Norway.10.1109/CEC.2009.4982978 ↗
39. Implicit memory-based technique in solving dynamic scheduling problems through response surface methodology—Part II: Experiments and analysis10.1108/IJICC-12-2013-0054 ↗
40. Morris, R. (2023, November 29). Genetic Algorithms with Implicit Memory. Available online: https://dora.dmu.ac.uk/server/api/core/bitstreams/e28b68b7-67ae-47a3-9b0c-7269ad77e2eb/content.
41. Winkler, S.M., Affenzeller, M., Kronberger, G., Kommenda, M., Burlacu, B., and Wagner, S. (2015). Genetic Programming Theory and Practice XII, Springer.
42. Greedy function approximation: A gradient boosting machine10.1214/aos/1013203451 ↗
43. A method for the solution of certain non-linear problems in least squares10.1090/qam/10666 ↗
44. COCO: A platform for comparing continuous optimizers in a black-box setting10.1080/10556788.2020.1808977 ↗