journal article·2023-10-20Open access

Online Machine Learning and Surrogate-Model-Based Optimization for Improved Production Processes Using a Cognitive Architecture

Alexander Hinterleitner, Richard Schulz, Lukas Hans, Aleksandr Subbotin, Nils Barthel, Noah Pütz, Martin Rosellen, Thomas Bartz-Beielstein, Christoph Geng, Phillip Priss

Applied Sciences· Volume 13· Issue 20

Abstract

Cyber-Physical Systems (CPS) play an essential role in today’s production processes, leveraging Artificial Intelligence (AI) to enhance operations such as optimization, anomaly detection, and predictive maintenance. This article reviews a cognitive architecture for Artificial Intelligence, which has been developed to establish a standard framework for integrating AI solutions into existing production processes. Given that machines in these processes continuously generate large streams of data, Online Machine Learning (OML) is identified as a crucial extension to the existing architecture. To substantiate this claim, real-world experiments using a slitting machine are conducted, to compare the performance of OML to traditional Batch Machine Learning. The assessment of contemporary OML algorithms using a real production system is a fundamental innovation in this research. The evaluations clearly indicate that OML adds significant value to CPS, and it is strongly recommended as an extension of related architectures, such as the cognitive architecture for AI discussed in this article. Additionally, surrogate-model-based optimization is employed, to determine the optimal hyperparameter settings for the corresponding OML algorithms, aiming to achieve peak performance in their respective tasks.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Data Stream Mining Techniques · Machine Learning and Data Classification · Metaheuristic Optimization Algorithms Research

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Institute for Data Science, Engineering and Analytics, TH Köln University of Applied Sciences, 51643 Gummersbach, Germany

Institute for Industrial Information Technology—inIT, OWL University of Applied Sciences and Arts, 32657 Lemgo, Germany

Funding

German Federal Ministry for Economic Affairs and Climate Action

References (21)

Show source-provided references
1. Adolphs, P., Bedenbender, H., Dirzus, D., Ehlich, M., Epple, U., Hankel, M., Heidel, R., Hoffmeister, M., Huhle, H., and Kaercher, B. (2015). Reference Architecture Model Industrie 4.0 (RAMI4.0), VDI. Tech. Rep.
2. Lin, S.W., Miller, B., Durand, J., Bleakley, G., Ghigani, A., Martin, R., Murphy, B., and Crawford, M. (2017). The Industrial Internet of Things Volume G1: Reference Architecture v1.80, Industrial Internet Consortium. Technical Report.
3. Cyber physical systems for predictive production systems10.1007/s11740-017-0729-4 ↗
4. SOAR: An Architecture for General Intelligence10.1016/0004-3702(87)90050-6 ↗
5. A Simple Theory of Complex Cognition10.1037/0003-066X.51.4.355 ↗
6. CAAI—A cognitive architecture to introduce artificial intelligence in cyber-physical production systems10.1007/s00170-020-06094-z ↗
7. Bartz-Beielstein, T., Zaefferer, M., and Mersmann, O. (2022). Hyperparameter Tuning for Machine and Deep Learning with R-A Practical Guide, Springer.10.1007/978-981-19-5170-1 ↗
8. River: Machine learning for streaming data in Python
9. Bifet, A., Gavalda, R., Holmes, G., and Pfahringer, B. (2018). Machine Learning for Data Streams with Practical Examples in MOA, MIT Press. Available online: https://moa.cms.waikato.ac.nz/book.10.7551/mitpress/10654.001.0001 ↗
10. Incremental on-line learning: A review and comparison of state of the art algorithms10.1016/j.neucom.2017.06.084 ↗
11. Domingos, P.M., and Hulten, G. (2000, January 20–23). Mining high-speed data streams. Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, USA.10.1145/347090.347107 ↗
12. Bifet, A., and Gavalda, R. (September, January 31). Adaptive learning from evolving data streams. Proceedings of the Advances in Intelligent Data Analysis VIII: 8th International Symposium on Intelligent Data Analysis, IDA 2009, Lyon, France.10.1007/978-3-642-03915-7_22 ↗
13. Bifet, A., and Gavalda, R. (2007, January 26–28). Learning from time-changing data with adaptive windowing. Proceedings of the 2007 SIAM International Conference on Data Mining, Minneapolis, MN, USA.10.1137/1.9781611972771.42 ↗
14. Scikit-learn: Machine Learning in Python
15. A Review and Taxonomy of Interactive Optimization Methods in Operations Research10.1145/2808234 ↗
16. Evolutionary Algorithms10.1002/widm.1124 ↗
17. Direct search methods: Then and now10.1016/S0377-0427(00)00423-4 ↗
18. Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., and Talwalkar, A. (2016). Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. arXiv.
19. Gramacy, R.B. (2020). Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, CRC Press.10.1201/9780367815493 ↗
20. Differential Evolution—A Simple and Efficient Heuristic for global Optimization over Continuous Spaces10.1023/A:1008202821328 ↗
21. Senn, S. (2021). Statistical Issues in Drug Development, John Wiley and Sons, Ltd.. Chapter 13.10.1002/9781119238614 ↗