journal article·2023-07-08Open access

RETRACTED: Continual Learning Approach for Continuous Data Stream Analysis in Dynamic Environments

K. Prasanna, Mudassir Khan, Saeed Alshahrani, Ajmeera Kiran, P. Phanindra Kumar Reddy, Mofadal Alymani, J. Chinna Babu

Applied Sciences· Volume 13· Issue 14
This record is marked as retracted by the source. Review the publisher’s notice before using its findings.

Abstract

Continuous data stream analysis primarily focuses on the unanticipated changes in the transmission of data distribution over time. Conceptual change is defined as the signal distribution changes over the transmission of continuous data streams. A drift detection scenario is set forth to develop methods and strategies for detecting, interpreting, and adapting to conceptual changes over data streams. Machine learning approaches can produce poor learning outcomes in the conceptual change environment if the sudden change is not addressed. Furthermore, due to developments in concept drift, learning methodologies have been significantly systematic in recent years. The research introduces a novel approach using the fully connected committee machine (FCM) and different activation functions to address conceptual changes in continuous data streams. It explores scenarios of continual learning and investigates the effects of over-learning and weight decay on concept drift. The findings demonstrate the effectiveness of the FCM framework and provide insights into improving machine learning approaches for continuous data stream analysis. We used a layered neural network framework to experiment with different scenarios of continual learning on continuous data streams in the presence of change in the data distribution using a fully connected committee machine (FCM). In this research, we conduct experiments in various scenarios using a layered neural network framework, specifically the fully connected committee machine (FCM), to address conceptual changes in continuous data streams for continual learning under a conceptual change in the data distribution. Sigmoidal and ReLU (Rectified Linear Unit) activation functions are considered for learning regression in layered neural networks. When the layered framework is trained from the input data stream, the regression scheme changes consciously in all scenarios. A fully connected committee machine (FCM) is trained to perform the tasks described in continual learning with M hidden units on dynamically generated inputs. In this method, we run Monte Carlo simulations with the same number of units on both sides, K and M, to define the advancement of intersections between several hidden units and the calculation of generalization error. This is applied to over-learnability as a method of over-forgetting, integrating weight decay, and examining its effects when a concept drift is presented.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Data Stream Mining Techniques · Machine Learning and Data Classification · Anomaly Detection Techniques and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of CSE(AI&ML), G. Narayanamma Institute of Technology and Science, (Autonomous), Hyderabad 500104, Telangana, India

Department of Computer Science, College of Science & Arts, King Khalid University, P.O. Box 960, Abha 61421, Saudi Arabia

Department of Computer Science, College of Computing and Information Technology, Shaqra University, P.O. Box 30, Shaqra 11961, Saudi Arabia

Department of Computer Science and Engineering, MLR Institute of Technology, Hyderabad 500043, Telangana, India

Department of Computer Science and Engineering, Annamacharya Institute of Technology and Sciences (Autonomous), Rajampet 516126, Andhra Pradesh, India

Department of Computer Engineering, College of Computing and Information Technology, Shaqra University, P.O. Box 30, Shaqra 11961, Saudi Arabia

Department of Electronics and Communications Engineering, Annamacharya Institute of Technology and Sciences (Autonomous), Rajampet 516126, Andhra Pradesh, India

References (40)

Show source-provided references
1. Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, Curran Associates, Inc.
2. Palm, R.B. (2012). Prediction as a Candidate for Learning Deep Hierarchical Models of Data. [Master’s Thesis, Technical University of Denmark].
3. Learning under concept drift: A review
4. Yoon, J., Yang, E., Lee, J., and Hwang, S.J. (May, January 30). Lifelong learning with dynamically expandable networks. Proceedings of the ICLR, Vancouver, BC, Canada.
5. A survey on concept drift adaptation10.1145/2523813 ↗
6. Transient dynamics of on-line learning in two-layered neural networks10.1088/0305-4470/29/16/005 ↗
7. Straat, M., and Biehl, M. (2019, January 3–5). Online learning dynamics of ReLU neural networks using statistical physics techniques. Proceedings of the CoRR, Timişoara, Romania.
8. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer. [2nd ed.].10.1007/978-0-387-84858-7 ↗
9. Statistical Mechanics of On-Line Learning under Concept Drift10.3390/e20100775 ↗
10. Supervised Learning in the Presence of Concept Drift: A modeling framework10.1007/s00521-021-06035-1 ↗
11. Aspect-Oriented Concept Drift Detection in High Dimensional Data Streams10.30534/ijatcse/2020/111922020 ↗
12. A concept drift-tolerant case-base editing technique10.1016/j.artint.2015.09.009 ↗
13. Incremental learning from noisy data10.1023/A:1022810614389 ↗
14. Losing, V., Hammer, B., and Wersing, H. (2016, January 12–15). Knn classifier with self-adjusting memory for heterogeneous concept drift. Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain.10.1109/ICDM.2016.0040 ↗
15. Data Mining Models of High Dimensional Data Streams, and Contemporary Concept Drift Detection Methods: A Comprehensive Review10.14419/ijet.v7i3.6.14959 ↗
16. Liu, A., Song, Y., Zhang, G., and Lu, J. (2017, January 19–25). Regional concept drift detection and density synchronized drift adaptation. Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17), Melbourne, Australia.10.24963/ijcai.2017/317 ↗
17. Accumulating regional density dissimilarity for concept drift detection in data streams10.1016/j.patcog.2017.11.009 ↗
18. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., and Grabska-Barwinska, A. (2016). Overcoming catastrophic forgetting in neural networks. arXiv.10.1073/pnas.1611835114 ↗
19. Learning by on-line gradient descent10.1088/0305-4470/28/3/018 ↗
20. Nuwan, G., Murilo, G.H., Albert, B., and Bernhard, P. (2022). Adaptive Online Domain Incremental Continual Learning. Artificial Neural Networks and Machine Learning—ICANN 2022, Proceedings of the 31st International Conference on Artificial Neural Networks, Bristol, UK, 6–9 September 2022, Springer.
21. Kingson, M., Antonio, D., and Hartmut, N. (2022). Need is All You Need: Homeostatic Neural Networks Adapt to Concept Shift. arXiv.
22. Marconato, E., Bontempo, G., Teso, S., Ficarra, E., Calderara, S., and Passerini, A. (2022, January 23–27). Catastrophic Forgetting in Continual Concept Bottleneck Models. Proceedings of the ICIAP International Workshops, Lecce, Italy.10.1007/978-3-031-13324-4_46 ↗
23. Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions
24. Zhai, R., Schroedl, S., Galstyan, A., Kumar, A., Steeg, G.V., and Natarajan, P. (2023, January 1–5). Online Continual Learning for Progressive Distribution Shift (OCL-PDS): A Practitioner’s Perspective. Proceedings of the ICLR 2023, Kigali, Rwanda.
25. Incremental on-line learning: A review and comparison of state of the art algorithms10.1016/j.neucom.2017.06.084 ↗
26. Continual learning for blind image quality assessment
27. Saad, D. (1999). On-Line Learning in Neural Networks, Cambridge University Press.10.1017/CBO9780511569920 ↗
28. Togbe, M.U., Chabchoub, Y., Boly, A., Barry, M., Chiky, R., and Bahri, M. (2021). Anomalies Detection Using Isolation in Concept-Drifting Data Streams. Computers, 10.10.3390/computers10010013 ↗
29. Concept Drift Adaptation Techniques in Distributed Environment for Real-World Data Streams10.3390/smartcities4010021 ↗
30. AlQabbany, A.O., and Azmi, A.M. (2021). Measuring the Effectiveness of Adaptive Random Forest for Handling Concept Drift in Big Data Streams. Entropy, 23.10.3390/e23070859 ↗
31. Hu, S., Fong, S., Yang, L., Yang, S.-H., Dey, N., Millham, R.C., and Fiaidhi, J. (2021). Fast and Accurate Terrain Image Classification for ASTER Remote Sensing by Data Stream Mining and Evolutionary-EAC Instance-Learning-Based Algorithm. Remote Sens., 13.10.3390/rs13061123 ↗
32. Marcus, G. (2018). Deep learning: A critical appraisal. arXiv.
33. On-Line Learning of a Time-Dependent Rule10.1209/0295-5075/20/8/012 ↗
34. Straat, M. (2018). Online Learning in Neural Networks with ReLU Activations, University of Groningen.
35. Dynamics and generalization ability of LVQ algorithms
36. Engel, A., and van den Broeck, C. (2001). The Statistical Mechanics of Learning, Cambridge University Press.10.1017/CBO9781139164542 ↗
37. The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain10.1037/h0042519 ↗
38. Online learning in soft committee machines
39. Exact solution for on-line learning in multilayer neural10.1103/PhysRevLett.74.4337 ↗
40. Learning drifting concepts with neural networks10.1088/0305-4470/26/11/014 ↗