journal article·2025-12-09Open access

SUN: Stochastic UNsupervised Learning for Data Noise and Uncertainty Reduction

Nicholas Christakis, Dimitris Drikakis

Applied Sciences· Volume 15· Issue 24

Abstract

Unsupervised learning methods significantly benefit various practical applications by effectively identifying intrinsic patterns within unlabelled data. However, inherent data noise and uncertainties often compromise model reliability, result interpretability, and the overall effectiveness of unsupervised learning strategies, particularly in complex fields such as biomedical, engineering, and physics research. To address these critical challenges, this study proposes SUN (Stochastic UNsupervised learning), a novel approach that integrates probabilistic unsupervised techniques—specifically Gaussian Mixture Models—into the RUN-ICON unsupervised learning algorithm to achieve optimal clustering, systematically reduce data noise, and quantify inherent uncertainties. The SUN methodology strategically leverages probabilistic modelling for robust classification and detection tasks, explicitly targeting particle dispersion scenarios related to environmental pollution and airborne viral transmission, with implications for minimising public health risks. By combining advanced uncertainty quantification methods and innovative unsupervised denoising techniques, the proposed study aims to provide more reliable and interpretable insights than conventional methods while alleviating issues such as computational complexity and reproducibility that limit traditional mathematical modelling. This research contributes to enhanced trustworthiness and interpretability of unsupervised learning systems, offering a robust methodological framework for handling significant uncertainty in complex real-world data environments.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Air Quality Monitoring and Forecasting · Gaussian Processes and Bayesian Inference · COVID-19 diagnosis using AI

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Institute for Advanced Modelling and Simulation, University of Nicosia, CY 2417 Nicosia, Cyprus

Funding

European Union’s Horizon Europe Research and Innovation Actions programme

References (100+)

Show source-provided references
1. Sammut, C., and Webb, G.I. (2010). K-Means Clustering. Encyclopedia of Machine Learning, Springer.10.1007/978-0-387-30164-8 ↗
2. Berry, M.W., Mohamed, A., and Yap, B.W. (2020). A Systematic Review on Supervised and Unsupervised Machine Learning Algorithms for Data Science. Supervised and Unsupervised Learning for Data Science, Springer International Publishing.10.1007/978-3-030-22475-2 ↗
3. Unsupervised Learning Methods for Molecular Simulation Data10.1021/acs.chemrev.0c01195 ↗
4. Neuer, M.J. (2025). Unsupervised Learning. Machine Learning for Engineers: Introduction to Physics-Informed, Explainable Learning Methods for AI in Engineering Applications, Springer.10.1007/978-3-662-69995-9 ↗
5. Learning from positive and unlabeled data: A survey10.1007/s10994-020-05877-5 ↗
6. A novel semi-supervised data-driven method for chiller fault diagnosis with unlabeled data10.1016/j.apenergy.2021.116459 ↗
7. A Survey on Active Deep Learning: From Model Driven to Data Driven
8. Exploring the Frontiers of Unsupervised Learning Techniques for Diagnosis of Cardiovascular Disorder: A Systematic Review10.1109/ACCESS.2024.3468163 ↗
9. A review of unsupervised learning in astronomy10.1016/j.ascom.2024.100851 ↗
10. Credit Risk Assessment Using a Combined Approach of Supervised and Unsupervised Learning
11. Unsupervised learning from EEG data for epilepsy: A systematic literature review10.1016/j.artmed.2025.103095 ↗
12. Kansal, T., Bahuguna, S., Singh, V., and Choudhury, T. (2018, January 21–22). Customer Segmentation using K-means Clustering. Proceedings of the 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS), Belgaum, India.10.1109/CTEMS.2018.8769171 ↗
13. Afzal, A., Khan, L., Hussain, M.Z., Hasan, M.Z., Mustafa, M., Khalid, A., Awan, R., Ashraf, F., Khan, Z.A., and Javaid, A. (2024, January 5–7). Customer Segmentation Using Hierarchical Clustering. Proceedings of the 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), Pune, India.10.1109/I2CT61223.2024.10543349 ↗
14. Supporting weather forecasting performance management at aerodromes through anomaly detection and hierarchical clustering10.1016/j.eswa.2022.119210 ↗
15. A K-Means clustering and SVM based hybrid concept drift detection technique for network anomaly detection10.1016/j.eswa.2022.116510 ↗
16. K-Means-based isolation forest10.1016/j.knosys.2020.105659 ↗
17. Content-based image retrieval system using ORB and SIFT features10.1007/s00521-018-3677-9 ↗
18. Study on density peaks clustering based on k-nearest neighbors and principal component analysis10.1016/j.knosys.2016.02.001 ↗
19. A recommender system based on collaborative filtering using ontology and dimensionality reduction techniques10.1016/j.eswa.2017.09.058 ↗
20. Multi-Dimensional Manifolds Consistency Regularization for semi-supervised remote sensing semantic segmentation10.1016/j.knosys.2024.112032 ↗
21. Unsupervised dimension-contribution-aware embeddings transformation for anomaly detection10.1016/j.knosys.2022.110209 ↗
22. Financial Fraud: A Review of Anomaly Detection Techniques and Recent Advances10.1016/j.eswa.2021.116429 ↗
23. A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system
24. Evaluation of deep unsupervised anomaly detection methods with a data-centric approach for on-line inspection10.1016/j.compind.2023.103852 ↗
25. An introduction to variational autoencoders10.1561/2200000056 ↗
26. adVAE: A self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection10.1016/j.knosys.2019.105187 ↗
27. Deep learning fault diagnosis method based on global optimization GAN for unbalanced data10.1016/j.knosys.2019.07.008 ↗
28. Unsupervised Medical Image Translation With Adversarial Diffusion Models10.1109/TMI.2023.3290149 ↗
29. Pythae: Unifying Generative Autoencoders in Python—A Benchmarking Use Case
30. A new hybrid feature selection approach using feature association map for supervised and unsupervised classification10.1016/j.eswa.2017.06.032 ↗
31. Shen, B. (2021, January 28–30). E-commerce Customer Segmentation via Unsupervised Machine Learning. Proceedings of the 2nd International Conference on Computing and Data Science, Stanford, CA, USA.10.1145/3448734.3450775 ↗
32. An unsupervised machine learning algorithms: Comprehensive review10.12785/ijcds/130172 ↗
33. A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends10.1109/TPAMI.2024.3415112 ↗
34. Big Self-Supervised Models are Strong Semi-Supervised Learners
35. Wang, X., Zhang, R., Shen, C., Kong, T., and Li, L. (2021, January 20–25). Dense Contrastive Learning for Self-Supervised Visual Pre-Training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA. Oral paper.10.1109/CVPR46437.2021.00304 ↗
36. Bootstrap Your Own Latent—A New Approach to Self-Supervised Learning
37. Koroteev, M.V. (2021). BERT: A review of applications in natural language processing and understanding. arXiv.
38. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv.
39. Pre-trained language models with domain knowledge for biomedical extractive summarization10.1016/j.knosys.2022.109460 ↗
40. A deeper graph neural network for recommender systems10.1016/j.knosys.2019.105020 ↗
41. A Comprehensive Survey on Graph Neural Networks10.1109/TNNLS.2020.2978386 ↗
42. Tan, Q., Liu, N., Huang, X., Choi, S.H., Li, L., Chen, R., and Hu, X. (March, January 27). S2GAE: Self-Supervised Graph Autoencoders are Generalizable Learners with Graph Masking. Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, Singapore.
43. Learning graph deep autoencoder for anomaly detection in multi-attributed networks10.1016/j.knosys.2022.110084 ↗
44. Sheng, V.S., Provost, F., and Ipeirotis, P.G. (, January 24–27August). Get another label? Improving data quality and data mining using multiple, noisy labelers. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, USA.
45. Aldoseri, A., Al-Khalifa, K.N., and Hamouda, A.M. (2023). Re-Thinking Data Strategy and Integration for Artificial Intelligence: Concepts, Opportunities, and Challenges. Appl. Sci., 13.10.20944/preprints202305.1565.v2 ↗
46. Towards Uncovering the Intrinsic Data Structures for Unsupervised Domain Adaptation Using Structurally Regularized Deep Clustering10.1109/TPAMI.2021.3087830 ↗
47. Unsupervised machine learning in urban studies: A systematic review of applications10.1016/j.cities.2022.103925 ↗
48. Christakis, N., and Drikakis, D. (2023). Reducing Uncertainty and Increasing Confidence in Unsupervised Learning. Mathematics, 11.10.3390/math11143063 ↗
49. Dealing with Noise Problem in Machine Learning Data-sets: A Systematic Review10.1016/j.procs.2019.11.146 ↗
50. Zhang, G., Hao, H., Wang, Y., Jiang, Y., Shi, J., Yu, J., Cui, X., Li, J., Zhou, S., and Yu, B. (2021). Optimized adaptive Savitzky-Golay filtering algorithm based on deep learning network for absorption spectroscopy. Spectrochim. Acta Part A Mol. Biomol. Spectrosc., 263.10.1016/j.saa.2021.120187 ↗
51. Denoising Autoencoders for Unsupervised Anomaly Detection in Brain MRI
52. Mansour, Y., and Heckel, R. (2023, January 17–24). Zero-Shot Noise2Noise: Efficient Image Denoising Without Any Data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01347 ↗
53. Krull, A., Buchholz, T.O., and Jug, F. (2019, January 15–20). Noise2Void - Learning Denoising From Single Noisy Images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00223 ↗
54. Quan, Y., Chen, M., Pang, T., and Ji, H. (2020, January 13–19). Self2Self With Dropout: Learning Self-Supervised Denoising From Single Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00196 ↗
55. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. Advances in Neural Information Processing Systems, Curran Associates, Inc.
56. Sucar, L.E. (2021). Probabilistic Graphical Models, Springer.10.1007/978-3-030-61943-5 ↗
57. Evaluating Approximate Inference in Bayesian Deep Learning
58. Modern Monte Carlo methods for efficient uncertainty quantification and propagation: A survey10.1002/wics.1539 ↗
59. A review of uncertainty quantification in deep learning: Techniques, applications and challenges10.1016/j.inffus.2021.05.008 ↗
60. Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., and Hassner, T. (2022). Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration. Computer Vision—ECCV 2022, Springer.
61. Li, Y., and Sur, P. (2025). Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling. arXiv.
62. Delving Deep Into Label Smoothing10.1109/TIP.2021.3089942 ↗
63. A hierarchical naive abas network classifier embedded GMM for textural image
64. Jiang, Z., Zheng, Y., Tan, H., Tang, B., and Zhou, H. (2017). Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering. arXiv.10.24963/ijcai.2017/273 ↗
65. Guo, X., Gao, L., Liu, X., and Yin, J. (2017, January 19–25). Improved deep embedded clustering with local structure preservation. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia.10.24963/ijcai.2017/243 ↗
66. Uncertainty-Aware Clustering for Unsupervised Domain Adaptive Object Re-Identification10.1109/TMM.2022.3149629 ↗
67. A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems10.1016/j.jcp.2022.111313 ↗
68. Uncertainty Estimation in Unsupervised MR-CT Synthesis of Scoliotic Spines10.1109/OJEMB.2023.3262965 ↗
69. On coughing and airborne droplet transmission to humans10.1063/5.0011960 ↗
70. Deep convolutional neural network for classification of sleep stages from single-channel EEG signals10.1016/j.jneumeth.2019.108312 ↗
71. Feature Alignment by Uncertainty and Self-Training for Source-Free Unsupervised Domain Adaptation10.1016/j.neunet.2023.02.009 ↗
72. Unsupervised domain adaptation based on the predictive uncertainty of models10.1016/j.neucom.2022.11.070 ↗
73. Eltouny, K., Gomaa, M., and Liang, X. (2023). Unsupervised learning methods for data-driven vibration-based structural health monitoring: A review. Sensors, 23.10.3390/s23063290 ↗
74. Christakis, N., and Drikakis, D. (2023). Unsupervised Learning of Particles Dispersion. Mathematics, 11.10.3390/math11173637 ↗
75. Unsupervised machine learning of virus dispersion indoors10.1063/5.0188610 ↗
76. On particle dispersion statistics using unsupervised learning and Gaussian mixture models10.1063/5.0229111 ↗
77. Advancing understanding of indoor conditions using artificial intelligence methods10.1063/5.0251749 ↗
78. Gaussian Mixture Model Based Classification Revisited: Application to the Bearing Fault Classification10.5545/sv-jme.2020.6563 ↗
79. A novel hybrid K-means and GMM machine learning model for breast cancer detection10.1109/ACCESS.2021.3123425 ↗
80. A comprehensive evaluation of OPTICS, GMM and K-means clustering methodologies for geochemical anomaly detection connected with sample catchment basins10.1016/j.chemer.2024.126094 ↗
81. An unsupervised machine-learning-based shock sensor: Application to high-order supersonic flow solvers10.1016/j.eswa.2024.126352 ↗
82. Statistical mixture modeling for cell subtype identification in flow cytometry10.1002/cyto.a.20583 ↗
83. Learning stable nonlinear dynamical systems with gaussian mixture models10.1109/TRO.2011.2159412 ↗
84. Wiest, J., Höffken, M., Kreßel, U., and Dietmayer, K. (2012, January 3–7). Probabilistic trajectory prediction with Gaussian mixture models. Proceedings of the 2012 IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain.10.1109/IVS.2012.6232277 ↗
85. Interpretable embeddings from molecular simulations using Gaussian mixture variational autoencoders10.1088/2632-2153/ab80b7 ↗
86. Bitaab, M., and Hashemi, S. (2017, January 6–7). Hybrid intrusion detection: Combining decision tree and gaussian mixture model. Proceedings of the 2017 14th International ISC (Iranian Society of Cryptology) Conference on Information Security and Cryptology (ISCISC), Shiraz, Iran.10.1109/ISCISC.2017.8488375 ↗
87. Wan, H., Wang, H., Scotney, B., and Liu, J. (2019, January 6–9). A novel Gaussian mixture model for classification. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.10.1109/SMC.2019.8914215 ↗
88. mclust 5: Clustering, classification and density estimation using Gaussian finite mixture models10.32614/RJ-2016-021 ↗
89. K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data10.1016/j.ins.2022.11.139 ↗
90. Reynolds, D.A. (1992). A Gaussian Mixture Modeling Approach to Text-Independent Speaker Identification, Georgia Institute of Technology.
91. Probability density estimation via an infinite Gaussian mixture model: Application to statistical process monitoring10.1111/j.1467-9876.2006.00560.x ↗
92. Fast and Efficient Limited Data Hyperspectral Remote Sensing Image Classification via GMM-Based Synthetic Samples10.1109/JSTARS.2019.2916495 ↗
93. Barton Tomas (2025, February 18). Clustering Benhmarks. Available online: https://github.com/deric/clustering-benchmark.
94. Aeberhard, S. (2025, November 14). Wine Data Set. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/109/wine.
95. Inertial microfluidic physics10.1039/c4lc00128a ↗
96. Saul, L., Weiss, Y., and Bottou, L. (2004). Self-Tuning Spectral Clustering. Advances in Neural Information Processing Systems, MIT Press.
97. Virus spreading in cruiser cabin10.1063/5.0169992 ↗
98. Mapping population and pathogen movements10.1093/inthealth/ihu006 ↗
99. Consideration of the aerosol transmission for COVID-19 and public health10.1111/risa.13500 ↗
100. Jarvis, M.C. (2020). Aerosol transmission of SARS-CoV-2: Physical principles and implications. Front. Public Health, 8.10.3389/fpubh.2020.590041 ↗