Class-Wise Classifier Design Capable of Continual Learning Using Adaptive Resonance Theory-Based Topological Clustering
Source abstract
Abstract
This paper proposes a supervised classification algorithm capable of continual learning by utilizing an Adaptive Resonance Theory (ART)-based growing self-organizing clustering algorithm. The ART-based clustering algorithm is theoretically capable of continual learning, and the proposed algorithm independently applies it to each class of training data for generating classifiers. Whenever an additional training data set from a new class is given, a new ART-based clustering will be defined in a different learning space. Thanks to the above-mentioned features, the proposed algorithm realizes continual learning capability. Simulation experiments showed that the proposed algorithm has superior classification performance compared with state-of-the-art clustering-based classification algorithms capable of continual learning.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Domain Adaptation and Few-Shot Learning · Neural Networks and Applications · Machine Learning and ELM
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Informatics, Osaka Metropolitan University, Osaka 599-8531, Japan
Faculty of Information Technology, University of Central Punjab, Lahore 54000, Pakistan
Faculty of Navigation, Dalian Maritime University, Dalian 116026, China
Funding
Japan Society for the Promotion of Science (JSPS) KAKENHI
References (55)
Show source-provided references
1. Catastrophic interference in connectionist networks: The sequential learning problem10.1016/S0079-7421(08)60536-8 ↗
2. Continual lifelong learning with neural networks: A review10.1016/j.neunet.2019.01.012 ↗
3. Van de Ven, G.M., and Tolias, A.S. (2019). Three scenarios for continual learning. arXiv.
4. Wiewel, F., and Yang, B. (2019). Localizing catastrophic forgetting in neural networks. arXiv.
5. Overcoming catastrophic forgetting in neural networks10.1073/pnas.1611835114 ↗
6. An incremental network for on-line unsupervised classification and topology learning10.1016/j.neunet.2005.04.006 ↗
7. A growing neural gas network learns topologies
8. A fast nearest neighbor classifier based on self-organizing incremental neural network10.1016/j.neunet.2008.07.001 ↗
9. Lifelong learning of human actions with deep neural network self-organization10.1016/j.neunet.2017.09.001 ↗
10. A self-organizing incremental neural network for continual supervised learning10.1016/j.eswa.2021.115662 ↗
11. Competitive learning: From interactive activation to adaptive resonance10.1111/j.1551-6708.1987.tb00862.x ↗
12. Correntropy: Properties and applications in non-Gaussian signal processing10.1109/TSP.2007.896065 ↗
13. Self-organizing maps with information theoretic learning10.1016/j.neucom.2013.12.059 ↗
14. Topological Clustering via Adaptive Resonance Theory With Information Theoretic Learning10.1109/ACCESS.2019.2921832 ↗
15. Masuyama, N., Amako, N., Nojima, Y., Liu, Y., Loo, C.K., and Ishibuchi, H. (2019, January 6–9). Fast Topological Adaptive Resonance Theory Based on Correntropy Induced Metric. Proceedings of the IEEE Symposium Series on Computational Intelligence, Xiamen, China.10.1109/SSCI44817.2019.9003098 ↗
16. Adaptive resonance theory-based topological clustering with a divisive hierarchical structure capable of continual learning10.1109/ACCESS.2022.3186479 ↗
17. Finite mixture models10.1146/annurev-statistics-031017-100325 ↗
18. Least squares quantization in PCM10.1109/TIT.1982.1056489 ↗
19. The ART of adaptive pattern recognition by a self-organizing neural network10.1109/2.33 ↗
20. SOINN+, a self-organizing incremental neural network for unsupervised learning from noisy data streams10.1016/j.eswa.2019.113069 ↗
21. A self-organising network that grows when required10.1016/S0893-6080(02)00078-3 ↗
22. Evolutionary fuzzy ARTMAP neural networks for classification of semiconductor defects
23. OnARTMAP: A fuzzy ARTMAP-based architecture10.1016/j.neunet.2017.11.012 ↗
24. A novel fuzzy ARTMAP with area of influence10.1016/j.neucom.2020.11.053 ↗
25. Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system10.1016/0893-6080(91)90056-B ↗
26. The Bayesian ARTMAP10.1109/TNN.2007.900234 ↗
27. Incremental Local Distribution-Based Clustering Using Bayesian Adaptive Resonance Theory10.1109/TNNLS.2019.2919723 ↗
28. Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence10.1016/j.neunet.2019.08.033 ↗
29. Kernel Bayesian ART and ARTMAP10.1016/j.neunet.2017.11.003 ↗
30. A Kernel Bayesian Adaptive Resonance Theory with a Topological Structure10.1142/S0129065718500521 ↗
31. Dual vigilance fuzzy adaptive resonance theory10.1016/j.neunet.2018.09.015 ↗
32. Zenke, F., Poole, B., and Ganguli, S. (2017, January 6–11). Continual learning through synaptic intelligence. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
33. Shin, H., Lee, J.K., Kim, J., and Kim, J. (2017, January 4–9). Continual learning with deep generative replay. Proceedings of the the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
34. Nguyen, C.V., Li, Y., Bui, T.D., and Turner, R.E. (May, January 30). Variational continual learning. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
35. An open-ended continual learning for food recognition using class incremental extreme learning machines10.1109/ACCESS.2020.2991810 ↗
36. An incremental kernel extreme learning machine for multi-label learning with emerging new labels10.1109/ACCESS.2020.2978648 ↗
37. Lifelong learning with shared and private latent representations learned through synaptic intelligence10.1016/j.neunet.2023.04.005 ↗
38. CRNet: A fast continual learning framework with random theory10.1109/TPAMI.2023.3262853 ↗
39. Generative negative replay for continual learning10.1016/j.neunet.2023.03.006 ↗
40. Lin, H., Zhang, B., Feng, S., Li, X., and Ye, Y. (2023, January 17–24). PCR: Proxy-based contrastive replay for online class-incremental continual learning. Proceedings of the the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02322 ↗
41. Federated probability memory recall for federated continual learning10.1016/j.ins.2023.02.015 ↗
42. Hao, H., Chu, Z., Zhu, S., Jiang, G., Wang, Y., Jiang, C., Zhang, J., Jiang, W., Xue, S., and Zhou, J. (2023). Continual learning in predictive autoscaling. arXiv.10.1145/3583780.3615463 ↗
43. A comprehensive study of class incremental learning algorithms for visual tasks10.1016/j.neunet.2020.12.003 ↗
44. Lifelong learning of spatiotemporal representations with dual-memory recurrent self-organization10.3389/fnbot.2018.00078 ↗
45. Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps10.1109/72.159059 ↗
46. Normal reference bandwidths for the general order, multivariate kernel density derivative estimator10.1016/j.spl.2012.07.020 ↗
47. Silverman, B.W. (2018). Density Estimation for Statistics and Data Analysis, Routledge.10.1201/9781315140919 ↗
48. Masuyama, N., Nojima, Y., Loo, C.K., and Ishibuchi, H. (2020, January 1–4). Multi-label classification based on adaptive resonance theory. Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence, Canberra, Australia.10.1109/SSCI47803.2020.9308356 ↗
49. K-means properties on six clustering benchmark datasets10.1007/s10489-018-1238-7 ↗
50. A survey of deep neural network architectures and their applications10.1016/j.neucom.2016.12.038 ↗
51. Dua, D., and Graff, C. (2019). UCI Machine Learning Repository, School of Information and Computer Sciences, University of California.
52. Cluster ensembles—A knowledge reuse framework for combining multiple partitions
53. Comparing partitions10.1007/BF01908075 ↗
54. Statistical comparisons of classifiers over multiple data sets
55. Learning under concept drift: A review