Transfer Incremental Learning Using Data Augmentation
Source abstract
Abstract
Deep learning-based methods have reached state of the art performances, relying on a large quantity of available data and computational power. Such methods still remain highly inappropriate when facing a major open machine learning problem, which consists of learning incrementally new classes and examples over time. Combining the outstanding performances of Deep Neural Networks (DNNs) with the flexibility of incremental learning techniques is a promising venue of research. In this contribution, we introduce Transfer Incremental Learning using Data Augmentation (TILDA). TILDA is based on pre-trained DNNs as feature extractors, robust selection of feature vectors in subspaces using a nearest-class-mean based technique, majority votes and data augmentation at both the training and the prediction stages. Experiments on challenging vision datasets demonstrate the ability of the proposed method for low complexity incremental learning, while achieving significantly better accuracy than existing incremental counterparts.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Domain Adaptation and Few-Shot Learning · Machine Learning and ELM
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Lab-STICC, IMT Atlantique, 29280 Plouzané, France
Montreal Institute for Learning Algorithms (MILA), Montréal, QC H3C 3J7, Canada
References (29)
Show source-provided references
1. Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14–18). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.10.1145/1553374.1553380 ↗
2. Kasabov, N. (2013). Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines, Springer Science & Business Media.
3. Catastrophic forgetting in connectionist networks10.1016/S1364-6613(99)01294-2 ↗
4. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (arXiv, 2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size, arXiv.
5. Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv.
6. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (arXiv, 2015). Rethinking the inception architecture for computer vision, arXiv.10.1109/CVPR.2016.308 ↗
7. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
8. A survey on transfer learning10.1109/TKDE.2009.191 ↗
9. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (arXiv, 2013). Intriguing properties of neural networks, arXiv.
10. Rebuffi, S.A., Kolesnikov, A., Sperl, G., and Lampert, C.H. (2017, January 21–26). iCaRL: Incremental Classifier and Representation Learning. Proceedings of the 2017 Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.587 ↗
11. Schlimmer, J.C., and Fisher, D. (1986, January 14–15). A case study of incremental concept induction. Proceedings of the 5th National Conference on Artificial Intelligence, AAAI 1986, Philadelphia, PA, USA.
12. Thrun, S. (1996). Is learning the n-th thing any easier than learning the first?. Advances in Neural Information Processing Systems, MIT Press.
13. Hybrid decision tree10.1016/S0950-7051(02)00038-2 ↗
14. Syed, N.A., Huan, S., Kah, L., and Sung, K. (1999). Incremental Learning with Support Vector Machines, CiteSeerX.10.1145/312129.312267 ↗
15. Incremental and decremental support vector machine learning
16. An online incremental learning support vector machine for large-scale data10.1007/s00521-011-0793-1 ↗
17. Learn++: An incremental learning algorithm for supervised neural networks10.1109/5326.983933 ↗
18. Learn++. NC: Combining Ensemble of Classifiers with Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes10.1109/TNN.2008.2008326 ↗
19. Pentina, A., Sharmanska, V., and Lampert, C.H. (2015, January 7–12). Curriculum learning of multiple tasks. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7299188 ↗
20. Distance-based image classification: Generalizing to new classes at near-zero cost10.1109/TPAMI.2013.83 ↗
21. Mensink, T., Verbeek, J., Perronnin, F., and Csurka, G. (2012, January 7–13). Metric learning for large scale image classification: Generalizing to new classes at near-zero cost. Proceedings of the Computer Vision–ECCV 2012, Florence, Italy.10.1007/978-3-642-33709-3_35 ↗
22. Ristin, M., Guillaumin, M., Gall, J., and Van Gool, L. (2014, January 23–28). Incremental learning of NCM forests for large-scale image classification. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.467 ↗
23. Hacene, G.B., Gripon, V., Farrugia, N., Arzel, M., and Jezequel, M. (2017, January 3–5). Budget restricted incremental learning with pre-trained convolutional neural networks and binary associative memories. Proceedings of the 2017 IEEE International Workshop on Signal Processing Systems (SiPS), Lorient, France.10.1109/SiPS.2017.8109978 ↗
24. Kuzborskij, I., Orabona, F., and Caputo, B. (2013, January 23–28). From n to n + 1: Multiclass transfer incremental learning. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.431 ↗
25. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, MIT Press.
26. Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2014, January 23–28). Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.10.1109/CVPR.2014.222 ↗
27. Hong, S., You, T., Kwak, S., and Han, B. (2015, January 6–11). Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network. Proceedings of the International Conference on Machine Learning, Lille, France.
28. Ciresan, D., Meier, U., Gambardella, L., and Schmidhuber, J. (arXiv, 2010). Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition, arXiv.10.1162/NECO_a_00052 ↗
29. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗