Formula-Driven Supervised Learning in Computer Vision: A Literature Survey
Source abstract
Abstract
Current computer vision research uses huge datasets with millions of images to pre-train vision models. This results in escalation of time and capital, ethical issues, moral issues, privacy issues, copyright issues, fairness issues, and others. To address these issues, several alternative learning schemes have been developed. One such scheme is formula-based supervised learning (FDSL). It is a form of supervised learning, which involves the use of mathematically generated images for the pre-training of deep models. Promising results have been obtained for computer-vision-related applications. In this comprehensive survey paper, a gentle introduction to FDSL is presented. The supporting theory, databases, experimentation and ensuing results are discussed. The research outcomes, issues and scope are also discussed. Finally, some of the most promising future directions for FDSL research are discussed. As FDSL is an important learning technique, this survey represents a useful resource for interested researchers working on solving various problem in computer vision and related areas of application.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image and Video Retrieval Techniques · Advanced Neural Network Applications · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electronics & Communication Engineering, Institute of Technology, University of Kashmir, Srinagar 190006, J&K, India
Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, AlKharj 16278, Saudi Arabia
Department of Computer Science, Faculty of Computers and Information, South Valley University, Qena 83523, Egypt
Department of Software Engineering, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, AlKharj 16278, Saudi Arabia
Funding
Prince Sattam bin Abdulaziz University
References (100+)
Show source-provided references
1. Hassaballah, M., and Awad, A.I. (2020). Deep Learning in Computer Vision: Principles and Applications, CRC Press.10.1201/9781351003827 ↗
2. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23-28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
3. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
4. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
5. An Efficient Deep Learning Framework for Distracted Driver Detection10.1109/ACCESS.2021.3138137 ↗
6. Fully Convolutional Networks for Semantic Segmentation10.1109/TPAMI.2016.2572683 ↗
7. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
8. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid Scene Parsing Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
9. A survey on instance segmentation: State of the art10.1007/s13735-020-00195-x ↗
10. Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015, January 7–12). Show and tell: A neural image caption generator. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298935 ↗
11. Amanat, A., Rizwan, M., Javed, A.R., Abdelhaq, M., Alsaqour, R., Pandya, S., and Uddin, M. (2022). Deep Learning for Depression Detection from Textual Data. Electronics, 11.10.3390/electronics11050676 ↗
12. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
13. Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey10.1109/TPAMI.2020.2992393 ↗
14. Hafiz, A.M., and Bhat, G.M. (2020). Deep Network Ensemble Learning applied to Image Classification using CNN Trees. arXiv.
15. Kaiser, M.S., Xie, J., and Rathore, V.S. (2020, January 11–12). Digit Image Recognition Using an Ensemble of One-Versus-All Deep Network Classifiers. Proceedings of the Information and Communication Technology for Competitive Strategies (ICTCS 2020), Jaipur, Rajasthan, India.
16. Fast training of deep networks with one-class CNNs
17. Hafiz, A.M., Parah, S.A., and Bhat, R.U.A. (2021). Attention mechanisms and deep learning for machine vision: A survey of the state of the art. arXiv.10.21203/rs.3.rs-510910/v1 ↗
18. Pereira, F., Burges, C., Bottou, L., and Weinberger, K. (2012, January 3–6). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
19. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
20. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
21. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
22. Huang, G., Liu, Z., Maaten, L.V.D., and Weinberger, K.Q. (2017, January 22–25). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
23. The open images dataset v410.1007/s11263-020-01316-z ↗
24. Hassaballah, M., and Khalid M., H. (2019). Recent Advances in Computer Vision: Theories and Applications, Springer.10.1007/978-3-030-03000-1 ↗
25. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 22–25). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
26. Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7–13). Learning Spatiotemporal Features With 3D Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.510 ↗
27. Tuba, M., Akashe, S., and Joshi, A. (2020, January 23–24). A Survey of Deep Learning Techniques for Medical Diagnosis. Proceedings of the Information and Communication Technology for Sustainable Development, Goa, India.10.1007/978-981-13-7166-0 ↗
28. Hafiz, A.M., Bhat, R.U.A., Parah, S.A., and Hassaballah, M. (2021). SE-MD: A Single-encoder multiple-decoder deep network for point cloud generation from 2D images. arXiv.
29. Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, P. (2017). The Kinetics Human Action Video Dataset. arXiv.
30. Yang, X., Song, Z., King, I., and Xu, Z. (2021). A Survey on Deep Semi-supervised Learning. arXiv.
31. A survey on semi-supervised learning10.1007/s10994-019-05855-6 ↗
32. Safe semi-supervised learning: A brief introduction10.1007/s11704-019-8452-2 ↗
33. Yalniz, I.Z., Jégou, H., Chen, K., Paluri, M., and Mahajan, D. (2019). Billion-scale semi-supervised learning for image classification. arXiv.
34. Sohn, K., Zhang, Z., Li, C.L., Zhang, H., Lee, C.Y., and Pfister, T. (2020). A Simple Semi-Supervised Learning Framework for Object Detection. arXiv.
35. Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., and Garnett, R. (2019, January 8–14). Consistency-based Semi-supervised Learning for Object detection. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
36. Jiang, B., Zhang, Z., Lin, D., Tang, J., and Luo, B. (2019, January 16–20). Semi-Supervised Learning With Graph Learning-Convolutional Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01157 ↗
37. Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018, January 8–14). Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
38. Li, Q., Han, Z., and Wu, X.M. (2018, January 2–7). Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence—AAAI’18/IAAI’18/EAAI’18, New Orleans Riverside, NO, USA.
39. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (2020, January 6–12). Graph Random Neural Networks for Semi-Supervised Learning on Graphs. Proceedings of the Advances in Neural Information Processing Systems, (Virtual-only Conference).
40. Saito, S., Yang, J., Ma, Q., and Black, M.J. (2021, January 20–25). SCANimate Weakly Supervised Learning of Skinned Clothed Avatar Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00291 ↗
41. Towards Safe Weakly Supervised Learning
42. Ahn, J., Cho, S., and Kwak, S. (2019, January 16–20). Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00231 ↗
43. A survey of semi-and weakly supervised semantic segmentation of images10.1007/s10462-019-09792-7 ↗
44. Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.M. (2020, January 23–28). Explanation-Based Weakly-Supervised Learning of Visual Relations with Graph Networks. Proceedings of the Computer Vision—ECCV 2020 (Online Conference).10.1007/978-3-030-58598-3 ↗
45. Gidaris, S., Singh, P., and Komodakis, N. (2018). Unsupervised Representation Learning by Predicting Image Rotations. arXiv.
46. Srivastava, N., Mansimov, E., and Salakhutdinov, R. (2015, January 6–11). Unsupervised Learning of Video Representations Using LSTMs. Proceedings of the 32nd International Conference on International Conference on Machine Learning—ICML’15, Lille, France.
47. Wang, X., and Gupta, A. (2015). Unsupervised Learning of Visual Representations using Videos. arXiv.10.1109/ICCV.2015.320 ↗
48. Lee, H.Y., Huang, J.B., Singh, M., and Yang, M.H. (2017, January 22–29). Unsupervised Representation Learning by Sorting Sequences. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.79 ↗
49. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 8–16). Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification. Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands.10.1007/978-3-319-46478-7 ↗
50. Doersch, C., Gupta, A., and Efros, A.A. (2015, January 7–13). Unsupervised Visual Representation Learning by Context Prediction. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.167 ↗
51. Zhang, R., Isola, P., and Efros, A.A. (2017, January 22–15). Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.76 ↗
52. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 8–16). Unsupervised Visual Representation Learning by Graph-Based Consistent Constraints. Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands.10.1007/978-3-319-46478-7 ↗
53. Caron, M., Bojanowski, P., Joulin, A., and Douze, M. (2018, January 8–14). Deep Clustering for Unsupervised Learning of Visual Features. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_9 ↗
54. Hoffer, E., Hubara, I., and Ailon, N. (2016). Deep unsupervised learning through spatial contrasting. arXiv.
55. Precup, D., and Teh, Y.W. (2017, January 6–11). Unsupervised Learning by Predicting Noise. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
56. Li, Y., Paluri, M., Rehg, J.M., and Dollar, P. (July, January 26). Unsupervised Learning of Edges. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.
57. Purushwalkam, S., and Gupta, A. (2016). Pose from Action: Unsupervised Learning of Pose Features based on Motion. arXiv.
58. Mahendran, A., Thewlis, J., and Vedaldi, A. (2018). Cross Pixel Optical Flow Similarity for Self-Supervised Learning. arXiv.10.1007/978-3-030-20873-8_7 ↗
59. Sayed, N., Brattoli, B., and Ommer, B. (2018). Cross and Learn: Cross-Modal Self-Supervision. arXiv.10.1007/978-3-030-12939-2_17 ↗
60. Korbar, B., Tran, D., and Torresani, L. (2018, January 2–8). Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization. Proceedings of the 32nd International Conference on Neural Information Processing Systems—NIPS’18, Montréal, QC, Canada.
61. Owens, A., and Efros, A.A. (2018). Audio-Visual Scene Analysis with Self-Supervised Multisensory Features. arXiv.10.1007/978-3-030-01231-1_39 ↗
62. Kim, D., Cho, D., and Kweon, I.S. (February, January 27). Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence—AAAI’19/IAAI’19/EAAI’19, Honolulu, HI, USA.
63. Jing, L., Yang, X., Liu, J., and Tian, Y. (2018). Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction. arXiv.
64. Fernando, B., Bilen, H., Gavves, E., and Gould, S. (2017, January 22–25). Self-Supervised Video Representation Learning with Odd-One-Out Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.607 ↗
65. Ren, Z., and Lee, Y. (2018, January 18–22). Cross-Domain Self-Supervised Multi-task Feature Learning Using Synthetic Imagery. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00086 ↗
66. Wang, X., He, K., and Gupta, A. (2017, January 22–29). Transitive Invariance for Self-Supervised Visual Representation Learning. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.149 ↗
67. Doersch, C., and Zisserman, A. (2017, January 22–29). Multi-task Self-Supervised Visual Learning. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.226 ↗
68. Mundhenk, T., Ho, D., and Chen, B.Y. (2018, January 18–22). Improvements to Context Based Self-Supervised Learning. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00973 ↗
69. Noroozi, M., Vinjimoor, A., Favaro, P., and Pirsiavash, H. (2018, January 18–22). Boosting Self-Supervised Learning via Knowledge Transfer. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00975 ↗
70. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018, January 8–14). Improving Spatiotemporal Self-supervision by Deep Reinforcement Learning. Proceedings of the Computer Vision—ECCV 2018, Munich, Germany.10.1007/978-3-030-01252-6 ↗
71. Liu, Y., Jin, M., Pan, S., Zhou, C., Zheng, Y., Xia, F., and Yu, P. (2022). Graph Self-Supervised Learning A Survey. IEEE Trans. Knowl. Data Eng.10.1109/TKDE.2022.3172903 ↗
72. Baevski, A., Hsu, W.N., Xu, Q., Babu, A., Gu, J., and Auli, M. (2022). data2vec A General Framework for Self-supervised Learning in Speech, Vision and Language. arXiv.
73. Self-supervised Learning Generative or Contrastive
74. Li, C.L., Sohn, K., Yoon, J., and Pfister, T. (2021, January 19–25). CutPaste Self-Supervised Learning for Anomaly Detection and Localization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Online Conference).10.1109/CVPR46437.2021.00954 ↗
75. Xie, Y., Xu, Z., Zhang, J., Wang, Z., and Ji, S. (2022). Self-Supervised Learning of Graph Neural Networks A Unified Review. IEEE Trans. Pattern Anal. Mach. Intell.10.1109/TPAMI.2022.3170559 ↗
76. VATT Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
77. Kataoka, H., Okayasu, K., Matsumoto, A., Yamagata, E., Yamada, R., Inoue, N., Nakamura, A., and Satoh, Y. (December, January 30). Pre-training without Natural Images. Proceedings of the Asian Conference on Computer Vision (ACCV), (Online Conference).
78. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018, January 8–14). Exploring the Limits of Weakly Supervised Pretraining. Proceedings of the Computer Vision—ECCV 2018, Munich, Germany.10.1007/978-3-030-01252-6 ↗
79. Li, W., Wang, L., Li, W., Agustsson, E., and Van Gool, L. (2017). WebVision Database: Visual Learning and Understanding from Web Data. arXiv.
80. A Survey on Semi-, Self- and Unsupervised Learning for Image Classification10.1109/ACCESS.2021.3084358 ↗
81. Song, X., and Yang, H. (2022). A Survey of Unsupervised Learning in Medical Image Registration. Int. J. Health Syst. Transl. Med. (IJHSTM), 2.10.4018/IJHSTM.2022010101 ↗
82. A Survey of Unsupervised Generative Models for Exploratory Data Analysis and Representation Learning
83. Stability estimation for unsupervised clustering: A Review10.1002/wics.1575 ↗
84. A formula predicting the effective dose of febuxostat in chronic kidney disease patients with asymptomatic hyperuricemia based on a retrospective study and a validation cohort10.5414/CN109867 ↗
85. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 8–16). Colorful Image Colorization. Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands.10.1007/978-3-319-46478-7 ↗
86. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (July, January 26). Context Encoders: Feature Learning by Inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.
87. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 8–16). Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles. Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands.10.1007/978-3-319-46478-7 ↗
88. Xie, S., Girshick, R., Dollar, P., Tu, Z., and He, K. (2017, January 22–25). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗
89. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv.
90. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–22). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
91. Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., and Pang, R. (November, January 27). Searching for MobileNetV3. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.10.1109/ICCV.2019.00140 ↗
92. Noroozi, M., Pirsiavash, H., and Favaro, P. (2017, January 22–29). Representation Learning by Learning to Count. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.628 ↗
93. III, H.D., and Singh, A. (2020, January 6–10). A Simple Framework for Contrastive Learning of Visual Representations. Proceedings of the 37th International Conference on Machine Learning, (Online Conference).
94. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 14–19). Momentum Contrast for Unsupervised Visual Representation Learning. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Online Conference).10.1109/CVPR42600.2020.00975 ↗
95. The Fractal Geometry of Nature10.1119/1.13295 ↗
96. Local connected fractal dimensions and lacunarity analyses of 60 degrees fluorescein angiograms
97. Fractal methods and results in cellular morphology - dimensions, lacunarity and multifractals10.1016/S0165-0270(96)00080-5 ↗
98. Barnsley, M.F. (2014). Fractals Everywhere, Academic Press.
99. Rendering algorithms for deterministic fractals10.1109/38.364961 ↗
100. 3-D IFS fractals as real-time graphics model10.1016/S0097-8493(97)00014-9 ↗