The Future of Artificial Intelligence in the Face of Data Scarcity
Source abstract
Abstract
Dealing with data scarcity is the biggest challenge faced by Artificial Intelligence (AI), and it will be interesting to see how we overcome this obstacle in the future, but for now, “THE SHOW MUST GO ON!!!” As AI spreads and transforms more industries, the lack of data is a significant obstacle: the best methods for teaching machines how real-world processes work. This paper explores the considerable implications of data scarcity for the AI industry, which threatens to restrict its growth and potential, and proposes plausible solutions and perspectives. In addition, this article focuses highly on different ethical considerations: privacy, consent, and non-discrimination principles during AI model developments under limited conditions. Besides, innovative technologies are investigated through the paper in aspects that need implementation by incorporating transfer learning, few-shot learning, and data augmentation to adapt models so they could fit effective use processes in low-resource settings. This thus emphasizes the need for collaborative frameworks and sound methodologies that ensure applicability and fairness, tackling the technical and ethical challenges associated with data scarcity in AI. This article also discusses prospective approaches to dealing with data scarcity, emphasizing the blend of synthetic data and traditional models and the use of advanced machine learning techniques such as transfer learning and few-shot learning. These techniques aim to enhance the flexibility and effectiveness of AI systems across various industries while ensuring sustainable AI technology development amid ongoing data scarcity.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Big Data and Business Intelligence
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (79)
Show source-provided references
1. The rise of synthetic data: enhancing AI and machine learning model training to address data scarcity and mitigate privacy risks
2. Panagiotou E, Qian H, Marx S, Ntoutsi E. Generative AI based augmentation for offshore jacket design: an integrated approach for mixed tabular data generation under data scarcity and imbalance. [cited 2025 Jan 1]. Available from: 10.2139/ssrn.4703856.
3. Reliability assessment of high-quality new products with data scarcity10.1080/00207543.2020.1758355 ↗
4. Digital duplicates and the scarcity problem: might AI make us less scarce and therefore less valuable?10.1007/s13347-024-00795-z ↗
5. Shen · AI in education: effective machine learning methods to improve data scarcity and knowledge generalization · 2023
6. Bai J, Alzubaidi L, Wang Q, Kuhl E, Bennamoun M, Gu Y. Utilising physics-guided deep learning to overcome data scarcity. arXiv:2211.15664. 2022.
7. Kaai · Addressing data scarcity in domain generalization for computer vision applications in image classification · 2024
8. Addressing data scarcity in the medical domain: a GPT-based approach for synthetic data generation and feature extraction10.3390/info15050264 ↗
9. How to build good AI solutions when data is scarce
10. Re-thinking data strategy and integration for artificial intelligence: concepts, opportunities, and challenges10.3390/app13127082 ↗
11. Audacity of huge: overcoming challenges of data scarcity and data quality for machine learning in computational materials discovery10.1016/j.coche.2021.100778 ↗
12. OpenAI, Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, et al. GPT-4 technical report. arXiv:2303.08774. 2023.
13. Scaling laws for generative mixed-modal language models
14. Position: will we run out of data? Limits of LLM scaling based on human-generated data
15. Villalobos P, Ho A, Sevilla J, Besiroglu T, Heim L, Hobbhahn M. Will we run out of data? Limits of LLM scaling based on human-generated data. arXiv:2211.04325. 2022. doi:10.48550/arXiv.2211.04325.
16. PaLM: scaling language modeling with pathways
17. Discovering data sets through machine learning: an ensemble approach to uncovering the prevalence of government-funded data sets
18. Data scarcity, robustness and extreme multi-label classification10.1007/s10994-019-05791-5 ↗
19. Larson J, Steven Truitt S. GraphRAG: unlocking LLM discovery on narrative private data. [cited 2025 Jan 15]. Available from: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/.
20. On the use of trajectory data for tackling data scarcity10.1016/j.is.2025.102523 ↗
21. Not enough data to be fair? Evaluating fairness implications of data scarcity solutions10.24251/HICSS.2025.822 ↗
22. Data scarcity in recommendation systems: a survey
23. Khalil M, Vadiee F, Shakya R, Liu Q. Creating artificial students that never existed: leveraging large language models and CTGANs for synthetic data generation. arXiv:2501.01793. 2025. doi:10.1145/3706468.10.1145/3706468.3706523 ↗
24. White J, Madaan P, Shenoy N, Agnihotri A, Sharma M, Doshi J. A case for rejection in low resource ML deployment. arXiv:2208.06359. 2022.
25. Systematic review of data-centric approaches in artificial intelligence and machine learning10.1016/j.dsm.2023.06.001 ↗
26. A mirror environment to produce artificial intelligence training data10.1109/ACCESS.2022.3154825 ↗
27. SynthEval: a framework for detailed utility and privacy evaluation of tabular synthetic data10.1007/s10618-024-01081-4 ↗
28. Current strategies to address data scarcity in artificial intelligence-based drug discovery: a comprehensive review10.1016/j.compbiomed.2024.108734 ↗
29. Niel O. A novel algorithm can generate data to train machine learning models in conditions of extreme scarcity of real world data. arXiv:2305.00987. 2023.
30. On the private data synthesis through deep generative models for data scarcity of industrial Internet of Things10.1109/TII.2021.3133625 ↗
31. A systematic review on data scarcity problem in deep learning: solution and applications10.1145/3502287 ↗
32. Nahid MMH, Bin Hasan S. SafeSynthDP: leveraging large language models for privacy-preserving synthetic data generation using differential privacy. arXiv:2412.20641. 2024.10.32388/PJIL3E ↗
33. Data augmentation based on topic relevance to enhance text classification in scarcity of training data
34. Hoang V. Mitigating data scarcity for large language models. arXiv:2302.01806. 2023.
35. A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications10.1186/s40537-023-00727-2 ↗
36. Less annotating, more classifying: addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI10.1017/pan.2023.20 ↗
37. Proposal for the generation of profiles using a synthetic database
38. Oxford · Data scarcity challenges AI developers globally · 2024
39. RCT: resource constrained training for edge AI10.1109/TNNLS.2022.3190451 ↗
40. [cited 2025 Mar 12]. Available from: https://brusselsprivacyhub.com/wp-content/uploads/2024/02/Personal-Data-Protection-in-Brazil.pdf.
41. [cited 2025 Mar 12]. Available from: https://www.novaoneadvisor.com/report/artificial-intelligence-market.
42. [cited 2025 Mar 12]. Available from: https://ourworldindata.org/data-insights/investment-in-generative-ai-has-surged-recently.
43. [cited 2025 Mar 12]. Available from: https://hai.stanford.edu/news/ai-index-state-ai-13-charts.
44. Addressing data scarcity in ML-based failure-cause identification in optical networks through generative models10.1016/j.yofte.2025.104137 ↗
45. [cited 2025 Mar 12]. Available from: https://www.fbcinc.com/source/virtualhall_images/2024_Virtual_Events/USDA_Innovation/Databricks/State_of_Data___AI_Resource.pdf.
46. Belkada Y, Dettmers T, Pagnoni A, Gugger S, Mangrulkar S. Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA. [cited 2025 Jan 15]. Available from: https://huggingface.co/blog/4bit-transformers-bitsandbytes.
47. A brief survey on big data: technologies, terminologies and data-intensive applications10.1186/s40537-022-00659-3 ↗
48. Comprehensive analysis of various big data classification techniques: a challenging overview10.1142/S0219649222500836 ↗
49. Big data: past, present, and future insights
50. [cited 2025 Mar 12]. Available from: https://www.sciencealert.com/the-world-is-running-out-of-data-to-feed-ai-experts-warn.
51. Harnessing the power of synthetic data in healthcare: innovation, application, and privacy10.1038/s41746-023-00927-3 ↗
52. Dang H, Mecke L, Lehmann F, Goller S, Buschek D. How to prompt? Opportunities and challenges of zero- and few-shot learning for human-AI interaction in creative applications of generative models. arXiv:2209.01390. 2022.
53. Yang S, Xiao W, Zhang M, Guo S, Zhao J, Shen F. Image data augmentation for deep learning: A survey. arXiv:2204.08610. 2022.
54. IoT-based healthcare-monitoring system towards improving quality of life: a review10.3390/healthcare10101993 ↗
55. A systematic review of synthetic data generation techniques using generative AI10.3390/electronics13173509 ↗
56. Self-supervised learning: a succinct review10.1007/s11831-023-09884-2 ↗
57. A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios: theories, applications and challenges10.1016/j.ymssp.2021.108487 ↗
58. [cited 2025 Mar 12]. Available from: https://technologymagazine.com/articles/how-googles-ai-breakthroughs-earned-nobel-recognition.
59. [cited 2025 Mar 12]. Available from: https://research.google/blog/automl-for-large-scale-image-classification-and-object-detection.
60. [cited 2025 Mar 12]. Available from: https://innovationexchange.mayoclinic.org/an-introduction-to-federal-funding-for-innovation/.
61. [cited 2025 Mar 12]. Available from: https://www.techmonitor.ai/risks/extreme-weather-events/ibm-weather-and-climate-model?cf-view.
62. Universities engaging social media users: an investigation of quantitative relationships between universities’ Facebook followers/interactions and university attributes
63. A critical AI view on autonomous vehicle navigation: the growing danger10.3390/electronics13183660 ↗
64. [cited 2025 Mar 12]. Available from: https://www.europarl.europa.eu/RegData/etudes/STUD/2020/641530/EPRS_STU(2020)641530_EN.pdf.
65. [cited 2025 Mar 12]. Available from: https://oag.ca.gov/privacy/ccpa.
66. [cited 2025 Mar 12]. Available from: https://www.twobirds.com/en/insights/2024/china/ai-governance-in-china-strategies-initiatives-and-key.
67. [cited 2025 Mar 12]. Available from: https://www.hhs.gov/hipaa/for-professionals/privacy/laws.
68. [cited 2025 Mar 12]. Available from: https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud/data.
69. [cited 2025 Mar 12]. Available from: https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic.
70. A comprehensive review of AI advancement using testFAILS and testFAILS-2 for the pursuit of AGI10.3390/electronics13244991 ↗
71. Haruni R. Nvidia CEO maps out bold vision: AGI and robotics set to merge. [cited 2025 Jan 15]. Available from: https://wallstreetpit.com/120150-nvidia-ceo-maps-out-bold-vision-agi-and-robotics-set-to-merge/.
72. Barlow G. Altman predicts artificial superintelligence (AGI) will happen this year. [cited 2025 Jan 15]. Available from: https://www.techradar.com/computing/artificial-intelligence/sam-altman-predicts-artificial-superintelligence-agi-will-happen-this-year.
73. Robison K. OpenAI cofounder Ilya Sutskever says the way AI is built is about to change. [cited 2025 Jan 15]. Available from: https://www.theverge.com/2024/12/13/24320811/what-ilya-sutskever-sees-openai-model-data-training.
74. Bias and cyberbullying detection and data generation using transformer artificial intelligence models and top large language models10.3390/electronics13173431 ↗
75. The AI-powered evolution of big data10.3390/app142210176 ↗
76. Edge D, Larson J. LazyGraphRAG: setting a new standard for quality and cost. [cited 2025 Jan 15]. Available from: https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/.
77. Feng Y, Shen A, Hu J, Liang Y, Wang S, Du J. Enhancing few-shot learning with integrated data and GAN model approaches. arXiv:2411.16567. 2024.10.1109/DSInS64146.2024.10992169 ↗
78. Wiethof C, Bittner EA. Hybrid intelligence—combining the human in the loop with the computer in the loop: a systematic literature review. [cited 2025 Jan 1]. Available from: https://www.researchgate.net/publication/356209722.
79. A systematic review of green AI10.1002/widm.1507 ↗