Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models
Source abstract
Abstract
Since the 1950s, when the Turing Test was introduced, there has been notable progress in machine language intelligence. Language modeling, crucial for AI development, has evolved from statistical to neural models over the las... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. A comprehensive overview of large language models
2. Pinker · The Language Instinct: How the Mind Creates Language · 2003 · 1 · 34
3. The faculty of language: What is it, who has it, and how did it evolve?10.1126/science.298.5598.1569 ↗
4. Learning machines
5. A survey of large language models
6. Jelinek · Statistical Methods for Speech Recognition · 1998 · 297
7. Two decades of statistical language modeling: Where do we go from here?10.1109/5.880083 ↗
8. Introduction to the special issue on word sense disambiguation: The state of the art
9. SRILM–an extensible language modeling toolkit
10. A second-order hidden Markov model for part-of-speech tagging
11. Statistical language models for information retrieval a critical review10.1561/1500000008 ↗
12. BERT: Pretraining of deep bidirectional transformers for language understanding
13. Exploring the limits of transfer learning with a unified text-to-text transformer
14. What do we mean by GenAI? a systematic mapping of the evolution, trends, and techniques involved in generative AI
15. Alayrac · Advances in Neural Information Processing Systems · 2022 · 35 · 23716
16. Visual instruction tuning
17. VideoChat: Chat-centric video understanding
18. Video- LLaMA: An instruction-tuned audio-visual language model for video understanding
19. WavCaps: A ChatGPT-assisted weaklylabelled audio captioning dataset for audio-language multimodal research
20. LLM: Multi-modal language modeling with image, audio, video, and text integration
21. Language is not all you need: Aligning perception with language models
22. A comprehensive survey of AI-generated content (AIGC): A history of generative ai from GAN to ChatGPT
23. PaLM-E: An embodied multimodal language model
24. Visual ChatGPT: Talking, drawing and editing with visual foundation models
25. GPT-4 technical report
26. Aligning large language models with human: A survey
27. Language models are few-shot learners
28. PaLM: Scaling language modeling with pathways
29. Galactica: A large language model for science
30. LLaMA: Open and efficient foundation language models
31. Chain-of-thought prompting elicits reasoning in large language models
32. Attention is all you need
33. Generating long sequences with sparse transformers
34. Fast transformer decoding: One write-head is all you need
35. FlashAttention: Fast and memory-efficient exact attention with IO-awareness
36. PyTorch: An imperative style, high-performance deep learning library
37. DeepSpeed: System optimizations enable training deep learning models with over 100 billion parameters
38. Megatron-LM: Training multi-billion parameter language models using model parallelism
39. FlAshattention-2: Faster attention with better parallelism and work partitioning
40. Efficient memory management for large language model serving with pagedattention
41. Cross-modal attention with semantic consistence for image– text matching10.1109/TNNLS.2020.2967597 ↗
42. A novel graph-attention based multimodal fusion network for joint classification of hyperspectral image and lidar data10.1016/j.eswa.2024.123587 ↗
43. CLIP-Nav: Using CLIP for zero-shot vision-andlanguage navigation
44. RoboCLIP: One demonstration is enough to learn robot policies
45. Improved baselines with visual instruction tuning
46. ERNIE 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation
47. PanGu-α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation
48. CPM-2: Large-scale cost-effective pre-trained language models10.1016/j.aiopen.2021.12.003 ↗
49. ERNIE 3.0 Titan: Exploring large-scale knowledge enhanced pre-training for language understanding and generation
50. GPT-NeoX-20B: An open-source autoregressive language model
51. BLOOM: A 176B-parameter open-access multilingual language model
52. GLaM: Efficient scaling of language models with mixture-of-experts
53. LaMDA: Language models for dialog applications
54. UL2: Unifying language learning paradigms
55. GLM-130B: An open bilingual pre-trained model
56. Jurassic-1: Technical details and evaluation
57. What changes can large-scale language models bring? intensive study on HyperCLOVA: Billions scale Korean generative pretrained transformers
58. Yuan 1.0: Large-scale pre-trained language model in zero-shot and few-shot learning
59. PanGu-Σ: Towards trillion parameter language model with sparse heterogeneous computing
60. XinYuan 2.0: A large chinese financial chat model with hundreds of billions parameters
61. OPT: Open pre-trained transformer language models
62. Scaling language models: Methods, analysis & insights from training gopher
63. Training compute-optimal large language models
64. CoQA: A conversational question answering challenge10.1162/tacl_a_00266 ↗
65. Transcending scaling laws with 0.1% extra compute
66. AlexaTM 20B: Few-shot learning using a large-scale multilingual seq2seq model
67. BloombergGPT: A large language model for finance
68. mT5: A massively multilingual pretrained text-to-text transformer
69. Competitionlevel code generation with AlphaCode10.1126/science.abq1158 ↗
70. CodeGen: An open large language model for code with multi-turn program synthesis
71. CodeT5+: Open code large language models for code understanding and generation
72. The ethical implications of DALL-E: Opportunities and challenges
73. Hierarchical text-conditional image generation with clip latents
74. Photorealistic text-to-image diffusion models with deep language understanding
75. Scaling autoregressive models for content-rich text-to-image generation
76. MiniGPT-4: Enhancing vision-language understanding with advanced large language models
77. Scaling up visual and visionlanguage representation learning with noisy text supervision
78. Learning transferable visual models from natural language supervision
79. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models
80. CogVLM: Visual expert for pretrained language models
81. PaLM 2 technical report
82. WizardCoder: Empowering code large language models with evol-instruct
83. Qwen-VL: A frontier large vision-language model with versatile abilities
84. MoE-LLaVA: Mixture of experts for large vision-language models
85. Sequence to sequence learning with neural networks
86. A. Radford and K. Narasimhan, “Improving language understanding by generative pre-training,” 2018.
87. Generating wikipedia by summarizing long sequences
88. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
89. RoBERTa: A robustly optimized BERT pretraining approach
90. ALBERT: A lite BERT for self-supervised learning of language representations
91. XLNet: Generalized autoregressive pretraining for language understanding
92. Transformer-XL: Attentive language models beyond a fixedlength context
93. ELECTRA: Pretraining text encoders as discriminators rather than generators
94. MM-LLMs: Recent advances in multimodal large language models
95. High-performance large-scale image recognition without normalization
96. An image is worth 16x16 words: Transformers for image recognition at scale
97. Exploring the limits of masked visual representation learning at scale
98. A survey on generative ai and LLM for video generation, understanding, and streaming
99. X-LLM: Bootstrapping advanced large language models by treating multi-modalities as foreign languages
100. HuBERT: Self-supervised speech representation learning by masked prediction of hidden units10.1109/TASLP.2021.3122291 ↗