A Generative Artificial Intelligence Using Multilingual Large Language Models for ChatGPT Applications
Source abstract
Abstract
ChatGPT plays significant roles in the third decade of the 21st Century. Smart cities applications can be integrated with ChatGPT in various fields. This research proposes an approach for developing large language models using generative artificial intelligence models suitable for small- and medium-sized enterprises with limited hardware resources. There are many generative AI systems in operation and in development. However, the technological, human, and financial resources required to develop generative AI systems are impractical for small- and medium-sized enterprises. In this study, we present a proposed approach to reduce training time and computational cost that is designed to automate question–response interactions for specific domains in smart cities. The proposed model utilises the BLOOM approach as its backbone for using generative AI to maximum the effectiveness of small- and medium-sized enterprises. We have conducted a set of experiments on several datasets associated with specific domains to validate the effectiveness of the proposed model. Experiments using datasets for the English and Vietnamese languages have been combined with model training using low-rank adaptation to reduce training time and computational cost. In comparative experimental testing, the proposed model outperformed the ‘Phoenix’ multilingual chatbot model by achieving a 92% performance compared to ‘ChatGPT’ for the English benchmark.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Recommender Systems and Techniques · IoT and Edge/Fog Computing · AI in Service Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Technology and Digital Economics, National Economics University, 207 Giai Phong Road, Hai Ba Trung District, Hanoi 10000, Vietnam
School of Information Science and Engineering, Lanzhou University, Feiyun Building, 222 Tianshui South Road, Chengguan Qu, Lanzhou 730030, China
School of Information and Communication Technology, Hanoi University of Science and Technology, 1 Dai Co Viet, Le Dai Hanh, Hai Ba Trung District, Hanoi 10000, Vietnam
Funding
National Economics University, Vietnam
References (61)
Show source-provided references
1. Smart care spaces: Pervasive sensing technologies for at-home care10.1504/IJAHUC.2014.064862 ↗
2. Proposed Intelligent Decision Support System Using Hedge Algebra Integrated with Picture Fuzzy Relations for Improvement of Decision-Making in Medical Diagnoses10.1007/s40815-023-01548-4 ↗
3. Disruptive Innovation: An Intellectual History and Directions for Future Research10.1111/joms.12349 ↗
4. Proposed Distance and Entropy Measures of Picture Fuzzy Sets in Decision Support Systems
5. A Novel Approach of Voterank-Based Knowledge Graph for Improvement of Multi-Attributes Influence Nodes on Social Networks10.2478/jaiscr-2023-0013 ↗
6. Pham, V.H., Nguyen, Q.H., Troung, V.P., and Tran, L.P.T. (2022, January 19–20). The Proposed Context Matching Algorithm and Its Application for User Preferences of Tourism in COVID-19 Pandemic. Proceedings of the International Conference on Innovative Computing and Communications, Delhi, India.10.1007/978-981-19-2535-1_22 ↗
7. Generative AI and the end of corpus-assisted data-driven learning? Not so fast!10.1016/j.acorp.2023.100066 ↗
8. Liu, X., Ji, K., Fu, Y., Tam, W., Du, Z., Yang, Z., and Tang, J. (2022, January 22–27). P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland.10.18653/v1/2022.acl-short.8 ↗
9. Marjanovic, O., Skaf-Molli, H., Molli, P., and Godart, C. (2007, January 12–15). Collaborative practice-oriented business processes Creating a new case for business process management and CSCW synergy. Proceedings of the 2007 International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom 2007), New York, NY, USA.10.1109/COLCOM.2007.4553874 ↗
10. How the Internet of Things reshapes the organization of innovation and entrepreneurship10.1016/j.technovation.2022.102644 ↗
11. What’s “New” About New Forms of Organizing?10.5465/amr.2011.0436 ↗
12. Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT10.1038/s43588-023-00527-x ↗
13. A comprehensive survey of ChatGPT: Advancements, applications, prospects, and challenges10.1016/j.metrad.2023.100022 ↗
14. GPT-3.5, GPT-4, or BARD? Evaluating LLMs reasoning ability in zero-shot setting and performance boosting through prompts10.1016/j.nlp.2023.100032 ↗
15. Scao, T.L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A.S., Yvon, F., and Gallé, M. (2022). BLOOM A 176B-Parameter Open-Access Multilingual Language Model. arXiv.
16. The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
17. Sun, X., Ji, Y., Ma, B., and Li, X. (2023). A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model. arxiv.
18. Are both generative AI and ChatGPT game changers for 21st-Century operations and supply chain excellence?10.1016/j.ijpe.2023.109015 ↗
19. Varghese, J., and Chapiro, J. (2023). ChatGPT: The transformative influence of generative AI on science and healthcare. J. Hepatol., 80.10.1016/j.jhep.2023.07.028 ↗
20. Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy10.1016/j.ijinfomgt.2023.102642 ↗
21. ChatGPT and the rise of generative AI: Threat to academic integrity?10.1016/j.jrt.2023.100060 ↗
22. ChatGPT impacts on access-efficiency, employment, education and ethics: The socio-economics of an AI language model
23. Baidoo-Anu, D., and Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. SSRN.10.2139/ssrn.4337484 ↗
24. Generative AI: Here to stay, but for good?10.1016/j.techsoc.2023.102372 ↗
25. Alabool, H.M. (2023, January 9–10). ChatGPT in Education: SWOT analysis approach. Proceedings of the 2023 International Conference on Information Technology (ICIT), Amman, Jordan.10.1109/ICIT58056.2023.10225801 ↗
26. Disruptive technologies: An expanded view10.1142/S1363919605001162 ↗
27. Christensen, C., Raynor, M.E., and McDonald, R. (2013). Disruptive Innovation, Harvard Business Review.
28. Technology, the Technology Complex and the Paradox of Technological Determinism10.1080/09537320127285 ↗
29. Technological determinism is dead; long live technological determinism
30. Embodying the past, designing the future: Technological determinism reconsidered in technology education
31. Generative AI and deceptive news consumption10.1016/j.econlet.2023.111317 ↗
32. Möslein, K.M., and Neyer, A.K. (2015). Wiley Encyclopedia of Management, John Wiley & Sons.
33. Disruptive Innovation: In Need of Better Theory10.1111/j.1540-5885.2005.00177.x ↗
34. Surprising Advances in Generative Artificial Intelligence Prompt Amazement—And Worries10.1016/j.eng.2023.04.004 ↗
35. Checkland, P., and Holwell, S. (1997). Information, Systems and Information Systems: Making Sense of the Field, John Wiley and Sons.
36. The influence of ethnicity and language variation on undergraduates’ evaluations of Dutch-speaking instructors in Belgium: A contextualized speaker evaluation experiment10.1016/j.langcom.2022.01.004 ↗
37. Chatbots in e-commerce: The effect of chatbot language style on customers’ continuance usage intention and attitude toward brand10.1016/j.jretconser.2022.103209 ↗
38. Xin, D., Mao, J., and Liu, M. (2010, January 7–9). The Effects of Parasocial Relationships in the Adoption of Mobile Commerce Application: A Conceptual Model. Proceedings of the 2010 International Conference on E-Business and E-Government, Guangzhou, China.10.1109/ICEE.2010.45 ↗
39. Li, R.Y., and Lin, S. (2018, January 23–27). Stages of Concern and Parasocial Interaction: Perception, Attitude, and Adoption of Social Media. Proceedings of the 2018 1st IEEE International Conference on Knowledge Innovation and Invention (ICKII), Jeju, Republic of Korea.10.1109/ICKII.2018.8569132 ↗
40. Hanief, S., Handayani, P.W., Azzahro, F., and Pinem, A.A. (2019, January 10–11). Parasocial Relationship Analysis on Digital Celebrities Follower’s Purchase Intention. Proceedings of the 2019 2nd International Conference of Computer and Informatics Engineering (IC2IE), Banyuwangi, Indonesia.10.1109/IC2IE47452.2019.8940853 ↗
41. Chen, W.K., Wen, H.Y., and Silalahi, A.D.K. (2021, January 29–31). Parasocial Interaction with YouTubers: Does Sensory Appeal in the YouTubers’ Video Influences Purchase Intention?. Proceedings of the 2021 IEEE International Conference on Social Sciences and Intelligent Management (SSIM), Taichung, Taiwan.10.1109/SSIM49526.2021.9555195 ↗
42. Chen, Z., Jiang, F., Chen, J., Wang, T., Yu, F., Chen, G., Zhang, H., Liang, J., Zhang, C., and Zhang, Z. (2023). Phoenix: Democratizing ChatGPT across Languages. arXiv.
43. Exploring generative artificial intelligence preparedness among university language instructors: A case study10.1016/j.caeai.2023.100156 ↗
44. Reconceptualizing ChatGPT and generative AI as a student-driven innovation in higher education10.1016/j.procir.2023.05.002 ↗
45. The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation10.1016/j.caeai.2023.100147 ↗
46. Reliance on metrics is a fundamental challenge for AI10.1016/j.patter.2022.100476 ↗
47. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv.
48. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. (2023). Stanford Alpaca: A Strong, Replicable Instruction-Following Model, Center for Research on Foundation Models, Stanford University.
49. Aghajanyan, A., Zettlemoyer, L., and Gupta, S. (2020). Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. arXiv.10.18653/v1/2021.acl-long.568 ↗
50. Ren, J., Rajbhandari, S., Aminabadi, R.Y., Ruwase, O., Yang, S., Zhang, M., Li, D., and He, Y. (2021). ZeRO-Offload: Democratizing Billion-Scale Model Training. arXiv.
51. Press, O., Smith, N.A., and Lewis, M. (2022). Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. arXiv.
52. Hu, E., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models. arXiv.
53. Inductive reasoning in humans and large language models10.1016/j.cogsys.2023.101155 ↗
54. Utilizing AI questionnaire translations in cross-cultural and intercultural research: Insights and recommendations10.1016/j.ijintrel.2023.101888 ↗
55. Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., and Xing, E.P. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv.
56. Cuong, L.A., Hieu, N.T., Cuong, N.V., Que, N.N., Nguyen, L.-M., and Nguyen, C.-T. (2024, March 30). Vlsp 2023 Challenge on Vietnamese Large Language Models 2023. Available online: https://vlsp.org.vn/vlsp2023/eval/vllm.
57. Beeching, E., Fourrier, C., Habib, N., Han, S., Lambert, N., Rajani, N., Sanseviero, O., Tunstall, L., and Wolf, T. (2024, March 30). Open LLM Leaderboard. Available online: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard.
58. Nguyen, X., Zhang, W., Li, X., Aljunied, M., Tan, Q., Cheng, L., Chen, G., Deng, Y., Yang, S., and Liu, C. (2023). SeaLLMs—Large Language Models for Southeast Asia. arXiv.
59. Nguyen, D.Q., Nguyen, L.T., Tran, C., Nguyen, D.N., Phung, D., and Bui, H. (2023). PhoGPT: Generative Pre-training for Vietnamese. arXiv.
60. Lester, B., Al-Rfou, R., and Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. arXiv.10.18653/v1/2021.emnlp-main.243 ↗
61. Futures of artificial intelligence through technology readiness levels10.1016/j.tele.2020.101525 ↗