An Intelligent Docent System with a Small Large Language Model (sLLM) Based on Retrieval-Augmented Generation (RAG)
Source abstract
Abstract
This study designed and empirically evaluated a method to enhance information accessibility for museum and art gallery visitors using a small Large Language Model (sLLM) based on the Retrieval-Augmented Generation (RAG) framework. Over 199,000 exhibition descriptions were collected and refined, and a question-answering dataset consisting of 102,000 pairs reflecting user personas was constructed to develop DocentGemma, a domain-optimized language model. This model was fine-tuned through Low-Rank Adaptation (LoRA) based on Google’s Gemma2-9B and integrated with FAISS and OpenSearch-based document retrieval systems within the LangChain framework. Performance evaluation was conducted using a dedicated Q&A benchmark for the docent domain, comparing the model against five commercial and open-source LLMs (including GPT-3.5 Turbo, LLaMA3.3-70B, and Gemma2-9B). DocentGemma achieved an accuracy of 85.55% and a perplexity of 3.78, demonstrating competitive performance in language generation and response accuracy within the domain-specific context. To enhance retrieval relevance, a Spatio-Contextual Retriever (SC-Retriever) was introduced, which combines semantic similarity and spatial proximity based on the user’s query and location. An ablation study confirmed that integrating both modalities improved retrieval quality, with the SC-Retriever achieving a recall@1 of 53.45% and a Mean Reciprocal Rank (MRR) of 68.12, representing a 17.5 20% gain in search accuracy compared to baseline models such as GTE and SpatialNN. System performance was further validated through field deployment at three major exhibition venues in Seoul (the Seoul History Museum, the Hwan-ki Museum, and the Hanseong Baekje Museum). A user test involving 110 participants indicated high response credibility and an average satisfaction score of 4.24. To ensure accessibility, the system supports various output formats, including multilingual speech and subtitles. This work illustrates a practical application of integrating LLM-based conversational capabilities into traditional docent services and suggests potential for further development toward location-aware interactive systems and AI-driven cultural content services.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Recommender Systems and Techniques · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, University of Hanyang, Seoul 04763, Republic of Korea
References (74)
Show source-provided references
1. Asakawa, S., Guerreiro, J., Ahmetovic, D., Kitani, K., and Asakawa, C. (2018, January 22–24). The Present and Future of Museum Accessibility for People with Visual Impairments. Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’18), Galway, Ireland.10.1145/3234695.3240997 ↗
2. Research on Museum Accessibility for the Visually Impaired10.1051/shsconf/202316302022 ↗
3. Mazzanti, P., Ferracani, A., Bertini, M., and Principi, F. (2025). Reshaping Museum Experiences with AI: The ReInHerit Toolkit. Heritage, 8.10.20944/preprints202505.2405.v1 ↗
4. Qu, G., Chen, Q., Wei, W., Lin, Z., Chen, X., and Huang, K. (2025). Mobile Edge Intelligence for Large Language Models: A Contemporary Survey. IEEE Commun. Surv. Tutor., Early Access.10.36227/techrxiv.172115025.57884352/v1 ↗
5. Trichopoulos, G., Konstantakis, M., Alexandridis, G., and Caridakis, G. (2023). Large Language Models as Recommendation Systems in Museums. Electronics, 12.10.20944/preprints202307.1393.v1 ↗
6. An, H., Park, W., Liu, P., and Park, S. (2025). Mobile-AI-Based Docent System: Navigation and Localization for Visually Impaired Gallery Visitors. Appl. Sci., 15.10.3390/app15095161 ↗
7. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
8. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P.J. (2023). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv.
9. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models are Few-Shot Learners. arXiv.
10. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., and Altman, S. (2024). GPT-4 Technical Report. arXiv.
11. Driess, D., Xia, F., Sajjadi, M.S.M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., and Yu, T. (2023). PaLM-E: An Embodied Multimodal Language Model. arXiv.
12. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., and Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv.
13. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T.L., Cao, Y., and Narasimhan, K. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv.
14. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training language models to follow instructions with human feedback. arXiv.
15. Christiano, P., Leike, J., Brown, T.B., Martic, M., Legg, S., and Amodei, D. (2023). Deep reinforcement learning from human preferences. arXiv.
16. Bender, E.M., Gebru, T., McMillan-Major, A., and Shmitchell, S. (2021, January 3–10). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), Virtual Event.10.1145/3442188.3445922 ↗
17. Guerrero, P., Pan, Y., and Kashyap, S. (2025). Efficient Deployment of Vision-Language Models on Mobile Devices: A Case Study on OnePlus 13R. arXiv.
18. Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. (2020). TinyBERT: Distilling BERT for Natural Language Understanding. arXiv.10.18653/v1/2020.findings-emnlp.372 ↗
19. Gemma Team, Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M.S., and Love, J. (2024). Gemma: Open Models Based on Gemini Research and Technology. arXiv.
20. Kim, J.-H., and Choi, Y.-S. (2025). Lightweight Pre-Trained Korean Language Model Based on Knowledge Distillation and Low-Rank Factorization. Entropy, 27.10.3390/e27040379 ↗
21. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., and Rocktäschel, T. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv.
22. A Balanced Approach of Rapid Genetic Exploration and Surrogate Exploitation for Hyperparameter Optimization
23. Wahsheh, F.R., Moaiad, Y.A., El-Ebiary, Y.A.B., Hamzah, W.M.A.F.W., Yusoff, M.H., and Pandey, B. (2023, January 10–12). E-Commerce Product Retrieval Using Knowledge from GPT-4. Proceedings of the 2023 International Conference on Computer Science and Emerging Technologies (CSET), Bangalore, India.10.1109/CSET58993.2023.10346860 ↗
24. The Probabilistic Relevance Framework: BM25 and Beyond10.1561/1500000019 ↗
25. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-T. (2020). Dense Passage Retrieval for Open-Domain Question Answering. arXiv.10.18653/v1/2020.emnlp-main.550 ↗
26. Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., and Grave, E. (2022). Atlas: Few-shot Learning with Retrieval Augmented Language Models. arXiv.
27. Lin, W., Chen, J., Mei, J., Coca, A., and Byrne, B. (2023). Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering. arXiv.10.18653/v1/2022.emnlp-main.772 ↗
28. Billion-Scale Similarity Search with GPUs10.1109/TBDATA.2019.2921572 ↗
29. Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L., and Jégou, H. (2024). The Faiss library. arXiv.
30. Khan, A., S, N.C., and Gangodkar, D.R. (2024, January 20–21). An Overview Recent Trends and Challenges in Multi-Modal Image Retrieval Using Deep Learning. Proceedings of the 2024 International Conference on Communication, Computing and Energy Efficient Technologies (I3CEET), Gautam Buddha Nagar, India.10.1109/I3CEET61722.2024.10993792 ↗
31. A Survey of Recommender Systems Based on Deep Learning10.1109/ACCESS.2018.2880197 ↗
32. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., and Chua, T.-S. (2017). Neural Collaborative Filtering. arXiv.10.1145/3038912.3052569 ↗
33. Deep Learning for Anomaly Detection: A Review10.1145/3439950 ↗
34. Akkiraju, R., Xu, A., Bora, D., Yu, T., An, L., Seth, V., Shukla, A., Gundecha, P., Mehta, H., and Jha, A. (2024). FACTS About Building Retrieval Augmented Generation-based Chatbots. arXiv.
35. Creating Large Language Model Applications Utilizing LangChain: A Primer on Developing LLM Apps Fast
36. Mavroudis, V. (2024). LangChain. Preprints, online.10.20944/preprints202411.0566.v1 ↗
37. LangChain AI (2025, August 18). Build a Customer Support Bot. LangGraph Tutorials. Available online: https://langchain-ai.github.io/langgraph/tutorials/customer-support/customer-support/.
38. AUTOMATING DATA ANALYSIS WITH LANGCHAIN
39. Felicetti, A., and Niccolucci, F. (2025). Artificial Intelligence and Ontologies for the Management of Heritage Digital Twins Data. Data, 10.10.20944/preprints202412.0333.v1 ↗
40. Al-Okby, M.F.R., Junginger, S., Roddelkopf, T., and Thurow, K. (2024). UWB-Based Real-Time Indoor Positioning Systems: A Comprehensive Review. Appl. Sci., 14.10.3390/app142311005 ↗
41. Qian, C., Xie, Z., Wang, Y., Liu, W., Zhu, K., Xia, H., Dang, Y., Du, Z., Chen, W., and Yang, C. (2025). Scaling Large Language Model-based Multi-Agent Collaboration. arXiv.
42. Patriwala, A. (2025, August 18). LangChain: A Comprehensive Framework for Building LLM Applications (with Code). Medium. Available online: https://medium.com/@patriwala/langchain-a-comprehensive-framework-for-building-llm-applications-e2800dba2753.
43. Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv.10.18653/v1/D19-1410 ↗
44. LangChain (2025, August 18). How-To Guides. LangChain Documentation. Available online: https://python.langchain.com/docs/how_to/.
45. Google (2025, August 18). Gemma: Introducing Google’s New Family of Lightweight, State-of-the-art Open Models. Available online: https://ai.google.dev/gemma.
46. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv.
47. Yang, J., and Zhu, C. (2023). Research on UWB Indoor Positioning System Based on TOF Combined Residual Weighting. Sensors, 23.10.3390/s23031455 ↗
48. Research on indoor positioning system algorithm based on UWB technology10.1016/j.measen.2024.101121 ↗
49. Zhao, W., Goudar, A., Tang, M., and Schoellig, A.P. (2024). Ultra-wideband Time Difference of Arrival Indoor Localization: From Sensor Placement to System Evaluation. arXiv.10.1177/02783649241230640 ↗
50. Verde, D., Romero, L., Faria, P.M., and Paiva, S. (2023). Indoor Content Delivery Solution for a Museum Based on BLE Beacons. Sensors, 23.10.3390/s23177403 ↗
51. BLE Beacons for Indoor Positioning at an Interactive IoT-Based Smart Museum10.1109/JSYST.2020.2969088 ↗
52. Gong, M., Li, Z., and Li, W. (2022, January 22–25). Research on ultra-wideband (UWB) indoor accurate positioning technology under signal interference. Proceedings of the 2022 IEEE International Conferences on Internet of Things (iThings) and IEEE Green Computing & Communications (GreenCom) and IEEE Cyber, Physical & Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics (Cybermatics), Espoo, Finland.10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics55523.2022.00060 ↗
53. Krebs, S., and Herter, T. (2024). Ultra-Wideband Positioning System Based on ESP32 and DWM3000 Modules. arXiv.
54. Kingsbury, B., Saon, G., Mangu, L., Padmanabhan, M., and Sarikaya, R. (2002, January 13–17). Robust speech recognition in noisy environments: The 2001 IBM SPINEevaluation system. Proceedings of the 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Orlando, FL, USA.10.1109/ICASSP.2002.1005673 ↗
55. Google Cloud (2025, August 18). Speech-to-Text. Available online: https://cloud.google.com/speech-to-text/.
56. Naver Cloud Platform (2025, August 18). Clova Speech Synthesis. Available online: https://guide.ncloud-docs.com/docs/clovaspeech-overview.
57. OpenAI (2025, August 18). Text to Speech. Available online: https://platform.openai.com/docs/guides/text-to-speech.
58. Raitio, T., Latorre, J., Davis, A., Morrill, T., and Golipour, L. (2023). Improving the quality of neural TTS using long-form content and multi-speaker multi-style modeling. arXiv.10.21437/SSW.2023-23 ↗
59. Does Use of Text-to-Speech and Related Read-Aloud Tools Improve Reading Comprehension for Students with Reading Disabilities? A Meta-Analysis10.1177/0022219416688170 ↗
60. Challenges in clinical natural language processing for automated disorder normalization10.1016/j.jbi.2015.07.010 ↗
61. National Museum of Korea (2025, August 18). Digital Collections. Available online: https://www.museum.go.kr/ENG/contents/E0402000000.do?searchId=search&schM=list.
62. Radeva, I., Popchev, I., Doukovska, L., and Dimitrova, M. (2024). Web Application for Retrieval-Augmented Generation: Implementation and Testing. Electronics, 13.10.20944/preprints202403.0844.v1 ↗
63. Ozdemir, A., Odaci, B., Tanatar Baruh, L., Varol, O., and Balcisoy, S. (2025). Enhancing Cultural Heritage Archive Analysis via Automated Entity Extraction and Graph-Based Representation Learning. J. Comput. Cult. Herit.10.1145/3746658 ↗
64. Zhang, J., Cui, Y., Wang, W., and Cheng, X. (2025). TrustDataFilter: Leveraging Trusted Knowledge Base Data for More Effective Filtering of Unknown Information. arXiv.
65. Elastic (2025, August 18). Nori Analysis Plugin. Available online: https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-nori.html.
66. Li, J., and Bikakis, A. (2023). Towards a Semantics-Based Recommendation System for Cultural Heritage Collections. Appl. Sci., 13.10.3390/app13158907 ↗
67. Lee, J., Cha, H., Hwangbo, Y., and Cheon, W. (2024). Enhancing Large Language Model Reliability: Minimizing Hallucinations with Dual Retrieval-Augmented Generation Based on the Latest Diabetes Guidelines. J. Pers. Med., 14.10.3390/jpm14121131 ↗
68. Chen, D., Sun, N., Lee, J.-H., Zou, C., and Jeon, W.-S. (2024). Digital Technology in Cultural Heritage: Construction and Evaluation Methods of AI-Based Ethnic Music Dataset. Appl. Sci., 14.10.3390/app142310811 ↗
69. Gîrbacia, F. (2024). An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage. Electronics, 13.10.3390/electronics13183738 ↗
70. Podsiadło, M., and Chahar, S. (2016, January 8–12). Text-to-Speech for Individuals with Vision Loss: A User Study. Proceedings of the Interspeech 2016, San Francisco, CA, USA.10.21437/Interspeech.2016-1376 ↗
71. Shin, D., Park, J., and Kim, H. (2024). LoRA versus Prompt-Tuning: A Comparative Study in Domain-Specific Language Model Adaptation. Electronics, 6.
72. Rasool, A., Shahzad, M.I., Aslam, H., Chan, V., and Arshad, M.A. (2025). Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation. AI, 6.10.3390/ai6030056 ↗
73. Yu, D., Bao, R., Ning, R., Peng, J., Mai, G., and Zhao, L. (2025). Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions. arXiv.
74. Theodoropoulos, G.S., Nørvåg, K., and Doulkeridis, C. (2024, January 25–28). Efficient Semantic Similarity Search over Spatio-Textual Data. Proceedings of the 27th International Conference on Extending Database Technology (EDBT), Paestum, Italy.