BIMCoder: A Comprehensive Large Language Model Fusion Framework for Natural Language-Based BIM Information Retrieval
Source abstract
Abstract
Building Information Modeling (BIM) has excellent potential to enhance building operation and maintenance. However, as a standardized data format in the architecture, engineering, and construction (AEC) industry, the retrieval of BIM information generally requires specialized software. Cumbersome software operations prevent its effective application in the actual operation and management of buildings. This paper presents BIMCoder, a model designed to translate natural language queries into structured query statements compatible with professional BIM software (e.g., BIMserver v1.5). It serves as an intermediary component between users and various BIM platforms, facilitating access for users without specialized BIM knowledge. A dedicated BIM information query dataset was constructed, comprising 1680 natural language query and structured BIM query string pairs, categorized into 12 groups. Three classical pre-trained large language models (LLMs) (ERNIE 3.0, Llama-13B, and SQLCoder) were evaluated on this dataset. A fine-tuned model based on SQLCoder was then trained. Subsequently, a fusion model (BIMCoder) integrating ERNIE and SQLCoder was designed. Test results demonstrate that the proposed BIMCoder model achieves an outstanding accurate matching rate of 87.16% and an Execution Accuracy rate of 88.75% for natural language-based BIM information retrieval. This study confirms the feasibility of natural language-based BIM information retrieval and offers a novel solution to reduce the complexity of BIM system interaction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: BIM and Construction Integration · Infrastructure Maintenance and Monitoring · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018, China
Funding
National Natural Science Foundation of China
References (39)
Show source-provided references
1. A Natural-Language-Based Approach to Intelligent Data Retrieval and Representation for Cloud BIM10.1111/mice.12151 ↗
2. Multi-scale Information Retrieval for BIM using Hierarchical Structure Modelling and Natural Language Processing
3. Yin, M., Li, H., Wu, Z., and Tang, L. (2023). Information Requirement Analysis for Establishing BIM-Oriented Natural Language Interfaces. International Conference on Sustainable Buildings and Structures Towards a Carbon Neutral Future, Springer.10.1007/978-981-99-7965-3_46 ↗
4. Multimodal interaction for information retrieval using natural language10.1016/j.csi.2012.11.002 ↗
5. Cloud-based building information modelling (Cloud-BIM): Systematic literature review and bibliometric-qualitative analysis10.1016/j.autcon.2022.104468 ↗
6. Dynamic prompt-based virtual assistant framework for BIM information search10.1016/j.autcon.2023.105067 ↗
7. Artificial intelligence-based voice assistant for BIM data management10.1016/j.autcon.2022.104320 ↗
8. Huang, C. (2017, January 29–30). Application of BIM Technology Based on Autodesk Revit in Construction and Installation Engineering. Proceedings of the 2017 3rd International Conference on Economics, Social Science, Arts, Education and Management Engineering (ESSAEME 2017), Huhhot, China.10.2991/essaeme-17.2017.262 ↗
9. Aish, R. (2025, April 30). Bentley Systems. Migration from an Individual to an Enterprise Computing Model and Its Implications for AEC Research. Available online: https://www.bentley.com/software/building-design/.
10. Dassault systèmes: A CAD success story10.1109/MAHC.2024.3457165 ↗
11. Liu, Z., and Wang, Y. (2020). Real-Time Interactive Online 3D Graphical User Interface (GUI) Technical Implementation and Usability Test for Architectural Technical Teaching. Lecture Notes in Computer Science, Springer.10.1007/978-3-030-49760-6_32 ↗
12. Sampaio, A.Z. (2017, January 22–24). 4D/BIM model linked to VR technology. Proceedings of the Virtual Reality International Conference-Laval Virtual 2017, Laval, France.10.1145/3110292.3110298 ↗
13. Loporcaro, G., Bellamy, L., McKenzie, P., and Riley, H. (2019). Evaluation of Microsoft HoloLens Augmented Reality Technology as a construction checking tool. Constr. Innov. Int. J., 124–135.
14. Implementing common data environments in architectural technology studies
15. BIMASR: Framework for voice-based BIM information retrieval10.1061/(ASCE)CO.1943-7862.0002138 ↗
16. Practical Deployment of BIM-Interoperable Voice-Based Intelligent Virtual Agent to Support Construction Worker Productivity10.1061/JCEMD4.COENG-14475 ↗
17. Zhoui, H., Wong, M.O., Ying, H., and Lee, S. (July, January 30). A framework of a multi-user voice-driven BIM-based navigation system for fire emergency response. Proceedings of the 26th International Workshop on Intelligent Computing in Engineering (EG-ICE 2019), The European Group for Intelligent Computing in Engineering (EG-ICE), Leuven, Belgium.
18. Park, D., and Kim, E. (2024). Method of interacting between humans and conversational voice agent systems. Heliyon, 10.10.1016/j.heliyon.2023.e23573 ↗
19. Cloud computing for chatbot in the construction industry: An implementation framework for conversational-BIM voice assistant10.1016/j.dte.2024.100031 ↗
20. Hu, Z., and Zhang, J. (2013, January 9–12). BIM oriented intelligent data mining and representation. Proceedings of the 30th CIB W78 International Conference, Beijing, China.
21. Leveraging Natural Language Processing for Automated Information Inquiry from Building Information Models
22. Matching real-world facilities to building information modeling data using natural language processing10.1109/ACCESS.2019.2937219 ↗
23. Natural-language-based intelligent retrieval engine for BIM object database10.1016/j.compind.2019.02.016 ↗
24. An ontology-aided, natural language-based approach for multi-constraint BIM model querying10.1016/j.jobe.2023.107066 ↗
25. A query expansion method for retrieving online BIM resources based on Industry Foundation Classes10.1016/j.autcon.2015.04.006 ↗
26. Construction risk knowledge management in BIM using ontology and semantic web technology10.1016/j.ssci.2016.04.008 ↗
27. BIM-enabled semantic web for automated safety checks in subway construction10.1016/j.autcon.2022.104454 ↗
28. BIM and ontology-based knowledge management for dam safety monitoring10.1016/j.autcon.2022.104649 ↗
29. Guo, D., Onstein, E., and Rosa, A.D.L. (2020). An approach of automatic SPARQL generation for BIM data extraction. Appl. Sci., 10.10.3390/app10248794 ↗
30. Cursi, S., Simeone, D., and Coraglia, U. (2017, January 20–22). An ontology-based platform for BIM semantic enrichment. Proceedings of the ShoCK—Sharing of Computable Knowledge, International Conference on Education and Research in Computer Aided Architectural Design in Europe, Rome, Italy.10.52842/conf.ecaade.2017.2.649 ↗
31. Fahad, M., Bus, N., and Fies, B. (2018). Semantic BIM reasoner for the verification of IFC Models. eWork and eBusiness in Architecture, Engineering and Construction, CRC Press.10.1201/9780429506215-45 ↗
32. A simplified relational database schema for transformation of BIM data into a query-efficient and spatially enabled database10.1016/j.autcon.2017.10.002 ↗
33. Cost estimate system using structured query language in BIM
34. BIM-based big data analytic system for healthcare facility management10.1016/j.jobe.2022.105713 ↗
35. Li, B., Luo, Y., Chai, C., Li, G., and Tang, N. (2024). The dawn of natural language to SQL: Are we fully ready?. arXiv.10.14778/3681954.3682003 ↗
36. Sun, Y., Wang, S., Feng, S., Ding, S., Pang, C., Shang, J., Liu, J., Chen, X., Zhao, Y., and Lu, Y. (2021). Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv.
37. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv.
38. Thakkar, H., and Manimaran, A. (2023, January 7–9). Comprehensive Examination of Instruction-Based Language Models: A Comparative Analysis of Mistral-7B and Llama-2-7B. Proceedings of the 2023 International Conference on Emerging Research in Computational Science (ICERCS), Coimbatore, India.10.1109/ICERCS57948.2023.10434081 ↗
39. GPT-3.5, GPT-4, or BARD? Evaluating LLMs reasoning ability in zero-shot setting and performance boosting through prompts10.1016/j.nlp.2023.100032 ↗