journal article·2026-06-23Open access

LLM-Integrated Semantic Deep Learning Framework for Automated Floor Plan Analysis, Area Estimation, and Compliance Assessment of Existing Buildings

Yuxuan Guo, Xiaodeng Zhou, Su-Kit Tang

Applied Sciences· Volume 16· Issue 13

Abstract

The digitization of existing building stock often depends on legacy 2D raster floor plans (scanned drawings, PDF exports, or photographs) because structured building information models are frequently unavailable for older properties. Manual measurement and visual inspection of such documents are time consuming and error prone. This paper presents an integrated deep learning pipeline that extracts semantic information from unstructured two-dimensional floor plan images of existing structures and supports preliminary compliance screening via locally deployed large language models. The pipeline employs YOLOv8 for the localization and classification of 18 architectural symbols and furniture items, and a U-Net with a ResNet34 encoder for the semantic segmentation of walls and interior room spaces. To translate pixel-level predictions into physical metrics, we implement an area calculation module based on user-defined reference scale calibration. An LLM evaluation module, deployed locally via Ollama with a retrieval-augmented generation pipeline, interprets extracted room metrics and flags potential non-compliance against referenced residential design guidelines; it is intended for the assessment of existing layouts rather than generative co-design. We expand a core dataset of 101 manually annotated source floor plans to 303 augmented instances using label-aligned geometric transformations, while reporting generalization in terms of the 101 unique source plans. On the held-out validation split (10 source plans), YOLOv8 achieves 92.3% mAP50 versus 87.2% for a Faster R-CNN reference model on the same data split (detection baselines differ in training epochs and pretraining; see Experiments); U-Net achieves 95.71% mIoU, surpassing DeepLabv3+ (93.2%) under matched segmentation training settings. The system is deployed as an interactive web application for legacy building survey and preliminary regulatory review when only two-dimensional documentation is available.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: 3D Surveying and Cultural Heritage · BIM and Construction Integration · Advanced Neural Network Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Applied Sciences, Macao Polytechnic University, Macao, China

References (30)

Show source-provided references
1. buildingSMART International (2026, June 17). Industry Foundation Classes (IFC). Available online: https://www.buildingsmart.org/standards/bsi-standards/industry-foundation-classes/.
2. Macé, S. (2015, January 23–26). Floor Plan Analysis for Room Detection. Proceedings of the 13th International Conference on Document Analysis and Recognition (ICDAR 2015), Nancy, France.
3. Deep Learning10.1038/nature14539 ↗
4. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
5. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
6. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020, January 6–12). Language Models are Few-Shot Learners. Proceedings of the Advances in Neural Information Processing Systems, Virtual.
7. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., and Rocktäschel, T. (2020, January 6–12). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Proceedings of the Advances in Neural Information Processing Systems, Virtual.
8. Streamlit Inc (2026, June 17). Streamlit: The Fastest Way to Build and Share Data Apps. Available online: https://streamlit.io.
9. Jocher, G., Chaurasia, A., and Qiu, J. (2026, June 17). Ultralytics YOLOv8. Available online: https://github.com/ultralytics/ultralytics.
10. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
11. Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Durnov, D., and Kalinin, A. (2020). Albumentations: Fast and Flexible Image Augmentations. Information, 11.10.3390/info11020125 ↗
12. (2026, June 17). Ollama: Get Up and Running with Large Language Models Locally. Available online: https://ollama.com.
13. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
14. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
15. Kalervo, A., Ylioinas, J., Häikiö, M., Karhu, A., and Kannala, J. (2019). CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis. arXiv.10.1007/978-3-030-20205-7_3 ↗
16. Wu, W., Zhang, J., Fu, X., Arnold, M., Shen, Z., Liu, Y., and Zhang, S. (2019). Data-driven Interior Plan Generation for Residential Buildings. ACM Trans. Graph., 38.10.1145/3355089.3356556 ↗
17. Lin, T. (2026, June 17). LabelImg: Graphical Image Annotation Tool. Available online: https://github.com/tzutalin/labelImg.
18. Wada, K. (2026, June 17). Labelme: Image Polygonal Annotation with Python. Available online: https://github.com/wkentaro/labelme.
19. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
20. Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25–28). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.10.1109/3DV.2016.79 ↗
21. Bradski, G. (2000). The OpenCV Library, Dr. Dobb’s Journal of Software Tools.
22. Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., and Huang, F. (2024). Qwen2 Technical Report. arXiv.
23. (2026, June 17). ChromaDB: The Open-Source Embedding Database. Available online: https://www.trychroma.com.
24. (2011). Code for Design of Residential Buildings (Standard No. GB 50096-2011). Clause 5.2.1 Minimum Bedroom Usable Areas (Single ≥ 5 m2, Double ≥ 9 m2).
25. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8–14). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
26. Loshchilov, I., and Hutter, F. (2019, January 6–9). Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
27. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
28. Yakubovskiy, P. (2026, June 17). Segmentation Models PyTorch. Available online: https://github.com/qubvel/segmentation_models.pytorch.
29. Norrby, H., Färm, G., Hernandez-Diaz, K., and Alonso-Fernandez, F. (2025). FGSSNet: Feature-Guided Semantic Segmentation of Real World Floorplans. arXiv.10.1007/978-3-032-11358-0_4 ↗
30. Wei, C., Gupta, M., and Czerniawski, T. (2023). Interoperability between Deep Neural Networks and 3D Architectural Modeling Software: Affordances of Detection and Segmentation. Buildings, 13.10.3390/buildings13092336 ↗