BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review
Source abstract
Abstract
In recent years, the use of digital twins (DT) to improve maintenance procedures has increased in various industrial sectors (e.g., manufacturing, energy industry, aerospace) but is more limited in the construction industry. However, the operation and maintenance (O&M) phase of a building’s life cycle is the most expensive. Smart buildings already use BIM (Building Information Modeling) for facility management, but they lack the predictive capabilities of DT. On the other hand, the use of extended reality (XR) technologies to improve maintenance operations has been a major topic of academic research in recent years, both through data display and remote collaboration. In this context, this paper focuses on reviewing projects using a combination of these technologies to improve maintenance operations in smart buildings. This review uses a combination of at least three of the terms “Digital Twin”, “Maintenance”, “BIM” and “Extended Reality”. Results show how a BIM can be used to create a DT and how this DT use combined with XR technologies can improve maintenance operations in a smart building. This paper also highlights the challenges for the correct implementation of a BIM-based DT combined with XR devices. An example of use is also proposed using a diagram of the possible interactions between the user, the DT and the application framework during maintenance operations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: BIM and Construction Integration · Digital Transformation in Industry · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
LARIS, SFR MATHSTIC, Univ. Angers, F-49000 Angers, France
LINEACT, CESI, 44 Avenue Frédéric Auguste Bartholdi, 72000 Le Mans, France
LINEACT, CESI, 80 rue Edmund Halley, 76808 Saint-Étienne-du-Rouvray, France
References (97)
Show source-provided references
1. Wahlster, W., Lukas, L.-D., and Kagermann, H. (2021, May 13). Industrie 4.0: Mit Dem Internet der Dinge auf dem Weg zur 4. Industriellen Revolution—Ingenieur.de. Available online: https://www.ingenieur.de/technik/fachbereiche/produktion/industrie-40-mit-internet-dinge-weg-4-industriellen-revolution/.
2. Oliveira, M., and Afonso, D. (2019). Industry Focused in Data Collection: How Industry 4.0 Is Handled by Big Data. ACM Int. Conf. Proc. Ser., 12–18.10.1145/3352411.3352414 ↗
3. (2021, February 08). McKinsey Digital America Full Report December 2015. Pdf. Available online: https://www.mckinsey.com/~/media/McKinsey/Industries/Technology%20Media%20and%20Telecommunications/High%20Tech/Our%20Insights/Digital%20America%20A%20tale%20of%20the%20haves%20and%20have%20mores/Digital%20America%20Full%20Report%20December%202015.pdf.
4. Schober, K.-S., and Hoff, P. (2016). Digitization in the Construction Industry: Building Europe’s Road to “Construction 4.0.”, Roland Berg. GMBH.
5. (2020, October 21). Gartner Top 10 Strategic Technology Trends for 2020. Available online: //www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2020/.
6. (2020, October 21). Gartner Top 10 Strategic Technology Trends for 2019. Available online: //www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2019/.
7. Augmented Reality Application to Support Remote Maintenance as a Service in the Robotics Industry10.1016/j.procir.2017.03.154 ↗
8. A Classification Proposal of Digital Twin Applications in the Safety Domain10.1016/j.cie.2021.107137 ↗
9. Maintenance 4.0 Technologies—new opportunities for sustainability driven maintenance
10. (2021, February 08). McKinsey Unlocking the Potential of the Internet of Things | McKinsey. Available online: https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/the-internet-of-things-the-value-of-digitizing-the-physical-world.
11. Eastman, C., Fisher, D., Lafue, G., Lividini, J., Stoker, D., and Yessios, C. (1974). An Outline of the Building Description System. University of Pittsburgh Carnegie-Mellon. Research Report No.50.
12. Design and Development of BIM Models to Support Operations and Maintenance10.1016/j.compind.2019.08.001 ↗
13. A Knowledge-Based BIM System for Building Maintenance10.1016/j.autcon.2012.09.008 ↗
14. Case-Based Reasoning and BIM Systems for Asset Management10.1108/BEPAM-02-2014-0006 ↗
15. Data-Driven Predictive Maintenance Planning Framework for MEP Components Based on BIM and IoT Using Machine Learning Algorithms10.1016/j.autcon.2020.103087 ↗
16. Designing a Unique Digital Twin for Linear Infrastructures Lifecycle Management10.1016/j.procir.2019.04.176 ↗
17. Kahlen, F.-J., Flumerfelt, S., and Alves, A. (2017). Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems. Transdisciplinary Perspectives on Complex Systems, Springer International Publishing.10.1007/978-3-319-38756-7 ↗
18. Glaessgen, E.H., and Stargel, D.S. (2012). The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles. Struct. Dyn., 14.10.2514/6.2012-1818 ↗
19. Digital Twin for Maintenance: A Literature Review10.1016/j.compind.2020.103316 ↗
20. Singh, M., Fuenmayor, E., Hinchy, E.P., Qiao, Y., Murray, N., and Devine, D. (2021). Digital Twin: Origin to Future. ASI, 4.10.3390/asi4020036 ↗
21. Digital Twin in Manufacturing: A Categorical Literature Review and Classification
22. Digital Twin as a Service (DTaaS) in Industry 4.0: An Architecture Reference Model10.1016/j.aei.2020.101225 ↗
23. Augmented Reality: An Application of Heads-up Display Technology to Manual Manufacturing Processes10.1109/HICSS.1992.183317 ↗
24. A Taxonomy of Mixed Reality Visual Displays
25. Prouzeau, A., Wang, Y., Ens, B., Willett, W., and Dwyer, T. (2020, January 24). Corsican Twin: Authoring In Situ Augmented Reality Visualisations in Virtual Reality. Proceedings of the International Conference on Advanced Visual Interfaces, Island of Ischia, Italy.10.1145/3399715.3399743 ↗
26. Autiosalo, J., Ala-Laurinaho, R., Mattila, J., Valtonen, M., Peltoranta, V., and Tammi, K. (2021). Towards Integrated Digital Twins for Industrial Products: Case Study on an Overhead Crane. Appl. Sci., 11.10.3390/app11020683 ↗
27. Derigent, W., Mcfarlane, D., and BRIL EL-HAOUZI, H. (2020, January 21). Intelligent Products through a SOHOMA Prism. Proceedings of the 10th Workshop on Service Oriented, Holonic and Multi-Agent Manufacturing Systems for Industry of the Future, SOHOMA 2020, Paris, France.10.1007/978-3-030-69373-2_26 ↗
28. Bolshakov, N., Badenko, V., Yadykin, V., Celani, A., and Fedotov, A. (2020). Digital Twins of Complex Technical Systems for Management of Built Environment. IOP Conf. Ser. Mater. Sci. Eng., 869.10.1088/1757-899X/869/6/062045 ↗
29. Jakobi, M., Kunath, L., and Witzig, A. (2018, January 10–13). BIM Use-Case: Model-Based Performance Optimization. Proceedings of the Ises Eurosun 2018 Conference—12th International Conference on Solar Energy for Buildings and Industry, Rapperswil, Switzerland.10.18086/eurosun2018.11.07 ↗
30. Integration of 3D Objects and Terrain for 3D Modelling Supporting the Digital Twin10.5194/isprs-annals-IV-4-W8-147-2019 ↗
31. Cureton, P., and Dunn, N. (2021). Digital Twins of Cities and Evasive Futures. Shaping Smart for Better Cities, Academic Press.10.1016/B978-0-12-818636-7.00017-2 ↗
32. Digital Twin Aided Sustainability-Based Lifecycle Management for Railway Turnout Systems10.1016/j.jclepro.2019.04.156 ↗
33. From BIM Towards Digital Twin: Strategy and Future Development for Smart Asset Management10.1007/978-3-030-27477-1_30 ↗
34. Moretti, N., Xie, X., Merino, J., Brazauskas, J., and Parlikad, A.K. (2020). An OpenBIM Approach to IoT Integration with Incomplete As-Built Data. Appl. Sci., 10.10.3390/app10228287 ↗
35. Digital Twin Hospital Buildings: An Exemplary Case Study through Continuous Lifecycle Integration10.1155/2020/8846667 ↗
36. BIM Technology and Changes in Traditional Design Process, Reliability of Data from Related Registers10.1088/1757-899X/960/3/032049 ↗
37. Digital Twin for Sustainability Evaluation of Railway Station Buildings10.3389/fbuil.2018.00077 ↗
38. Kaewunruen, S., Peng, S., and Phil-Ebosie, O. (2020). Digital Twin Aided Sustainability and Vulnerability Audit for Subway Stations. Sustainability, 12.10.3390/su12197873 ↗
39. Angjeliu, G., Coronelli, D., and Cardani, G. (2020). Development of the Simulation Model for Digital Twin Applications in Historical Masonry Buildings: The Integration between Numerical and Experimental Reality. Comput. Struct., 238.10.1016/j.compstruc.2020.106282 ↗
40. Digital Twin: Vision, Benefits, Boundaries, and Creation for Buildings10.1109/ACCESS.2019.2946515 ↗
41. Digital Twin Development for Airport Management
42. From Geological Mapping to Representation in BIM10.1002/geot.201900076 ↗
43. Visualised Inspection System for Monitoring Environmental Anomalies during Daily Operation and Maintenance10.1108/ECAM-11-2019-0640 ↗
44. Sofia, H., Anas, E., and Faiz, O. (2020, January 11–13). Mobile Mapping, Machine Learning and Digital Twin for Road Infrastructure Monitoring and Maintenance: Case Study of Mohammed VI Bridge in Morocco. Proceedings of the 2020 IEEE International Conference of Moroccan Geomatics, MORGEO, Casablance, Morocco.10.1109/Morgeo49228.2020.9121882 ↗
45. Nahangi, M., and Kim, M. (2019). Use Cases for Subcontractors and Fabricators. Infrastruct. Comput. Vis., 315–361.10.1016/B978-0-12-815503-5.00007-3 ↗
46. SmartBridge Hamburg—Prototypische Pilotierung Eines Digitalen Zwillings10.1002/bate.201900108 ↗
47. Joy, D., and Smith, D. (2019). Processing Asset Data at the Intelligent Edge: Implementation of an Industrial IoT Architecture to Drive Business Value. Proc. SPE Annu. Tech. Conf. Exhib., 2019.10.2118/195830-MS ↗
48. Lu, Q., Xie, X., Parlikad, A.K., and Schooling, J.M. (2020). Digital Twin-Enabled Anomaly Detection for Built Asset Monitoring in Operation and Maintenance. Autom. Constr., 118.10.1016/j.autcon.2020.103277 ↗
49. Lu, Q., Parlikad, A.K., Woodall, P., Ranasinghe, G.D., and Heaton, J. (2019). Developing a Dynamic Digital Twin at a Building Level: Using Cambridge Campus as Case Study. Proceedings of the International Conference on Smart Infrastructure and Construction 2019 (ICSIC), ICE Publishing.
50. Daniotti, B., Gianinetto, M., and Della Torre, S. (2020). The Effect of Real-Time Sensing of a Window on Energy Efficiency, Comfort, Health and User Behavior. Digital Transformation of the Design, Construction and Management Processes of the Built Environment, Springer International Publishing. Research for Development.10.1007/978-3-030-33570-0 ↗
51. Varé, C., and Morilhat, P. (2020). Digital Twins, a New Step for Long Term Operation of Nuclear Power Plants. Lect. Notes Mech. Eng., 96–103.10.1007/978-3-030-48021-9_11 ↗
52. Lu, Q., Parlikad, A.K., Woodall, P., Ranasinghe, G.D., Xie, X., Liang, Z., Konstantinou, E., Heaton, J., and Schooling, J. (2020). Developing a Digital Twin at Building and City Levels: Case Study of West Cambridge Campus. J. Manag. Eng., 36.10.1061/(ASCE)ME.1943-5479.0000763 ↗
53. Ubiquitous Knowledge Empowers the Smart Factory: The Impacts of a Service-Oriented Digital Twin on Enterprises’ Performance10.1016/j.arcontrol.2019.01.001 ↗
54. Office Building Occupancy Monitoring through Image Recognition Sensors
55. Augmented Reality Assisted Calibration of Digital Twins of Mobile Robots
56. Anderson, S., and Sigve, B. (2020, January 13–15). Advanced Offshore Digital Inspection Methods. Proceedings of the International Petroleum Technology Conference, IPTC, Dhahran, Saudi Arabia.10.2523/IPTC-20087-Abstract ↗
57. Bridge Assessment for PSC Girder Bridge Using Digital Twins Model10.1007/978-981-15-0802-8_199 ↗
58. Shim, C., Kang, H., and Dang, N. (2019, January 8–10). Digital Twin Models for Maintenance of Cable-Supported Bridges. Proceedings of the International Conference on Smart Infrastructure and Construction 2019 (ICSIS), Churchill College, Cambridge, UK.10.1680/icsic.64669.737 ↗
59. Development of a Bridge Maintenance System for Prestressed Concrete Bridges Using 3D Digital Twin Model10.1080/15732479.2019.1620789 ↗
60. Grosse, C.U. (2019, January 14–17). Monitoring and Inspection Techniques Supporting a Digital Twin Concept in Civil Engineering. Proceedings of the 5th International Conference on Sustainable Construction Materials and Technologies (SCMT5), London, UK.
61. Tahmasebinia, F., Fogerty, D., Wu, L.O., Li, Z., Sepasgozar, S.M.E., Zhang, K., Sepasgozar, S., and Marroquin, F.A. (2019). Numerical Analysis of the Creep and Shrinkage Experienced in the Sydney Opera House and the Rise of Digital Twin as Future Monitoring Technology. Buildings, 9.10.3390/buildings9060137 ↗
62. Improving Building Energy Footprint and Asset Performance Using Digital Twin Technology
63. Prediction of Highway Tunnel Pavement Performance Based on Digital Twin and Multiple Time Series Stacking10.1155/2020/8824135 ↗
64. Bevilacqua, M., Bottani, E., Ciarapica, F.E., Costantino, F., Donato, L.D., Ferraro, A., Mazzuto, G., Monteriu, A., Nardini, G., and Ortenzi, M. (2020). Digital Twin Reference Model Development to Prevent Operators’ Risk in Process Plants. Sustainability, 12.10.3390/su12031088 ↗
65. BIM and AM to Manage Critical and Relevant Water and Wastewater Utilities Assets10.1007/978-3-030-51295-8_49 ↗
66. System-Based Concept for a Mixed Reality Supported Maintenance Phase of an Industrial Plant10.1016/j.procir.2020.03.096 ↗
67. Zhao, X., and Sun, Y. (2020, January 19–21). Augmented Reality Assembly Guidance Method Based on Situation Awareness. Proceedings of the 2020 the 10th International Workshop on Computer Science and Engineering, WCSE 2020, Shanghai, China.
68. IIoT-Based Fatigue Life Indication Using Augmented Reality
69. Towards Improving the Future of Manufacturing through Digital Twin and Augmented Reality Technologies10.1016/j.promfg.2018.10.070 ↗
70. Smart Industry Testbed10.1016/j.procir.2019.04.215 ↗
71. Corneli, A., Naticchia, B., Cabonari, A., and Bosché, F. (2019, January 21–24). Augmented Reality and Deep Learning towards the Management of Secondary Building Assets. Proceedings of the 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, AB, Canada.10.22260/ISARC2019/0045 ↗
72. Utzig, S., Kaps, R., Azeem, S.M., and Gerndt, A. (2019). Augmented Reality for Remote Collaboration in Aircraft Maintenance Tasks. Proceedings of the 2019 IEEE Aerospace Conference, IEEE.10.1109/AERO.2019.8742228 ↗
73. van Lopik, K., Sinclair, M., Sharpe, R., and Conway, P. (2020). Developing Augmented Reality Capabilities for Industry 4.0 Small Enterprises: Lessons Learnt from a Content Authoring Case Study. Comput. Ind.10.1016/j.compind.2020.103208 ↗
74. Pileggi, P., Lazovik, E., Broekhuijsen, J., Borth, M., and Verriet, J. (2020). Lifecycle Governance for Effective Digital Twins: A Joint Systems Engineering and IT Perspective. 2020 IEEE International Systems Conference (SysCon), Montreal, QC, Canada 24 August to 20 September, 8.10.1109/SysCon47679.2020.9275662 ↗
75. A Enhanced Interaction Framework Based on VR, AR and MR in Digital Twin10.1016/j.procir.2019.04.103 ↗
76. Prajapat, N., Tiwari, A., Tiwari, D., Turner, C., and Hutabarat, W. (2019, January 22–25). A Framework for Next Generation Interactive and Immersive DES Models. Proceedings of the 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), Helsinki, Finland.10.1109/INDIN41052.2019.8972266 ↗
77. Dezen-Kempter, E., Mezencio, D.L., Miranda, E.D.M., Pico de Sa, D., and Dias, U. (2020, January 5–6). Towards a Digital Twin for Heritage Interpretation. Proceedings of the RE: Anthropocene, Design in the Age of Humans—Proceedings of the 25th International Conference on Computer-Aided Architectural Design Research in Asia, CAADRIA 2020, Bangkok, Thailand.10.52842/conf.caadria.2020.2.183 ↗
78. Novack, J. (2019, January 11–14). Digital Twins and Industry 4.0: Videogamers Will Staff and Manage Industrial Projects in the near Future. Proceedings of the Society of Petroleum Engineers—Abu Dhabi International Petroleum Exhibition and Conference 2019, ADIP, Abu Dhabi, United Arab Emirates.10.2118/197538-MS ↗
79. Use of Gaming Technology to Bring Bridge Inspection to the Office10.1080/15732479.2019.1615962 ↗
80. Visualisation of the Digital Twin Data in Manufacturing by Using Augmented Reality10.1016/j.procir.2019.03.223 ↗
81. Stojanovic, V., Trapp, M., Richter, R., Hagedorn, B., and Döllner, J. (2018, January 3–5). Towards the Generation of Digital Twins for Facility Management Based on 3D Point Clouds. Proceedings of the 34th Annual ARCOM Conference, ARCOM, Belfast, UK.
82. (2021, April 22). Microsoft HoloLens | Mixed Reality Technology for Business. Available online: https://www.microsoft.com/en-us/hololens.
83. (2021, June 28). VIVE Pro Eye Office | VIVE Business United States. Available online: https://business.vive.com/us/product/vive-pro-eye-office/.
84. Joao, D.V., Lodetti, P.Z., Martins, M.A.I., and Almeida, J.F.B. (2020, January 3–6). Virtual and Augmented Reality Applied in Power Electric Utilities for Human Interface Improvement-A Study Case for Best Practices. Proceedings of the 2020 IEEE Technology and Engineering Management Conference, TEMSCON, Novi, MI, USA.10.1109/TEMSCON47658.2020.9140129 ↗
85. (2021, June 03). GPS.Gov: GPS Accuracy, Available online: https://www.gps.gov/systems/gps/performance/accuracy/.
86. Design and Management of Digital Manufacturing and Assembly Systems in the Industry 4.0 Era10.1007/s00170-019-04595-0 ↗
87. Iung, B. (2019, January 22–27). De La Maintenance Prédictive à La Maintenance Prescriptive: Une Évolution Nécessaire Pour l’industrie Du Futur. Proceedings of the Conference on Complexity Analysis of Industrial Systems and Advanced Modeling, CAISAM 2019, Ben Guerir, Morocco.
88. (2020, October 21). Gartner Top 10 Strategic Technology Trends for 2018. Available online: //www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2018/.
89. Agnusdei, G.P., Elia, V., and Gnoni, M.G. (2021). Is Digital Twin Technology Supporting Safety Management? A Bibliometric and Systematic Review. Appl. Sci., 11.10.3390/app11062767 ↗
90. The ‘How’ of Benefits Management for Digital Technology: From Engineering to Asset Management10.1016/j.autcon.2019.102930 ↗
91. Operator 4.0—Emerging Job Categories in Manufacturing10.1007/978-981-13-2375-1_16 ↗
92. Future Maintenance: Transitioning from Digitalisation to Industry 4.0
93. Nyvlt, V. (2020). BIM within Current Building Facilities and Infrastructure. IOP Conf. Ser. Mater. Sci. Eng., 972.10.1088/1757-899X/972/1/012040 ↗
94. Longo, F., Padovano, A., and Umbrello, S. (2020). Value-Oriented and Ethical Technology Engineering in Industry 5.0: A Human-Centric Perspective for the Design of the Factory of the Future. Appl. Sci., 10.10.3390/app10124182 ↗
95. Havard, V., Jeanne, B., Savatier, X., and Baudry, D. (2017, January 5–7). Inoovas—Industrial Ontology for Operation in Virtual and Augmented Scene: The Architecture. Proceedings of the 2017 4th International Conference on Control, Decision and Information Technologies (CoDIT), Barcelona, Spain.10.1109/CoDIT.2017.8102608 ↗
96. Blockchain for Industry 4.0: A Comprehensive Review10.1109/ACCESS.2020.2988579 ↗
97. (2020, October 21). Gartner Top Strategic Technology Trends for 2021. Available online: //www.gartner.com/smarterwithgartner/gartner-top-strategic-technology-trends-for-2021/.