Augmented Reality in Surgery: A Scoping Review
Source abstract
Abstract
Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Augmented Reality Applications · Surgical Simulation and Training · Anatomy and Medical Technology
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Engineering, University of Florence, 50139 Florence, Italy
Department of Biomedical Experimental and Clinical Sciences “Mario Serio”, University of Florence, 50139 Florence, Italy
Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
Epica Imaginalis, 50019 Sesto Fiorentino, Italy
Funding
Fondazione Cassa di Risparmio di Firenze, Florence, Italy
References (97)
Show source-provided references
1. Augmented and Mixed Reality: Technologies for Enhancing the Future of IR10.1016/j.jvir.2019.09.020 ↗
2. X-ray computed tomography: From medical imaging to dimensional metrology10.1016/j.precisioneng.2019.06.007 ↗
3. Lee, N. (2017). Augmented Reality in Image-Guided Surgery. Encyclopedia of Computer Graphics and Games, Springer International Publishing.
4. Allison, B., Ye, X., and Janan, F. (2020). MIXR: A Standard Architecture for Medical Image Analysis in Augmented and Mixed Reality, IEEE Computer Society.10.1109/AIVR50618.2020.00053 ↗
5. An augmented reality system for image-guided surgery: This article is derived from a previous article published in the journal International Congress Series10.1016/j.ijom.2005.05.004 ↗
6. Image-guidance for surgical procedures10.1088/0031-9155/51/14/R01 ↗
7. Augmented Reality in Medicine: Systematic and Bibliographic Review
8. Virtual Reality and Augmented Reality in Plastic Surgery: A Review10.5999/aps.2017.44.3.179 ↗
9. Augmented and Virtual Reality in Surgery10.1109/MCSE.2020.2972822 ↗
10. Image Overlay Surgery Based on Augmented Reality: A Systematic Review
11. A Survey of Indoor Localization Systems and Technologies10.1109/COMST.2019.2911558 ↗
12. Cheng, J., Chen, K., and Chen, W. (2017, January 4–7). Comparison of marker-based AR and markerless AR: A case study on indoor decoration system. Proceedings of the Lean and Computing in Construction Congress (LC3): Proceedings of the Joint Conference on Computing in Construction (JC3), Heraklion, Greece.10.24928/JC3-2017/0231 ↗
13. Thangarajah, A., Wu, J., Madon, B., and Chowdhury, A.K. (2015, January 19–21). Vision-based registration for augmented reality-a short survey. Proceedings of the 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia.10.1109/ICSIPA.2015.7412236 ↗
14. Virtual and Augmented Reality in Oncologic Liver Surgery10.1016/j.soc.2018.08.002 ↗
15. Calibration requirements and procedures for a monitor-based augmented reality system10.1109/2945.466720 ↗
16. Overview of Navigation Systems in Image-Guided Interventions10.1053/j.tvir.2013.02.008 ↗
17. Zhanat, M., and Vslor, H.A. (2020). Augmented Reality for Robotics: A Review. Robotics, 9.10.3390/robotics9020021 ↗
18. Intelligent Predictive Maintenance and Remote Monitoring Framework for Industrial Equipment Based on Mixed Reality10.3389/fmech.2020.578379 ↗
19. A Survey of Visual, Mixed, and Augmented Reality Gaming
20. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation10.7326/M18-0850 ↗
21. The utility of augmented reality in lateral skull base surgery: A preliminary report10.1016/j.amjoto.2021.102942 ↗
22. The Potential Applications of Augmented Reality in Fetoscopic Surgery for Antenatal Treatment of Myelomeningocele10.1016/j.wneu.2021.11.133 ↗
23. Breast cancer surgery with augmented reality10.1016/j.breast.2021.01.004 ↗
24. Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy10.1016/j.cmpb.2021.105952 ↗
25. Augmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System10.1007/s11605-020-04519-4 ↗
26. Augmented Reality for Head and Neck Carcinoma Imaging: Description and Feasibility of an Instant Calibration, Markerless Approach10.1016/j.cmpb.2020.105854 ↗
27. Clinical Accuracy, Technical Precision, and Workflow of the First in Human Use of an Augmented-Reality Head-Mounted Display Stereotactic Navigation System for Spine Surgery10.1093/ons/opaa398 ↗
28. Ackermann, J., Florentin, L., Armando, H., Jess, S., Mazda, F., Stefan, R., Patrick, Z., and Furnstahl, P. (2021). Augmented Reality Based Surgical Navigation of Complex Pelvic Osteotomies—A Feasibility Study on Cadavers. Appl. Sci., 11.10.3390/app11031228 ↗
29. A Wearable Augmented Reality Navigation System for Surgical Telementoring Based on Microsoft HoloLens10.1007/s10439-020-02538-5 ↗
30. Augmented Reality Guided Laparoscopic Surgery of the Uterus10.1109/TMI.2020.3027442 ↗
31. Design, implementation, and metrological characterization of a wearable, integrated AR-BCI hands-free system for health 4.0 monitoring10.1016/j.measurement.2021.109280 ↗
32. A novel enhanced energy function using augmented reality for a bowel: Modified region and weighted factor10.1007/s11042-021-10606-8 ↗
33. Wei, W., Ho, E., McCay, K., Damasevicius, R., Maskeliunas, R., and Esposito, A. (2021). Assessing Facial Symmetry and Attractiveness using Augmented Reality. Pattern Anal. Appl., 1–17.
34. Vision-based tracking system for augmented reality to localize recurrent laryngeal nerve during robotic thyroid surgery10.1038/s41598-020-65439-6 ↗
35. Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery10.1038/s41598-020-63839-2 ↗
36. Ultrasound in augmented reality: A mixed-methods evaluation of head-mounted displays in image-guided interventions10.1007/s11548-020-02236-6 ↗
37. Indocyanine green angiography visualized by augmented reality in aneurysm surgery10.1016/j.wneu.2020.06.219 ↗
38. Augmented reality for image guidance in transoral robotic surgery10.1007/s11701-019-01030-0 ↗
39. Augmented Reality and Robotic-Assistance for Percutaneous Nephrolithotomy10.1109/LRA.2020.3002216 ↗
40. Augmented reality and artificial intelligence-based navigation during percutaneous vertebroplasty: A pilot randomised clinical trial10.1007/s00586-019-06054-6 ↗
41. Use of Augmented Reasameility During Inhaled Induction of General Anesthesia in 3 Pediatric Patients: A Case Report10.1213/XAA.0000000000001219 ↗
42. Supporting Fibula Free Flap Harvest With Augmented Reality: A Proof-of-Concept Study10.1002/lary.28090 ↗
43. HoloLens-Based Vascular Localization System: Precision Evaluation Study With a Three-Dimensional Printed Model10.2196/16852 ↗
44. A partial augmented reality system with live ultrasound and registered preoperative MRI for guiding robot-assisted radical prostatectomy10.1016/j.media.2019.101588 ↗
45. Development of augmented-reality applications in otolaryngology-head and neck surgery: Augmented Reality Applications10.1002/lary.28098 ↗
46. Augmented reality in transsphenoidal surgery10.1016/j.wneu.2019.01.202 ↗
47. A novel augmented reality visualization in jaw surgery: Enhanced ICP based modified rotation invariant and modified correntropy10.1007/s11042-021-10787-2 ↗
48. Innovative Surgical Planning in Resecting Soft Tissue Sarcoma of the Foot Using Augmented Reality With a Smartphone10.1053/j.jfas.2020.03.011 ↗
49. Upbeat: Augmented Reality-Guided Dancing for Prosthetic Rehabilitation of Upper Limb Amputees10.1155/2019/2163705 ↗
50. Augmented Reality Based Navigation for Distal Interlocking of Intramedullary Nails Utilizing Microsoft HoloLens 210.1016/j.compbiomed.2021.104402 ↗
51. Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance10.3389/fsurg.2021.657901 ↗
52. HoloPointer: A virtual augmented reality pointer for laparoscopic surgery training10.1007/s11548-020-02272-2 ↗
53. Brookes, M.J., Chan, C.D., Baljer, B., Wimalagunaratna, S., Crowley, T.P., Ragbir, M., Irwin, A., Gamie, Z., Beckingsale, T., and Ghosh, K.M. (2021). Surgical Advances in Osteosarcoma. Cancers, 13.10.3390/cancers13030388 ↗
54. Three-dimensional virtual surgical planning in the oncologic treatment of the mandible10.1111/odi.13631 ↗
55. 3D Printing, Augmented Reality, and Virtual Reality for the Assessment and Management of Kidney and Prostate Cancer: A Systematic Review10.1016/j.urology.2020.03.066 ↗
56. Augmented reality in intracranial meningioma surgery: A case report and systematic review
57. Virtual reality and augmented reality in the management of intracranial tumors: A review10.1016/j.jocn.2018.12.036 ↗
58. Brain shift in neuronavigation of brain tumors: A review10.1016/j.media.2016.08.007 ↗
59. Preliminary study on the clinical application of augmented reality neuronavigation10.1055/s-0032-1333415 ↗
60. Augmented reality-guided neurosurgery: Accuracy and intraoperative application of an image projection technique10.3171/2014.9.JNS141001 ↗
61. Augmented reality-assisted skull base surgery10.1016/j.neuchi.2014.07.001 ↗
62. Intraoperative clinical application of augmented reality in neurosurgery: A systematic review10.1016/j.clineuro.2018.11.018 ↗
63. A Smart IoT Platform for Oncology Patient Diagnosis based on AI: Towards the Human Digital Twin10.1016/j.procir.2021.11.284 ↗
64. Augmented Reality in Orthopedic Surgery Is Emerging from Proof of Concept Towards Clinical Studies: A Literature Review Explaining the Technology and Current State of the Art10.1007/s12178-021-09699-3 ↗
65. Evolution of augmented reality applications in Orthopaedics: A systematic review10.1016/j.jajs.2021.02.006 ↗
66. The role of virtual and augmented reality in orthopedic trauma surgery: From diagnosis to rehabilitation10.1016/j.cmpb.2020.105407 ↗
67. Jud, L., Fotouhi, J., Andronic, O., Aichmair, A., Osgood, G., Navab, N., and Farshad, M. (2020). Applicability of augmented reality in orthopedic surgery—A systematic review. BMC Musculoskelet. Disord., 21.10.1186/s12891-020-3110-2 ↗
68. 151. A cadaveric precision and accuracy analysis of augmented reality mediated percutaneous pedicle implant insertion10.1016/j.spinee.2020.05.557 ↗
69. Augmented reality navigation in spine surgery: A systematic review10.1007/s00701-021-04708-3 ↗
70. Current innovation in virtual and augmented reality in spine surgery10.21037/atm-20-1132 ↗
71. Augmented, virtual and mixed reality in spinal surgery: A real-world experience
72. Robotic Spine Surgery and Augmented Reality Systems: A State of the Art10.14245/ns.2040060.030 ↗
73. Augmented reality in neurosurgical navigation: A survey
74. Easy-to-Use Augmented Reality Neuronavigation Using a Wireless Tablet PC10.1159/000354816 ↗
75. BrainLab VectorVision Neuronavigation System: Technology and clinical experiences in 131 cases10.1097/00006123-199901000-00056 ↗
76. Computer-aided navigation in neurosurgery10.1007/s10143-003-0262-0 ↗
77. Image-Guided Interventions: Technology Review and Clinical Applications10.1146/annurev-bioeng-070909-105249 ↗
78. Clinical Feasibility of a Wearable Mixed-Reality Device in Neurosurgery10.1016/j.wneu.2018.06.208 ↗
79. HoloLens and mobile augmented reality in medical and health science education: A randomised controlled trial10.1111/bjet.13049 ↗
80. A Novel Three-Dimensional Interactive Virtual Face to Facilitate Facial Anatomy Teaching Using Microsoft HoloLens10.1007/s00266-020-02110-5 ↗
81. Using Holograms to Enhance Learning in Health Sciences and Medicine10.1007/s40670-020-01051-7 ↗
82. Current Perspectives on Augmented Reality in Medical Education: Applications, Affordances and Limitations10.2147/AMEP.S249891 ↗
83. Augmented reality in surgical training: A systematic review10.1136/postgradmedj-2020-137600 ↗
84. Virtual or Augmented Reality to Enhance Surgical Education and Surgical Planning10.1016/j.thorsurg.2019.03.010 ↗
85. Virtual and Augmented Reality Applications in Medicine: Analysis of the Scientific Literature10.2196/25499 ↗
86. Virtual Reality and Augmented Reality—Translating Surgical Training into Surgical Technique10.1007/s12178-020-09667-3 ↗
87. Exposure to low-dose ionizing radiation from medical imaging procedures10.1056/NEJMoa0901249 ↗
88. Association of exposure to diagnostic low-dose ionizing radiation with risk of cancer among youths in South Korea10.1001/jamanetworkopen.2019.10584 ↗
89. Effects of ionizing radiation on biological molecules–mechanisms of damage and emerging methods of detection10.1089/ars.2013.5489 ↗
90. Minimum redundancy and maximum relevance feature selection and recent advances in cancer classification
91. A Report on Registration Problems in Augmented Reality
92. An overview of augmented reality technology10.1088/1742-6596/1237/2/022082 ↗
93. Prospects and challenges in augmented reality displays10.3724/SP.J.2096-5796.2018.0009 ↗
94. Ren, D., Goldschwendt, T., Chang, Y., and Höllerer, T. (2016, January 19–23). Evaluating wide-field-of-view augmented reality with mixed reality simulation. Proceedings of the 2016 IEEE Virtual Reality (VR), Greenville, SC, USA.10.1109/VR.2016.7504692 ↗
95. Vergence-accommodation conflict in optical see-through display: Review and prospect10.1016/j.rio.2021.100160 ↗
96. 19-2: Vergence-Accommodation Conflicts in Augmented Reality: Impacts on Perceived Image Quality10.1002/sdtp.13855 ↗
97. The rate of change of vergence–Accommodation conflict affects visual discomfort10.1016/j.visres.2014.10.021 ↗