Augmented Reality in Surgical Navigation: A Review of Evaluation and Validation Metrics
Source abstract
Abstract
Pre-operative imaging has been used earlier to guide traditional surgical navigation systems. There has been a lot of effort in the last decade to integrate augmented reality into the operating room to help surgeons intra-operatively. An augmented reality (AR) based navigation system provides a clear three-dimensional picture of the interested areas over the patient to aid surgical navigation and operations, which is a promising approach. The goal of this study is to review the application of AR technology in various fields of surgery and how the technology is used for its performance in each field. Assessment of the available AR assisted navigation systems being used for surgery is reviewed in this paper. Furthermore, a discussion about the required evaluation and validation metric for these systems is also presented. The paper comprehensively reviews the literature since the year 2008 for providing relevant information on applying the AR technology for training, planning and surgical navigation. It also describes the limitations which need to be addressed before one can completely rely on this technology for surgery. Thus, additional research is desirable in this emerging field, particularly to evaluate and validate the use of AR technology for surgical navigation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Augmented Reality Applications · Surgical Simulation and Training · Anatomy and Medical Technology
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Qatar University, Doha 2713, Qatar
Department of Surgery, Hamad Medical Corporation (HMC), Doha 3050, Qatar
Funding
Qatar National Research Fund
References (100+)
Show source-provided references
1. Image-guided interventions: Technology review and clinical applications10.1146/annurev-bioeng-070909-105249 ↗
2. Navigation in surgery10.1007/s00423-013-1059-4 ↗
3. A survey of augmented reality10.1162/pres.1997.6.4.355 ↗
4. Advanced medical displays: A literature review of augmented reality10.1109/JDT.2008.2001575 ↗
5. Comprehensible visualization for augmented reality10.1109/TVCG.2008.96 ↗
6. Augmented reality in surgery10.1001/archsurg.139.2.170 ↗
7. Virtual reality applied to hepatic surgery simulation: The next revolution10.1097/00000658-199811000-00001 ↗
8. Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation10.1016/j.compmedimag.2014.11.003 ↗
9. Augmented reality in neurosurgery: A systematic review10.1007/s10143-016-0732-9 ↗
10. Augmented reality in neurosurgery10.5114/aoms.2016.58690 ↗
11. Si, W., Liao, X., Wang, Q., and Heng, P.A. (2018, January 18–22). Augmented reality-based personalized virtual operative anatomy for neurosurgical guidance and training. Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen/Reutlingen, Germany.10.1109/VR.2018.8446450 ↗
12. Lee, J.D., Wu, H.K., and Wu, C.T. (2018, January 9–12). A projection-based AR system to display brain angiography via stereo vision. Proceedings of the 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), Nara, Japan.10.1109/GCCE.2018.8574761 ↗
13. Shirai, R., Chen, X., Sase, K., Komizunai, S., Tsujita, T., and Konno, A. (2019, January 10–13). AR brain-shift display for computer-assisted neurosurgery. Proceedings of the 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Hiroshima, Japan.10.23919/SICE.2019.8859884 ↗
14. Fallavollita, P., Wang, L., Weidert, S., and Navab, N. (2016). Computational Radiology for Orthopaedic Interventions, Springer.
15. Zhang, X., Fan, Z., Wang, J., and Liao, H. (2016). Computational Radiology for Orthopaedic Interventions, Springer.
16. Worldwide survey on the use of navigation in spine surgery10.1016/j.wneu.2012.03.011 ↗
17. Augmented reality in laparoscopic surgical oncology10.1016/j.suronc.2011.07.002 ↗
18. The status of augmented reality in laparoscopic surgery as of 201610.1016/j.media.2017.01.007 ↗
19. Image-guided laparoscopic surgery in an open MRI operating theater10.1007/s00464-012-2737-y ↗
20. Trans-thoracic minimally invasive liver resection guided by augmented reality10.1016/j.jamcollsurg.2014.12.053 ↗
21. Augmented reality technology for preoperative planning and intraoperative navigation during hepatobiliary surgery: A review of current methods10.1016/j.hbpd.2018.02.002 ↗
22. Design of navigation system for liver surgery guided by augmented reality10.1109/ACCESS.2020.3008690 ↗
23. Navigated liver surgery: State of the art and future perspectives10.1016/j.hbpd.2021.09.002 ↗
24. Utility of augmented reality system in hepatobiliary surgery10.1007/s00534-012-0504-z ↗
25. Video see-through augmented reality for oral and maxillofacial surgery10.1002/rcs.1754 ↗
26. Registration technology of augmented reality in oral medicine: A review10.1109/ACCESS.2019.2912949 ↗
27. Clinical application of navigation surgery using augmented reality in the abdominal field10.1007/s00595-014-0946-9 ↗
28. Camera augmented mobile C-arm (CAMC): Calibration, accuracy study, and clinical applications10.1109/TMI.2009.2021947 ↗
29. Camera-augmented mobile C-arm (CamC): A feasibility study of augmented reality imaging in the operating room10.1002/rcs.1885 ↗
30. Multimodal mixed reality visualisation for intraoperative surgical guidance10.1007/s11548-020-02165-4 ↗
31. The state of the art of visualization in mixed reality image guided surgery10.1016/j.compmedimag.2013.01.009 ↗
32. Simulators and virtual reality in surgical education10.1016/j.ogc.2006.01.007 ↗
33. Shamir, R.R., Horn, M., Blum, T., Mehrkens, J., Shoshan, Y., Joskowicz, L., and Navab, N. (April, January 30). Trajectory planning with augmented reality for improved risk assessment in image-guided keyhole neurosurgery. Proceedings of the 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Chicago, IL, USA.10.1109/ISBI.2011.5872773 ↗
34. Augmented reality for anatomical education10.3109/17453050903557359 ↗
35. Evaluation of a haptics-based virtual reality temporal bone simulator for anatomy and surgery training10.1016/j.cmpb.2013.11.005 ↗
36. Stefan, P., Wucherer, P., Oyamada, Y., Ma, M., Schoch, A., Kanegae, M., Shimizu, N., Kodera, T., Cahier, S., and Weigl, M. (April, January 29). An AR edutainment system supporting bone anatomy learning. Proceedings of the 2014 IEEE Virtual Reality (VR), Minneapolis, MN, USA.10.1109/VR.2014.6802077 ↗
37. Personalized augmented reality for anatomy education10.1002/ca.22675 ↗
38. An interactive, stereoscopic virtual environment for medical imaging visualization, simulation and training
39. Exploring learner acceptance of the use of virtual reality in medical education: A case study of desktop and projection-based display systems10.1080/10494820.2013.817436 ↗
40. Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load10.1002/ase.1603 ↗
41. The effectiveness of virtual and augmented reality in health sciences and medical anatomy10.1002/ase.1696 ↗
42. Analysis of traditional versus three-dimensional augmented curriculum on anatomical learning outcome measures10.1002/ase.1612 ↗
43. Augmented reality in the surgery of cerebral arteriovenous malformations: Technique assessment and considerations10.1007/s00701-014-2183-9 ↗
44. Augmented reality–guided neurosurgery: Accuracy and intraoperative application of an image projection technique10.3171/2014.9.JNS141001 ↗
45. Augmented reality in neurovascular surgery: Feasibility and first uses in the operating room10.1007/s11548-015-1163-8 ↗
46. Training for planning tumour resection: Augmented reality and human factors10.1109/TBME.2014.2385874 ↗
47. Current accuracy of augmented reality neuronavigation systems: Systematic review and meta-analysis10.1016/j.wneu.2020.11.029 ↗
48. Intraoperative clinical application of augmented reality in neurosurgery: A systematic review10.1016/j.clineuro.2018.11.018 ↗
49. Future directions of operative neuro-oncology10.1007/s11060-016-2180-3 ↗
50. Applicability of augmented reality in orthopedic surgery–a systematic review10.1186/s12891-020-3110-2 ↗
51. Breaking through the “glass ceiling” of minimally invasive spine surgery
52. Guided pedicle screw insertion: Techniques and training10.1016/j.spinee.2013.03.029 ↗
53. The accuracy of pedicle screw placement using intraoperative image guidance systems: A systematic review10.3171/2013.11.SPINE13413 ↗
54. Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: A pilot study10.1007/s11548-017-1652-z ↗
55. A randomized trial of laparoscopic versus open surgery for rectal cancer10.1056/NEJMoa1414882 ↗
56. Lamata, P., Ali, W., Cano, A., Cornella, J., Declerck, J., Elle, O.J., Freudenthal, A., Furtado, H., Kalkofen, D., and Naerum, E. (2010). Augmented Reality, IntechOpen.
57. Telestration with augmented reality for visual presentation of intraoperative target structures in minimally invasive surgery: A randomized controlled study10.1007/s00464-022-09158-1 ↗
58. Fuchs, H., Livingston, M.A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., and Meyer, A.A. (1998). International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.
59. An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions10.1007/s11548-019-02001-4 ↗
60. Evaluation of a novel navigation platform for laparoscopic liver surgery with organ deformation compensation using injected fiducials10.1016/j.media.2020.101946 ↗
61. Augmented reality navigation for stereoscopic laparoscopic anatomical hepatectomy of primary liver cancer: Preliminary experience
62. The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery10.1038/s41598-019-54915-3 ↗
63. Preclinical evaluation of ultrasound-augmented needle navigation for laparoscopic liver ablation10.1007/s11548-020-02164-5 ↗
64. Live augmented reality: A new visualization method for laparoscopic surgery using continuous volumetric computed tomography10.1007/s00464-010-0890-8 ↗
65. Augmented reality navigation for liver resection with a stereoscopic laparoscope10.1016/j.cmpb.2019.105099 ↗
66. Performance of image guided navigation in laparoscopic liver surgery–A systematic review10.1016/j.suronc.2021.101637 ↗
67. Automatic localization of endoscope in intraoperative CT image: A simple approach to augmented reality guidance in laparoscopic surgery10.1016/j.media.2016.01.008 ↗
68. Jayarathne, U.L., Moore, J., Chen, E., Pautler, S.E., and Peters, T.M. (2017). International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.
69. Evaluation of a navigation system for dental implantation as a tool to train novice dental practitioners10.1016/j.joms.2011.04.026 ↗
70. Yamaguchi, S., Ohtani, T., Yatani, H., and Sohmura, T. (2009). International Conference on Virtual and Mixed Reality, Springer.
71. Tran, H.H., Suenaga, H., Kuwana, K., Masamune, K., Dohi, T., Nakajima, S., and Liao, H. (2011). International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.
72. Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery10.1109/TBME.2014.2301191 ↗
73. Katić, D., Sudra, G., Speidel, S., Castrillon-Oberndorfer, G., Eggers, G., and Dillmann, R. (2010). International Workshop on Medical Imaging and Virtual Reality, Springer.
74. A cognitive engineering framework for the specification of information requirements in medical imaging: Application in image-guided neurosurgery10.1007/s11548-012-0781-7 ↗
75. Presurgical and intraoperative augmented reality in neuro-oncologic surgery: Clinical experiences and limitations10.1016/j.wneu.2019.04.256 ↗
76. On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial10.1117/1.JMI.5.2.021209 ↗
77. Wang, J., Suenaga, H., Yang, L., Liao, H., Kobayashi, E., Takato, T., and Sakuma, I. (2013). Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, Springer.
78. 3D augmentation of the surgical video stream: Toward a modular approach10.1016/j.cmpb.2020.105505 ↗
79. Virtual and augmented medical imaging environments: Enabling technology for minimally invasive cardiac interventional guidance10.1109/RBME.2010.2082522 ↗
80. Usability of a real-time tracked augmented reality display system in musculoskeletal injections
81. Cercenelli, L., Carbone, M., Condino, S., Cutolo, F., Marcelli, E., Tarsitano, A., Marchetti, C., Ferrari, V., and Badiali, G. (2020). The wearable VOSTARS system for augmented reality-guided surgery: Preclinical phantom evaluation for high-precision maxillofacial tasks. J. Clin. Med., 9.10.3390/jcm9113562 ↗
82. Evaluation of wearable haptic systems for the fingers in augmented reality applications10.1109/TOH.2017.2691328 ↗
83. Software framework for customized augmented reality headsets in medicine10.1109/ACCESS.2019.2962122 ↗
84. Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: Phantom studies10.1007/s11548-011-0660-7 ↗
85. Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms10.1016/S1361-8415(02)00088-9 ↗
86. The Surgical Approach Visualization and Navigation (SAVN) System reduces radiation dosage and surgical trauma due to accurate intraoperative guidance10.1016/j.injury.2019.03.030 ↗
87. Hand gesture guided robot-assisted surgery based on a direct augmented reality interface10.1016/j.cmpb.2013.12.018 ↗
88. Wen, R., Chng, C.B., and Chui, C.K. (2017). Augmented reality guidance with multimodality imaging data and depth-perceived interaction for robot-assisted surgery. Robotics, 6.10.3390/robotics6020013 ↗
89. Giannone, F., Felli, E., Cherkaoui, Z., Mascagni, P., and Pessaux, P. (2021). Augmented Reality and Image-Guided Robotic Liver Surgery. Cancers, 13.10.3390/cancers13246268 ↗
90. Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery10.1016/j.media.2013.04.003 ↗
91. Peters, T.M., Linte, C.A., Yaniv, Z., and Williams, J. (2018). Mixed and Augmented Reality in Medicine, CRC Press.10.1201/9781315157702 ↗
92. Jannin, P., and Korb, W. (2008). Image-Guided Interventions, Springer.
93. The surgeon’s contribution to image-guided oncology10.1159/000366458 ↗
94. Comparative effectiveness and safety of image guidance systems in surgery: A preclinical randomised study10.1016/S0140-6736(15)60379-8 ↗
95. Augmented and virtual reality in surgery-the digital surgical environment: Applications, limitations and legal pitfalls10.21037/atm.2016.12.23 ↗
96. Tissue structure updating for In situ augmented reality navigation using calibrated ultrasound and two-level surface warping
97. An augmented reality system for liver thermal ablation: Design and evaluation on clinical cases10.1016/j.media.2009.02.003 ↗
98. Hand-eye calibration using a target registration error model10.1049/htl.2017.0072 ↗
99. A computationally efficient method for hand-eye calibration10.1007/s11548-017-1646-x ↗
100. Contact-less stylus for surgical navigation: Registration without digitization10.1007/s11548-017-1576-7 ↗