Accelerating Surgical Skill Acquisition by Using Multi-View Bullet-Time Video Generation
Source abstract
Abstract
Surgical education and training have seen significant advancements with the integration of innovative technologies. This paper presents a novel approach to surgical education using a multi-view capturing system and bullet-time generation techniques to enhance the learning experience for aspiring surgeons. The proposed system leverages an array of synchronized cameras strategically positioned around a surgical simulation environment, enabling the capture of surgical procedures from multiple angles simultaneously. The captured multi-view data is then processed using advanced computer vision and image processing algorithms to create a “bullet-time” effect, similar to the iconic scenes from The Matrix movie, allowing educators and trainees to manipulate time and view the surgical procedure from any desired perspective. In this paper, we propose the technical aspects of the multi-view capturing system, the bullet-time generation process, and the integration of these technologies into surgical education programs. We also discuss the potential applications in various surgical specialties and the benefits of utilizing this system for both novice and experienced surgeons. Finally, we present preliminary results from pilot studies and user feedback, highlighting the promising potential of this innovative approach to revolutionize surgical education and training.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Surgical Simulation and Training · Augmented Reality Applications · Anatomy and Medical Technology
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Ph.D. Program in Humanics, School of Integrative and Global Majors, University of Tsukuba, Tsukuba 305-8577, Japan
Center for Computational Sciences, University of Tsukuba, Tsukuba 305-8577, Japan
Faculty of Medicine, University of Tsukuba, Tsukuba 305-8575, Japan
Funding
JSPS Grant-in-Aid for Scientific Research · MEXT Promotion of Development of a Joint Usage/Research System Project: Coalition of Universities for Research Excellence Program (CURE)
References (33)
Show source-provided references
1. Video Technologies for Recording Open Surgery: A Systematic Review10.1177/1553350619853099 ↗
2. Motion Tracking System in Surgical Training
3. Vajsbaher, T., Schultheis, H., and Francis, N.K. (2018). Spatial cognition in minimally invasive surgery: A systematic review. BMC Surg., 18.10.1186/s12893-018-0416-1 ↗
4. Shahrezaei, A., Sohani, M., Taherkhani, S., and Zarghami, S.Y. (2024). The impact of surgical simulation and training technologies on general surgery education. BMC Med. Educ., 24.10.1186/s12909-024-06299-w ↗
5. Innovations in surgical training: Exploring the role of artificial intelligence and large language models (LLM)
6. Wang, Y., Xie, C., Shishido, H., Hashimoto, S., Oda, T., and Kitahara, I. (2021, January 12–15). A surgical bullet-time video capturing system depending on surgical situation. Proceedings of the 10th IEEE Global Conference on Consumer Electronics (GCCE 2021), Kyoto, Japan.10.1109/GCCE53005.2021.9622047 ↗
7. Surgery recording without occlusions by multi-view surgical videos
8. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv.
9. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
10. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 11–14). SSD: Single Shot MultiBox Detector. Proceedings of the Computer Vision—ECCV 2016. ECCV 2016, Amsterdam, The Netherlands. Lecture Notes in Computer Science.10.1007/978-3-319-46466-4 ↗
11. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.10.1109/CVPR52729.2023.00721 ↗
12. Wachowski, L. (Directors). The Matrix [Film]. Warner Bros.; Village Roadshow Pictures; Groucho Film Partnership. 1999.
13. Akechi, N., Kitahara, I., Sakamoto, R., and Ohta, Y. (2014, January 3–6). Multi-resolution bullet-time effect. Proceedings of the SIGGRAPH-ASIA, Shenzhen, China.10.1145/2668975.2669015 ↗
14. Nagai, T., Shishido, H., Kameda, Y., and Kitahara, I. (2018, January 26). An on-site visual feedback method using bullet-time video. Proceedings of the 1st International Workshop on Multimedia Content Analysis in Sports, Seoul, Republic of Korea.10.1145/3265845.3265853 ↗
15. Depth over RGB: Automatic evaluation of open surgery skills using depth camera10.1007/s11548-024-03158-3 ↗
16. Schönberger, J.L., and Frahm, J.M. (2016, January 27–30). Structure-from-Motion Revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.445 ↗
17. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
18. XMOS Limited (2022, May 14). AN00127: USB Video Class Device (Version 2.0.1 Release Candidate 1) [Application Note]. n.d. Available online: https://www.xmos.com/download/AN00127:-USB-Video-Class-Device(2.0.1rc1).pdf/.
19. Microsoft (2022, May 14). KSMedia.h Header. Windows Driver Kit (WDK) Documentation. n.d. Available online: https://learn.microsoft.com/en-us/windows-hardware/drivers/.
20. OpenCV Python Team (2022, May 14). Opencv-Python (Version 4.11.0) [Python Package]. Available online: https://pypi.org/project/opencv-python/.
21. Jiang, K., Pan, S., Yang, L., Yu, J., Lin, Y., and Wang, H. (2023). Surgical Instrument Recognition Based on Improved YOLOv5. Appl. Sci., 13.10.3390/app132111709 ↗
22. Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018). CBAM: Convolutional block attention module. arXiv.10.1007/978-3-030-01234-2_1 ↗
23. Deep High-Resolution Representation Learning for Visual Recognition10.1109/TPAMI.2020.2983686 ↗
24. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields10.1109/TPAMI.2019.2929257 ↗
25. Monitor height ergonomics: A comparison of operating room video display terminals
26. Ha, H., Xiao, L., Richardt, C., Nguyen-Phuoc, T., Kim, C., Kim, M.H., Lanman, D., and Khan, N. (2025, January 11–15). Geometry-guided Online 3D Video Synthesis with Multi-View Temporal Consistency. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.
27. Chen, X., Wang, Y., and Zhang, Z. (2025, January 11–15). CBAM-STN-TPS-YOLO: Enhancing agricultural object detection through spatially adaptive attention mechanisms. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.
28. Qin, Q., Zhou, X., Gao, J., Wang, Z., Naer, A., Hai, L., Alatan, S., Zhang, H., and Liu, Z. (2025). YOLOv8-CBAM: A study of sheep head identification in Ujumqin sheep. Front. Vet. Sci., 12.10.3389/fvets.2025.1514212 ↗
29. Gonzalez-Navarro, A.R., Quiroga-Garza, A., Acosta-Luna, A.S., Salinas-Alvarez, Y., Martinez-Garza, J.H., de la Garza-Castro, O., Gutierrez-de la O, J., de la Fuente-Villarreal, D., Elizondo-Omaña, R.E., and Guzman-Lopez, S. (2021). Comparison of suturing models: The effect on perception of basic surgical skills. BMC Med. Educ., 21.10.1186/s12909-021-02692-x ↗
30. Learning Surgical Skills Through Video-Based Education: A Systematic Review10.1177/15533506221120146 ↗
31. Pan, M., Wang, S., Li, J., Li, J., Yang, X., and Liang, K. (2023). An Automated Skill Assessment Framework Based on Visual Motion Signals and a Deep Neural Network in Robot-Assisted Minimally Invasive Surgery. Sensors, 23.10.3390/s23094496 ↗
32. Ebina, K., Abe, T., Yan, L., Hotta, K., Shichinohe, T., Higuchi, M., Iwahara, N., Hosaka, Y., Harada, S., and Kikuchi, H. (2024). A surgical instrument motion measurement system for skill evaluation in practical laparoscopic surgery training. PLoS ONE, 19.10.1371/journal.pone.0305693 ↗
33. Bayareh-Mancilla, R., Medina-Ramos, L.A., Toriz-Vázquez, A., Hernández-Rodríguez, Y.M., and Cigarroa-Mayorga, O.E. (2023). Automated Computer-Assisted Medical Decision-Making System Based on Morphological Shape and Skin Thickness Analysis for Asymmetry Detection in Mammographic Images. Diagnostics, 13.10.3390/diagnostics13223440 ↗