EasyVizAR: Supporting First Responders Through the Use of Collaborative Augmented Reality Tools
Source abstract
Abstract
First responders operate in high-stakes environments, demanding rapid, accurate decision-making. Recent research has provided recommendations on how Augmented Reality (AR) could be utilized to support their efforts. Building off of these studies, this paper presents EasyVizAR, an AR system designed to enhance situational awareness and operational efficiency in challenging indoor scenarios. Leveraging edge computing and advanced computer vision techniques, EasyVizAR addresses critical challenges, such as object detection, localization, and information sharing. This research details the system’s architecture, including its use of ParaDrop-based edge computing, and explores its application in rescue and active-shooter scenarios. We present our work in developing key features, including real-time object saliency cues, improved object detection, person identification, multi-user 3D map generation and visualization, and multimodal AR navigation cues.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Augmented Reality Applications · Visual Attention and Saliency Detection · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Wisconsin Institute for Discovery, University of Wisconsin-Madison, Madison, WI 53706, USA
Department of Computer Sciences, School of Computer, Data & Information Sciences University of Wisconsin-Madison, Madison, WI 53706, USA
Funding
National Institute of Standards and Technology
References (46)
Show source-provided references
1. Morrison, K., Dawkins, S., Choong, Y.Y., Theofanos, M.F., Greene, K., and Furman, S. (2021, January 24–29). Current problems, future needs: Voices of first responders about communication technology. Proceedings of the Human-Computer Interaction. Design and User Experience Case Studies: Thematic Area, HCI 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual Event. Proceedings, Part III 23.10.1007/978-3-030-78468-3_26 ↗
2. The physical and mental health challenges experienced by 9/11 first responders and recovery workers: A review of the literature10.1017/S1049023X19004989 ↗
3. Dimou, A., Kogias, D.G., Trakadas, P., Perossini, F., Weller, M., Balet, O., Patrikakis, C.Z., Zahariadis, T., and Daras, P. (2021). FASTER: First Responder Advanced Technologies for Safe and Efficient Emergency Response. Technology Development for Security Practitioners, Springer.10.1007/978-3-030-69460-9_26 ↗
4. Gutiérrez, Á., Blanco, P., Ruiz, V., Chatzigeorgiou, C., Oregui, X., Álvarez, M., Navarro, S., Feidakis, M., Azpiroz, I., and Izquierdo, G. (2023). Biosignals monitoring of first responders for cognitive load estimation in real-time operation. Appl. Sci., 13.10.3390/app13137368 ↗
5. Zechner, O., García Guirao, D., Schrom-Feiertag, H., Regal, G., Uhl, J.C., Gyllencreutz, L., Sjöberg, D., and Tscheligi, M. (2023). NextGen Training for Medical First Responders: Advancing Mass-Casualty Incident Preparedness through Mixed Reality Technology. Multimodal Technol. Interact., 7.10.3390/mti7120113 ↗
6. Exploring wearable technology use and importance of health monitoring in the hazardous occupations of first responders and professional drivers10.1093/joccuh/uiad002 ↗
7. Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., and Varol, G. (October, January 29). Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. Proceedings of the Computer Vision—ECCV 2024, Milan, Italy.10.1007/978-3-031-72980-5 ↗
8. Zhang, K., Cochran, B.R., Chen, R., Hartung, L., Sprecher, B., Tredinnick, R., Ponto, K., Banerjee, S., and Zhao, Y. (2024, January 11–16). Exploring the design space of optical see-through AR head-mounted displays to support first responders in the field. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613904.3642195 ↗
9. Argo, E., Ahmed, T., Gable, S., Hampton, C., Grandi, J., and Kopper, R. (2025). Augmented Reality User Interfaces for First Responders: A Scoping Literature Review. arXiv.
10. Towards Extended Reality in Emergency Response: Guidelines and Challenges for First Responder Friendly Augmented Interfaces10.1002/cav.70056 ↗
11. Grandi, J.G., Cao, Z., Ogren, M., and Kopper, R. (April, January 27). Design and Simulation of Next-Generation Augmented Reality User Interfaces in Virtual Reality. Proceedings of the 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Virtual Event.10.1109/VRW52623.2021.00011 ↗
12. Jiang, X., Chen, N.Y., Hong, J.I., Wang, K., Takayama, L., and Landay, J.A. (2004, January 14–17). Siren: Context-aware computing for firefighting. Proceedings of the International Conference on Pervasive Computing, Orlando, FL, USA.10.1007/978-3-540-24646-6_6 ↗
13. Killeen, J.P., Chan, T.C., Buono, C., Griswold, W.G., and Lenert, L.A. (2006, January 11–15). A Wireless First Responder Handheld Device for Rapid Triage, Patient Assessment and Documentation During Mass Casualty Incidents. Proceedings of the AMIA Annual Symposium Proceedings, Washington, DC, USA.
14. KYNG, M., Nielsen, E.T., and Kristensen, M. (2006, January 26–28). Challenges in Designing Interactive Systems for Emergency Response. Proceedings of the 6th Conference on Designing Interactive Systems (University Park, PA, USA) (DIS ’06), University Park, PA, USA.10.1145/1142405.1142450 ↗
15. Mobile computing in urban emergency situations: Improving the support to firefighters in the field10.1016/j.eswa.2010.05.018 ↗
16. “Sizing Up” Emerging Technology for Firefighting: Augmented Reality for Incident Assessment10.1177/1541931218621332 ↗
17. Design of monocular head-mounted displays for increased indoor firefighting safety and efciency
18. Design and manufacture AR head-mounted displays: A review and outlook
19. (2025, October 20). Augview. Available online: http://www.augview.net.
20. BIM-based augmented reality inspection and maintenance of fire safety equipment10.1016/j.autcon.2019.103041 ↗
21. Datcu, D., Lukosch, S.G., and Lukosch, H.K. (2016, January 13–16). Handheld augmented reality for distributed collaborative crime scene investigation. Proceedings of the 19th International Conference on Supporting Group Work, Sanibel Island, FL, USA.10.1145/2957276.2957302 ↗
22. Nunes, I.L., Lucas, R., Simões-Marques, M., and Correia, N. (2018, January 26–30). An Augmented Reality Application to Support Deployed Emergency Teams. Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018), Florence, Italy.10.1007/978-3-319-96077-7_21 ↗
23. Sebillo, M., Tortora, G., Vitiello, G., Paolino, L., and Ginige, A. (2024, January 29). The use of augmented reality interfaces for on-site crisis preparedness. Proceedings of the International Conference on Learning and Collaboration Technologies, Washington, DC, USA.
24. Training Emergency Responders through Augmented Reality Mobile Interfaces10.1007/s11042-015-2955-0 ↗
25. Siu, T., and Herskovic, V. (2013, January 11–15). SidebARs: Improving awareness of off-screen elements in mobile augmented reality. Proceedings of the 2013 Chilean Conference on Human-Computer Interaction, Temuco, Chile.10.1145/2535597.2535608 ↗
26. Smart augmented fields for emergency operations10.1016/j.procs.2015.08.358 ↗
27. Evaluation of augmented reality for rapid assessment of earthquake-induced building damage10.1061/(ASCE)0887-3801(2007)21:5(303) ↗
28. Sainidis, D., Tsiakmakis, D., Konstantoudakis, K., Albanis, G., Dimou, A., and Daras, P. (2021, January 23–26). Single-handed gesture UAV control and video feed AR visualization for first responders. Proceedings of the International Conference on Information Systems for Crisis Response and Management (ISCRAM), Blacksburg, VA, USA.
29. Wani, A.R., Shabir, S., and Naaz, R. (2013, January 8–9). Augmented reality for fire and emergency services. Proceedings of the Int. Conf. on Recent Trends in Communication and Computer Networks, Byderabad, India.
30. Nelson, C.R., Conwell, J., Lally, S., Cohn, M., Tanous, K., Moats, J., and Gabbard, J.L. (2022, January 10–14). Exploring Augmented Reality Triage Tools to Support Mass Casualty Incidents. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Atlanta, GA, USA.
31. Cagiltay, B., Oztank, F., and Balcisoy, S. (2025, January 27–29). The Case for Audio-First Mixed Reality: An AI-Enhanced Framework. Proceedings of the 2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Lisbon, Portugal.10.1109/AIxVR63409.2025.00032 ↗
32. Wilchek, M., Wang, L., Dickinson, S., Feuerbacher, E., Luther, K., and Batarseh, F.A. (2025, January 24–27). KHAIT: K-9 Handler Artificial Intelligence Teaming for Collaborative Sensemaking. Proceedings of the 30th International Conference on Intelligent User Interfaces, Cagliari, Italy.10.1145/3708359.3712107 ↗
33. Liu, P., Willis, D., and Banerjee, S. (2016, January 27–28). ParaDrop: Enabling Lightweight Multi-tenancy at the Network’s Extreme Edge. Proceedings of the 2016 IEEE/ACM Symposium on Edge Computing (SEC), Washington, DC, USA.10.1109/SEC.2016.39 ↗
34. Edge-SLAM: Edge-assisted visual simultaneous localization and mapping
35. Chen, Y., Inaltekin, H., and Gorlatova, M. (2023, January 17–20). AdaptSLAM: Edge-assisted adaptive SLAM with resource constraints via uncertainty minimization. Proceedings of the IEEE INFOCOM 2023-IEEE Conference on Computer Communications, New York City, NY, USA.10.1109/INFOCOM53939.2023.10229009 ↗
36. van Toll, W., Cook, A.F., and Geraerts, R. (2011, January 25–30). Navigation Meshes for Realistic Multi-Layered Environments. Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.10.1109/IROS.2011.6048397 ↗
37. van Toll, W., Triesscheijn, R., Kallmann, M., Oliva, R., Pelechano, N., Pettré, J., and Geraerts, R. (2016, January 10–12). A Comparative Study of Navigation Meshes. Proceedings of the 9th International Conference on Motion in Games, New York, NY, USA. MIG ’16.10.1145/2994258.2994262 ↗
38. Performance analysis of navigation ai on commercial game engine: Autodesk stingray and Unity3D
39. Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., and Ding, G. (2024, January 10–15). YOLOv10: Real-Time End-to-End Object Detection. Proceedings of the 38th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
40. A Benchmark of Facial Recognition Pipelines and Co-Usability Performances of Modules
41. Wu, Y., Zhang, Z., Chen, J., Tang, H., Li, D., Fang, Y., Zhu, L., Xie, E., Yin, H., and Yi, L. (2024). Vila-u: A unified foundation model integrating visual understanding and generation. arXiv.
42. Ren, T., Jiang, Q., Liu, S., Zeng, Z., Liu, W., Gao, H., Huang, H., Ma, Z., Jiang, X., and Chen, Y. (2024). Grounding dino 1.5: Advance the“ edge” of open-set object detection. arXiv.
43. Chen, R., Jiang, J., Maheshwary, P., Cochran, B.R., and Zhao, Y. (May, January 26). VisiMark: Characterizing and Augmenting Landmarks for People with Low Vision in Augmented Reality to Support Indoor Navigation. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan. CHI ’25.10.1145/3706598.3713847 ↗
44. Ansari, M.M. (2025). Evaluating Stenosis Detection with Grounding DINO, YOLO, and DINO-DETR. arXiv.
45. Teixeira, E.H., Melo, M.C., Junior, W.B., Silva, E.C., Cruz, M.R., Pimenta, T.C., Aquino, G.P., and Boas, E.C.V. (2024, January 20–21). Brain Tumor Images Class-Based and Prompt-Based Detectors and Segmenter: Performance Evaluation of YOLO, SAM and Grounding DINO. Proceedings of the 2024 International Conference on Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD), Bali, Indonesia.10.1109/ICoABCD63526.2024.10704507 ↗
46. Moore, L.K. (2016, June 17). The First Responder Network (FirstNet) and Next-Generation Communications for Public Safety: Issues for Congress. Congressional Research Service. Available online: https://allthingsfirstnet.com/wp-content/uploads/2017/05/CRS-FirstNet-Linda-Moore-Report-2016.pdf.