journal article·2022-12-08Open access

Towards the Evaluation of Augmented Reality in the Metaverse: Information Presentation Modes

Michele Gattullo, Enricoandrea Laviola, Alessandro Evangelista, Michele Fiorentino, Antonio Emmanuele Uva

Applied Sciences· Volume 12· Issue 24

Abstract

In the future, many activities will be carried out in the Metaverse: hybrid offices and video-based education are just some examples. The way research is carried out could change, too. In this context, this work investigates the possibility of simulating Augmented Reality (AR) user studies on information presentation in a virtual environment. Organizing an industrial setup is complex; thus, most studies are executed in laboratories. However, lab experiments present limitations, e.g., the number and variety of participants and the availability of facilities. User studies may also be carried out by exploiting simulated AR, as an initial step for the Metaverse, where people are connected regardless of their location. This alternative could be used to carry out experiments on AR information presentation to solve common issues, such as the lack of physical equipment to perform component location tasks and the long time required to collect a large sample of users. Indeed, researchers could propose CAD models with information that simulates the same visual realism achieved with true AR. Moreover, multiple tests could be conducted in parallel by not relying on a limited amount of physical equipment per user. In this work, we developed and evaluated a desktop-simulated testing environment (DSTE) to conduct AR information presentation experiments remotely. We applied it in a pilot user study, revealing that the proposed DSTE was effective for the related research goals. Furthermore, 40 participants reported a positive user experience. The evaluation confirms that using a DSTE is promising for collecting and analyzing data from a wide range of people.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Virtual Reality Applications and Impacts · Augmented Reality Applications · 3D Surveying and Cultural Heritage

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Mechanics, Mathematics, and Management, Polytechnic Institute of Bari, Via Orabona, 4, 70125 Bari, Italy

Funding

Italian Ministry of Education, University and Research under the Program “Department of Excellence” Law 232/2016

References (45)

Show source-provided references
1. A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard10.1109/ACCESS.2018.2808326 ↗
2. Augmented Reality in Industry 4.0
3. Adopting augmented reality in the age of industrial digitalisation10.1016/j.compind.2019.07.002 ↗
4. Uva, A., Fiorentino, M., Gattullo, M., Colaprico, M., de Ruvo, M.F., Marino, F., Trotta, G.F., Manghisi, V.M., Boccaccio, A., and Bevilacqua, V. (2016). Design of a Projective AR Workbench for Manual Working Stations. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer.10.1007/978-3-319-40621-3_25 ↗
5. Boccaccio, A., Cascella, G.L., Fiorentino, M., Gattullo, M., Manghisi, V.M., Monno, G., and Uva, A.E. (2019). Exploiting Augmented Reality to Display Technical Information on Industry 4.0 P&ID. Lecture Notes in Mechanical Engineering, Springer.10.1007/978-3-030-12346-8_28 ↗
6. A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 201410.3389/frobt.2018.00037 ↗
7. A Study on Metaverse Hype for Sustainable Growth
8. Virtual World as a Resource for Hybrid Education10.3991/ijet.v15i15.13025 ↗
9. A Study on Metaverse Culture Contents Matching Platform
10. Lorenz, M., Knopp, S., and Klimant, P. (2018, January 16–20). Industrial Augmented Reality: Requirements for an Augmented Reality Maintenance Worker Support System. Proceedings of the Adjunct Proceedings—2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, Munich, Germany.10.1109/ISMAR-Adjunct.2018.00055 ↗
11. Jeffri, N.F.S., and Rambli, D.R.A. (2020, January 14–18). Problems with Physical Simulation in a Virtual Lego-based Assembly Task using Unity3D Engine. Proceedings of the 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), Utrecht, The Netherlands.10.1109/AIVR50618.2020.00060 ↗
12. Indirect augmented reality10.1016/j.cag.2011.04.010 ↗
13. A Prototyping Method to Simulate Wearable Augmented Reality Interaction in a Virtual Environment—A Pilot Study
14. Lee, C., Bonebrake, S., Bowman, D.A., and Hollerer, T. (2010, January 20–24). The role of latency in the validity of AR simulation. Proceedings of the 2010 IEEE Virtual Reality Conference (VR), Boston, MA, USA.10.1109/VR.2010.5444820 ↗
15. Lee, C., Gauglitz, S., Hollerer, T., and Bowman, D.A. (2012, January 4–8). Examining the equivalence of simulated and real AR on a visual following and identification task. Proceedings of the 2012 IEEE Virtual Reality Workshops (VRW), Costa Mesa, CA, USA.10.1109/VR.2012.6180890 ↗
16. Ragan, E., Wilkes, C., Bowman, D.A., and Hollerer, T. (2009, January 14–18). Simulation of Augmented Reality Systems in Purely Virtual Environments. Proceedings of the 2009 IEEE Virtual Reality Conference, Lafayette, LA, USA.10.1109/VR.2009.4811058 ↗
17. Influences of Augmented Reality Assistance on Performance and Cognitive Loads in Different Stages of Assembly Task10.3389/fpsyg.2019.01703 ↗
18. Tang, A., Owen, C., Biocca, F., and Mou, W. (2003, January 5–10). Comparative effectiveness of augmented reality in object assembly. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA.10.1145/642611.642626 ↗
19. Funk, M., Kosch, T., and Schmidt, A. (2016, January 12–16). Interactive worker assistance: Comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, New York, NY, USA.10.1145/2971648.2971706 ↗
20. Renner, P., and Pfeiffer, T. (2017, January 9–13). [POSTER] Augmented Reality Assistance in the Central Field-of-View Outperforms Peripheral Displays for Order Picking: Results from a Virtual Reality Simulation Study. Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), Nantes, France.10.1109/ISMAR-Adjunct.2017.59 ↗
21. Minimal AR: Visual asset optimization for the authoring of augmented reality work instructions in manufacturing10.1007/s00170-021-08449-6 ↗
22. Construction of a Benchmark for the User Experience Questionnaire (UEQ)
23. A survey of industrial augmented reality10.1016/j.cie.2019.106159 ↗
24. Agati, S.S., Bauer, R.D., Hounsell, M.D.S., and Paterno, A.S. (2020, January 7–10). Augmented Reality for Manual Assembly in Industry 4.0: Gathering Guidelines. Proceedings of the 2020 22nd Symposium on Virtual and Augmented Reality (SVR), Porto de Galinhas, Brazil.10.1109/SVR51698.2020.00039 ↗
25. An augmented reality tool to detect and annotate design variations in an Industry 4.0 approach10.1007/s00170-019-04254-4 ↗
26. Boboc, R.G., Gîrbacia, F., and Butilă, E.V. (2020). The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review. Appl. Sci., 10.10.3390/app10124259 ↗
27. Integrating Augmented Reality in the Assembly Domain—Fundamentals, Benefits and Applications10.1016/S0007-8506(07)60517-4 ↗
28. Chen, C., Liang, R., Pan, Y., Li, D., Zhao, Z., Guo, Y., and Zhang, Q. (2022). A Quick Development Toolkit for Augmented Reality Visualization (QDARV) of a Factory. Appl. Sci., 12.10.3390/app12168338 ↗
29. Marques, B., Ferreira, C., Silva, S., Santos, A., Dias, P., and Santos, B.S. (2022). Are the Instructions Clear? Evaluating the Visual Characteristics of Augmented Reality Content for Remote Guidance. Multimodal Technol. Interact., 6.10.3390/mti6100092 ↗
30. Davies, R.C. (2002). Applications of systems design using virtual environments. Handbook of Virtual Environments, CRC Press.
31. The Effects of Visual Realism on Search Tasks in Mixed Reality Simulation10.1109/TVCG.2013.41 ↗
32. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments10.1162/pres.1997.6.6.603 ↗
33. Bowman, D., Kruijff, E., LaViola, J.J., and Poupyrev, I. (2004). 3D User Interfaces: Theory and Practice, CourseSmart eTextbook, Taylor Francis Group.
34. Assembly planning and evaluation in an augmented reality environment10.1080/00207543.2013.837986 ↗
35. Tran, V.T., and Kim, D. (2016). An Application of Virtual Reality in E-learning based LEGO-Like Brick Assembling. Proc. Korea Inf. Process. Soc. Conf., 783–786.
36. What, How, and Why are Visual Assets Used in Industrial Augmented Reality? A Systematic Review and Classification in Maintenance, Assembly, and Training (From 1997 to 2019)10.1109/TVCG.2020.3014614 ↗
37. Gattullo, M., Scurati, G.W., Evangelista, A., Ferrise, F., Fiorentino, M., and Uva, A.E. (2020). Informing the Use of Visual Assets in Industrial Augmented Reality. Lecture Notes in Mechanical Engineering, Springer.10.1007/978-3-030-31154-4_10 ↗
38. Robertson, C.M., MacIntyre, B., and Walker, B.N. (2008, January 15–18). An evaluation of graphical context when the graphics are outside of the task area. Proceedings of the 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK.10.1109/ISMAR.2008.4637328 ↗
39. Alves, J., Marques, B., Oliveira, M., Araujo, T., Dias, P., and Santos, B.S. (2019, January 24–26). Comparing Spatial and Mobile Augmented Reality for Guiding Assembling Procedures with Task Validation. Proceedings of the 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Porto, Portugal.10.1109/ICARSC.2019.8733642 ↗
40. Marques, B., Alves, J., Neves, M., Justo, I., Santos, A., Rainho, R., Maio, R., Costa, D., Ferreira, C., and Dias, P. (2020, January 4). Interaction with Virtual Content using Augmented Reality: A User Study in Assembly Procedures. Proceedings of the ACM on Human-Computer Interaction, New York, NY, USA.10.1145/3427324 ↗
41. Easy-to-use virtual brick manipulation techniques using hand gestures10.1007/s11227-015-1588-4 ↗
42. Direct Measurement of Cognitive Load in Multimedia Learning10.1207/S15326985EP3801_7 ↗
43. Lu, F., and Bowman, D.A. (April, January 27). Evaluating the Potential of Glanceable AR Interfaces for Authentic Everyday Uses. Proceedings of the 2021 IEEE Virtual Reality and 3D User Interfaces (VR), Lisboa, Portugal.10.1109/VR50410.2021.00104 ↗
44. Popovici, I., and Vatavu, R.-D. (2019, January 14–18). Understanding Users’ Preferences for Augmented Reality Television. Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Beijing, China.10.1109/ISMAR.2019.00024 ↗
45. Häkkilä, J., Colley, A., and Rantakari, J. (2014, January 17–19). Exploring Mixed Reality Window Concept for Car Passengers. Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, New York, NY, USA.10.1145/2667239.2667288 ↗