Evaluating Levels of Automation in Human–Robot Collaboration at Different Workload Levels
Source abstract
Abstract
This study explored how levels of automation (LOA) influence human robot collaboration when operating at different levels of workload. Two LOA modes were designed, implemented, and evaluated in an experimental collaborative assembly task setup for four levels of workload composed of a secondary task and task complexity. A user study conducted involving 80 participants was assessed through two constructs especially designed for the evaluation (quality of task execution and usability) and user preferences regarding the LOA modes. Results revealed that the quality of task execution and usability was better at high LOA for low workload. Most of participants also preferred high LOA when the workload increases. However, when complexity existed within the workload, most of the participants preferred the low LOA. The results reveal the benefits of high and low LOA in different workload situations. This study provides insights related to shared control designs and reveals the importance of considering different levels of workload as influenced by secondary tasks and task complexity when designing LOA in human–robot collaborations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human-Automation Interaction and Safety · Healthcare Technology and Patient Monitoring · Occupational Health and Safety Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, P.O. Box 653, Be’er Sheva 8410501, Israel
Funding
Horizon 2020 Framework Programme
References (64)
Show source-provided references
1. Human-Robot collaboration: A survey10.1142/S0219843608001303 ↗
2. Goodrich, M.A., and Schultz, A.C. (2007). Human-Robot Interaction: A Survey, Now Publishers Inc.10.1561/9781601980939 ↗
3. Human-robot interaction in industrial collaborative robotics: A literature review of the decade 2008–201710.1080/01691864.2019.1636714 ↗
4. Communication in human-robot interaction10.1007/s43154-020-00026-1 ↗
5. Human-Robot interaction in rehabilitation and assistance: A review10.1007/s43154-020-00015-4 ↗
6. How to include user experience in the design of human-robot interaction10.1016/j.rcim.2020.102072 ↗
7. Future directions and research issues for ergonomics and advanced manufacturing technology (AMT)10.1016/0003-6870(88)90191-3 ↗
8. Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety
9. Level of automation effects on performance, situation awareness and workload in a dynamic control task10.1080/001401399185595 ↗
10. Levels of automation in manufacturing
11. Shi, J., Jimmerson, G., Pearson, T., and Menassa, R. (2012, January 20–22). Levels of human and robot collaboration for automotive manufacturing. Proceedings of the Workshop on Performance Metrics for Intelligent Systems, College Park, MD, USA.10.1145/2393091.2393111 ↗
12. Improving the interaction of older adults with a socially assistive table setting robot
13. Wang, W., Chen, Y., Li, R., and Jia, Y. (2019). Learning and comfort in human-robot interaction: A review. Appl. Sci., 9.10.3390/app9235152 ↗
14. Human performance measures for the evaluation of process control human-system interfaces in high-fidelity simulations10.1016/j.apergo.2018.06.008 ↗
15. Hart, S.G., and Wickens, C.D. (1990). Workload assessment and prediction. Manprint, Springer.10.1007/978-94-009-0437-8_9 ↗
16. NASA-Task load index (NASA-TLX); 20 years later
17. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research10.1016/S0166-4115(08)62386-9 ↗
18. Hancock, P.A., and Desmond, P.A. (2001). Workload and air traffic control. Human Factors in Transportation. Stress, Workload, and Fatigue, Lawrence Erlbaum Associates Publishers.
19. Dissociation of performance and subjective measures of workload10.1177/001872088803000110 ↗
20. Task complexity: Definition of the construct10.1016/0749-5978(86)90044-0 ↗
21. Task complexity as a performance shaping factor: A review and recommendations in standardized plant analysis risk-human reliability analysis (SPAR-H) adaption10.1016/j.ssci.2015.03.005 ↗
22. Olsen, D.R., and Goodrich, M.A. (2021, August 09). Metrics for Evaluating Human-Robot Interactions. Available online: https://faculty.cs.byu.edu/~mike/mikeg/papers/OlsenGoodrichPERMIS2003.pdf.
23. Task complexity: A review and analysis10.2307/258353 ↗
24. Subjective task complexity and subjective workload: Criterion validity for complex team tasks10.1207/S15327566IJCE0503_7 ↗
25. Neill, T.A.O., Mcneese, N.J., Carolina, S., and Barron, A. (2017). Human—Autonomy teaming: A Review and analysis of the empirical literature. J. Hum. Factors Ergon. Soc.
26. Lindblom, J. Foundation for a classification of collaboration levels for human-robot cooperation in manufacturing
27. Aligning manufacturing strategy and levels of automation: A case study10.1016/j.jengtecman.2010.06.002 ↗
28. Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., and Goodrich, M. (2006, January 2–3). Common metrics for human-robot interaction. Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (H 06), Salt Lake City, UT, USA.10.1145/1121241.1121249 ↗
29. Metrics and benchmarks in human-robot interaction: Recent advances in cognitive robotics10.1016/j.cogsys.2016.06.002 ↗
30. Bensch, S., Jevtíc, A., and Hellström, T. (2017, January 24–26). On interaction quality in human-robot interaction. In Proceedings of the 9th International Conference on Agents and Artificial Intelligence (ICAART 2017), Porto, Portugal.10.5220/0006191601820189 ↗
31. Efthimiou, E., Papageorgiou, X.S., Fotinea, S.E., Karavasili, A., Vacalopoulou, A., and Goulas, T. (2019, January 5–7). User centered design in practice adapting HRI to real user needs. Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments, Rhodes, Greece.10.1145/3316782.3322778 ↗
32. Lindblom, J., Alenljung, B., and Billing, E. (2020). Evaluating the User Experience of Human-Robot Interaction, Springer.10.1007/978-3-030-42307-0_9 ↗
33. Honig, S.S., Oron-Gilad, T., Serna-Fleischmann, V., Olatunji, S., and Edan, Y. (2018, January 1–5). A user-needs based approach for designing human-robot interactions. Proceedings of the Workshop on Robotic Co-Workers 4.0: Human Safety and Comfort in Human-Robot Interactive Social Environments, IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain.
34. Designing automation for human use: Empirical studies and quantitative models10.1080/001401300409125 ↗
35. Human-Robot collaboration acceptance model: Development and comparison for Germany, Japan, China and the USA10.1007/s12369-019-00593-0 ↗
36. Renner, P., and Pfeiffer, T. (2014, January 26–28). Model-based acquisition and analysis of multimodal interactions for improving human-robot interaction. Proceedings of the Symposium on Eye Tracking Research and Applications, Safety Harbor, FL, USA.10.1145/2578153.2582176 ↗
37. Multiple resources and mental workload10.1518/001872008X288394 ↗
38. Stages and levels of automation: An integrated meta-analysis10.1177/154193121005400425 ↗
39. Onnasch, L., Wickens, C.D., Li, H., and Manzey, D. (2014). Human performance consequences of stages and levels of automation: An integrated meta-analysis. J. Hum. Factors Ergon. Soc., 56.10.1177/0018720813501549 ↗
40. ISO (2020, February 29). ISO 9241-11:2018(en). ERGONOMICS of Human-System Interaction—Part 11: Usability: Definitions and Concepts., Available online: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en.
41. Ashcraft, C.C., Goodrich, M.A., and Crandall, J.W. (2019, January 6–9). Moderating operator influence in human-swarm systems. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.10.1109/SMC.2019.8914592 ↗
42. The Out-of-the-loop performance problem and level of control in automation10.1518/001872095779064555 ↗
43. Syrdal, D.S., Dautenhahn, K., Koay, K., and Walters, M.L. (2009, January 6–9). The negative attitudes towards robots scale and reactions to robot behaviour in a live human-robot interaction study. Proceedings of the 23rd Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour, Edinburgh, UK.
44. Perceived usefulness, perceived ease of use, and user acceptance of information technology10.2307/249008 ↗
45. Design, user experience and usability. Web, mobile, and product design
46. International Organization for Standardization (ISO) (2020, October 05). 9000 Store. ISO 9001 Processes, Procedures and Work Instructions. Available online: https://the9000store.com/iso-9001-2015-requirements/iso-9001-2015-context-of-the-organization/processes-procedures-work-instructions/.
47. Baraglia, M., Cakmak, Y., Nagai, R., Rao, R., and Asada, M. (2016, January 7–10). Initiative in robot assistance during collaborative task execution. In Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand.10.1109/HRI.2016.7451735 ↗
48. Humans and automation: Use, misuse, disuse, abuse10.1518/001872097778543886 ↗
49. Task complexity: Extending a core concept10.5465/amr.2013.0350 ↗
50. Agent-based systems for manufacturing10.1016/j.cirp.2006.10.004 ↗
51. Human-Robot collaborative assembly in cyber-physical production: Classification framework and implementation10.1016/j.cirp.2017.04.101 ↗
52. Issues in human-automation interaction modeling: Presumptive aspects of frameworks of types and levels of automation10.1177/1555343417737203 ↗
53. Confiabilidade e garantia de produto: Visão geral e pesquisas futuras. Product reliability and warranty: An overview and future research
54. Relationship between automation trust and operator performance for the novice and expert in spacecraft rendezvous and docking (RVD)10.1016/j.apergo.2018.03.014 ↗
55. The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task10.1080/1463922021000054335 ↗
56. Brynjolfsson, E., and McAfee, A. (2016). The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies, WW. Norton & Company.
57. Design principles for safety in human-robot interaction10.1007/s12369-010-0052-0 ↗
58. Lenz, A., Skachek, S., Hamann, K., Steinwender, J., Pipe, A.G., and Melhuish, C. (2010, January 6–8). The BERT2 infrastructure: An integrated system for the study of human-robot interaction. Proceedings of the 10th IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA.10.1109/ICHR.2010.5686319 ↗
59. Moniz, A.B., and Krings, B.-J. (2016). Robots Working with humans or humans working with robots? Searching for social dimensions in new human-robot interaction in industry. Societies, 6.10.3390/soc6030023 ↗
60. Moniz, A.B. (2014). Human Resource Management and Technological Challenges, Springer.
61. Gutman, D. (2020). Levels of Automation in Human-Robot Collaboration. [Master’s Thesis, Industrial Engineering and Management, Ben-Gurion University of the Negev].
62. Markfeld, N. (2020). Feedback for Human-Robot Collaboration. [Master’s Thesis, Industrial Engineering and Management, Ben-Gurion University of the Negev].
63. Interactions with robots: The truths we reveal about ourselves10.1146/annurev-psych-010416-043958 ↗
64. Acemoglu, D., and Restrepo, P. (2021, August 09). Demographics and Robots. Available online: http://www.nber.org/papers/w24421.