Smart Interactive Technologies in the Human-Centric Factory 5.0: A Survey
Source abstract
Abstract
In this survey paper, we focus on smart interactive technologies and providing a picture of the current state of the art, exploring the way new discoveries and recent technologies changed workers’ operations and activities on the factory floor. We focus in particular on the Industry 4.0 and 5.0 visions, wherein smart interactive technologies can bring benefits to the intelligent behavior machines can expose in a human-centric AI perspective. We consider smart technologies wherein the intelligence may be in and/or behind the user interfaces, and for both groups we try to highlight the importance of designing them with a human-centric approach, framed in the smart factory context. We review relevant work in the field with the aim of highlighting the pros and cons of each technology and its adoption in the industry. Furthermore, we try to collect guidelines for the human-centric integration of smart interactive technologies in the smart factory. In this wa y, we hope to provide the future designers and adopters of such technologies with concrete help in choosing among different options and implementing them in a user-centric manner. To this aim, surveyed works have been also classified based on the supported task(s) and production process phases/activities: access to knowledge, logistics, maintenance, planning, production, security, workers’ wellbeing, and warehousing.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Digital Transformation in Industry · Augmented Reality Applications · IoT and Edge/Fog Computing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science Department, University of Turin, 10149 Turin, Italy
Funding
Regione Piemonte
References (100+)
Show source-provided references
1. Implementing Smart Factory of Industrie 4.0: An Outlook
2. Towards Socio-Cyber-Physical Systems in Production Networks10.1016/j.procir.2013.05.009 ↗
3. Directorate-General for Research and Innovation European Commission, Breque, M., De Nul, L., and Petridis, A. (2021). Industry 5.0: Towards a Sustainable, Human-Centric and Resilient European Industry.
4. Directorate-General for Research and Innovation European Commission, and Müller, J. (2020). Enabling Technologies for Industry 5.0—Results of a Workshop with Europe’s Technology Leaders.
5. Industry 4.0 and Industry 5.0—Inception, conception and perception10.1016/j.jmsy.2021.10.006 ↗
6. Sonntag, D. (2017). Intelligent User Interfaces—A Tutorial. arXiv.
7. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy10.1080/10447318.2020.1741118 ↗
8. Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-Centered AI Systems10.1145/3419764 ↗
9. Costabile, M.F., Gena, C., Matera, M., Paternò, F., Tortora, G., and Zancanaro, M. (2022, July 29). Teaching HCI for AI: Co-Design of a Syllabus. Final Report by the Workshop Organizers. Available online: http://sigchitaly.eu/en/hci4ai-syllabus-it/workshop-results/.
10. Charade: Remote Control of Objects Using Free-hand Gestures10.1145/159544.159562 ↗
11. Blumendorf, M., Feuerstack, S., and Albayrak, S. (2008, January 28–30). Multimodal User Interfaces for Smart Environments: The Multi-Access Service Platform. Proceedings of the Working Conference on Advanced Visual Interfaces, AVI’08, Napoli, Italy.10.1145/1385569.1385665 ↗
12. Dobbie, G., Frank, U., Kappel, G., Liddle, S.W., and Mayr, H.C. (2020, January 3). Modeling Interactive Smart Spaces. Proceedings of the Conceptual Modeling, Vienna, Austria.10.1007/978-3-030-62522-1 ↗
13. Tan, D.S., Amershi, S., Begole, B., Kellogg, W.A., and Tungare, M. (2011, January 7–12). How can we support users’ preferential choice?. Proceedings of the International Conference on Human Factors in Computing Systems, CHI 2011, Extended Abstracts Volume, Vancouver, BC, Canada.
14. Jannach, D., Zanker, M., Felfernig, A., and Friedrich, G. (2010). Recommender Systems—An Introduction, Cambridge University Press.10.1017/CBO9780511763113 ↗
15. Gena, C., Grillo, P., Lieto, A., Mattutino, C., and Vernero, F. (2019). When Personalization Is Not an Option: An In-The-Wild Study on Persuasive News Recommendation. Information, 10.10.3390/info10100300 ↗
16. A review of recent advances in learner and skill modeling in intelligent learning environments10.1007/s11257-011-9106-8 ↗
17. Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications10.1016/j.mechatronics.2018.02.009 ↗
18. Enabling personalisation of remote elderly assistance10.1007/s11042-019-7449-z ↗
19. Big data and IoT-based applications in smart environments: A systematic review10.1016/j.cosrev.2020.100318 ↗
20. Mobile Service Technician 4.0: Knowledge-Sharing Solutions for Industrial Field Maintenance10.55612/s-5002-038-001 ↗
21. A New Human-centric Factory Model10.1016/j.procir.2014.07.112 ↗
22. Benedetto, F., Brunetti, D., Gena, C., Lai, M., Meo, R., and Vernero, F. (2020, January 17–20). Intelligent monitoring applications for Industry 4.0. Proceedings of the IUI ’20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy.10.1145/3379336.3381484 ↗
23. Andolina, S., Ariano, P., Brunetti, D., Celadon, N., Coppo, G., Favetto, A., Gena, C., Giordano, S., and Vernero, F. (2019, January 6–9). Experimenting with Large Displays and Gestural Interaction in the Smart Factory. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019, Bari, Italy.10.1109/SMC.2019.8913900 ↗
24. Introducing Gestural Interaction on the Shop Floor: Empirical Evaluations10.1007/978-3-030-85607-6_56 ↗
25. Krupitzer, C., Müller, S., Lesch, V., Züfle, M., Edinger, J., Lemken, A., Schäfer, D., Kounev, S., and Becker, C. (2020). A Survey on Human Machine Interaction in Industry 4.0. arXiv.
26. Manufacturing in the cloud: A human factors perspective10.1016/j.ergon.2016.05.011 ↗
27. Human-Computer Interactions and User Interfaces for Remote Control of Manufacturing Systems
28. Gorecky, D., Schmitt, M., Loskyll, M., and Zühlke, D. (2014, January 27–30). Human-machine-interaction in the industry 4.0 era. Proceedings of the 2014 12th IEEE International Conference on Industrial Informatics (INDIN), Porto Alegre, Brazil.10.1109/INDIN.2014.6945523 ↗
29. Aehnelt, M., Klamma, R., and Pammer, V. (2019). Human Computer Interaction Perspectives on Industry 4.0. Interact. Des. Archit., 38, Available online: https://www.researchgate.net/publication/332466625_Human_Computer_Interaction_Perspectives_on_Industry_40_Interaction_Design_Architectures_Vol_38.10.55612/s-5002-038-001fsi ↗
30. Rethinking Human-Machine Learning in Industry 4.0: How Does the Paradigm Shift Treat the Role of Human Learning?10.1016/j.promfg.2018.04.003 ↗
31. Deneen, K. (2022, July 29). Human-Machine Interface Technologies: What Impact on Industry 4.0?. Available online: https://medium.com/astercapital/human-machine-interface-technologies-what-impact-on-industry-4-0-6a105f97529d/.
32. Production logistics and human-computer interaction—State-of-the-art, challenges and requirements for the future10.1007/s00170-019-03785-0 ↗
33. Ras, E., Wild, F., Stahl, C., and Baudet, A. (2017, January 21–23). Bridging the Skills Gap of Workers in Industry 4.0 by Human Performance Augmentation Tools: Challenges and Roadmap. Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA’17, Island of Rhodes, Greece.10.1145/3056540.3076192 ↗
34. Merz, E. (2022, July 29). Industrie 4.0: Usability Design for the Industry of Tomorrow. Available online: https://www.amnytt.no/getfile.php/2875229.2265.rtbaurxvwt/Usability_Industrie_4.0_EN.pdf.
35. Deloitte (2022, July 29). The Industry 4.0 Paradox. Available online: https://www2.deloitte.com/global/en/pages/energy-and-resources/articles/the-industry-4-0-paradox.html.
36. Romero, D., Stahre, J., Wuest, T., Noran, O., Bernus, P., Fast-Berglund, Å., and Gorecky, D. (2016, January 29–31). Towards an Operator 4.0 Typology: A Human-Centric Perspective on the Fourth Industrial Revolution Technologies. Proceedings of the International Conference on Computers & Industrial Engineering (CIE46), Tianjin, China.
37. Interaction with Large Displays: A Survey10.1145/2682623 ↗
38. Sambrooks, L., and Wilkinson, B. (2013, January 25–29). Comparison of Gestural, Touch, and Mouse Interaction with Fitts’ Law. Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, OzCHI’13, Adelaide, Australia.10.1145/2541016.2541066 ↗
39. Lacueva-Pérez, F.J., Khakurel, J., Brandl, P., Hannola, L., Gracia-Bandrés, M.A., and Schafler, M. (2018, January 26–29). Assessing TRL of HCI Technologies Supporting Shop Floor Workers. Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference, PETRA ’18, Corfu, Greece.10.1145/3197768.3203175 ↗
40. Gestural interaction with 3D objects shown on public displays: An elicitation study
41. Meixner, G., Petersen, N., and Koessling, H. (2010, January 6–10). User Interaction Evolution in the SmartFactoryKL. Proceedings of the 24th BCS Interaction Specialist Group Conference, BCS ’10, Dundee, UK.10.14236/ewic/HCI2010.27 ↗
42. Garzotto, F., and Valoriani, M. (2013, January 16–20). Touchless Gestural Interaction with Small Displays: A Case Study. Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI, Trento, Italy.10.1145/2499149.2499154 ↗
43. From Space to Stage: How Interactive Screens Will Change Urban Life10.1109/MC.2011.135 ↗
44. Natural User Interfaces Are Not Natural10.1145/1744161.1744163 ↗
45. Horejsi, P., Polcar, J., and Rohlíková, L. (2016). Digital Factory and Virtual Reality: Teaching Virtual Reality Principles with Game Engines, IntechOpen Limited.10.5772/65218 ↗
46. Angelini, L., Mugellini, E., Abou Khaled, O., and Couture, N. (2018). Internet of Tangible Things (IoTT): Challenges and Opportunities for Tangible Interaction with IoT. Informatics, 5.10.3390/informatics5010007 ↗
47. Lutsenko, Y.V. (2016). Shine and Poverty of Virtual Reality. Polythematic Online Sci. J. Kuban State Agrar. Univ., 124.10.21515/1990-4665-124-001 ↗
48. Facility Layout Planning: A Review10.15680/IJIRSET.2015.0403027 ↗
49. Development of virtual reality support to factory layout planning10.1007/s12008-019-00538-x ↗
50. Collaborative Factory Planning in Virtual Reality10.1016/j.procir.2012.07.055 ↗
51. The Current and Future Challenges for Virtual Commissioning and Digital Twins of Production Lines
52. Survey on the virtual commissioning of manufacturing systems
53. Rasheed, G., Khan, M., Malik, N., and Akhunzada, A. (2021). Measuring Learnability through Virtual Reality Laboratory Application: A User Study. Sustainability, 13.10.3390/su131910812 ↗
54. Augmented reality: A class of displays on the reality-virtuality continuum10.1117/12.197321 ↗
55. Speicher, M., Hall, B., and Nebeling, M. (2019, January 4–9). What is Mixed Reality?. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), Glasgow, Scotland, UK.10.1145/3290605.3300767 ↗
56. Perez, F.J.L., Brandl, P., and Bandres, M.A.G. (2006). Technolgy Monitoring: Report on Information Needed for Workers in the Smart Factory, FACTS4WORKERS (Worker Centric Workplace in Smart Factories). Technical Report.
57. Perey, C., Wild, F., Helin, K., Janak, M., Davies, P., and Ryan, P. (2014, January 10–12). Advanced manufacturing with augmented reality. Proceedings of the 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.10.1109/ISMAR.2014.6948518 ↗
58. Towards augmented reality manuals for industry 4.0: A methodology10.1016/j.rcim.2018.10.001 ↗
59. Deac, C., Popa, C.L., Ghinea, M., and Cotet, C. (2017, January 8–11). Using Augmented Reality in Smart Manufacturing. Proceedings of the 28th DAAAM International Symposium, Zadar, Croatia.10.2507/28th.daaam.proceedings.102 ↗
60. Augmented Reality Smart Glasses in the Smart Factory: Product Evaluation Guidelines and Review of Available Products10.1109/ACCESS.2017.2703952 ↗
61. Antecedents to the adoption of augmented reality smart glasses: A closer look at privacy risks10.1016/j.jbusres.2018.08.008 ↗
62. Ashton, K. (2022, July 29). That Internet of Things Thing: In the Real World Things Matter More than Ideas. Available online: https://www.rfidjournal.com/that-internet-of-things-thing.
63. The Internet of Things: A survey10.1016/j.comnet.2010.05.010 ↗
64. On the way from Industry 4.0 to Industry 5.0: From digital manufacturing to digital society
65. Sagirlar, G., Carminati, B., and Ferrari, E. (2018). Decentralizing Privacy Enforcement for Internet of Things Smart Objects. arXiv.10.1016/j.comnet.2018.07.019 ↗
66. Wehle, H.D. (2022, July 29). Augmented Reality and the Internet of Things (IoT)/Industry 4.0. Available online: https://www.researchgate.net/profile/Hans-Dieter-Wehle/publication/288642701_Augmented_Reality_and_the_Internet_of_Things_IoT_Industry_40_en/links/5682703308aebccc4e0df03f/Augmented-Reality-and-the-Internet-of-Things-IoT-Industry-40-en.pdf.
67. A Review of Research Relevant to the Emerging Industry Trends: Industry 4.0, IoT, Block Chain, and Business Analytics10.1142/S2424862219500192 ↗
68. Fatima, Z., Tanveer, M.H., Zardari, S., Naz, L.F., Khadim, H., Ahmed, N., and Tahir, M. (2022). Production Plant and Warehouse Automation with IoT and Industry 5.0. Appl. Sci., 12.10.3390/app12042053 ↗
69. Internet of things and supply chain management: A literature review10.1080/00207543.2017.1402140 ↗
70. Vaccaro, L., Sansonetti, G., and Micarelli, A. (2021). An Empirical Review of Automated Machine Learning. computers, 10.10.3390/computers10010011 ↗
71. Internet of Things (IOT) for Manufacturing Logistics on SAP ERP Applications
72. A Survey on Wireless Indoor Localization from the Device Perspective
73. Case Study of BIM and Cloud–Enabled Real-Time RFID Indoor Localization for Construction Management Applications10.1061/(ASCE)CO.1943-7862.0001125 ↗
74. RFID Localisation For Internet Of Things Smart Homes: A Survey10.5121/ijcnc.2017.9107 ↗
75. Integrating Internet of Things (IoT) into Enterprise Organizations: Socio-Technical Issues and Guidelines
76. A Survey on Challenges of Software Project Management
77. European Commission, and EPoSS (2022, July 29). Internet of Things in 2020 Roadmap for the Future. Available online: https://docbox.etsi.org/erm/Open/CERP%2020080609-10/Internet-of-Things_in_2020_EC-EPoSS_Workshop_Report_2008_v1-1.pdf.
78. Interaction Methods for Smart Glasses: A Survey10.1109/ACCESS.2018.2831081 ↗
79. Zhou, J., Zhang, Y., Laput, G., and Harrison, C. (2016, January 16–19). AuraSense: Enabling Expressive Around-Smartwatch Interactions with Electric Field Sensing. Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan.10.1145/2984511.2984568 ↗
80. Quint, F., Loch, F., Weber, H., Venitz, J., Gröber, M., and Liedel, J. (2016, January 4–7). Evaluation of Smart Glasses for Documentation in Manufacturing. Proceedings of the Mensch und Computer 2016—Workshopband, Aachen, Germany.
81. Ishii, H. (2003, January 12–15). Tangible Bits: Designing the Seamless Interface between People, Bits, and Atoms. Proceedings of the 8th International Conference on Intelligent User Interfaces, IUI’03, Miami, FL, USA.10.1145/604046.604048 ↗
82. Angelini, L., Mugellini, E., Lechelt, Z., Hornecker, E., Marshall, P., Liu, C., Brereton, M., Soro, A., Couture, N., and Abou Khaled, O. (2018, January 21–26). Internet of Tangible Things: Workshop on Tangible Interaction with the Internet of Things. Proceedings of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3170427.3170634 ↗
83. Funk, M., Korn, O., and Schmidt, A. (May, January 26). An Augmented Workplace for Enabling User-Defined Tangibles. Proceedings of the CHI’14: CHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada.
84. Exploring Tangible Interactions with Radar Sensing10.1145/3287078 ↗
85. Wallbaum, T., Matviienko, A., Heuten, W., and Boll, S. (2017, January 19–23). Challenges For Designing Tangible Systems. Proceedings of the 3rd European Tangible Interaction Studio (ETIS 2017), Luxembourg.
86. Colgate, J., Wannasuphoprasit, W., and Peshkin, M. (1996, January 17–22). Cobots: Robots for collaboration with human operators. Proceedings of the 1996 ASME International Mechanical Engineering Congress and Exposition, Atlanta, GA, USA.10.1115/IMECE1996-0367 ↗
87. Silverman, R.E. (2022, July 29). The Words of Tomorrow. Available online: https://www.wsj.com/articles/SB944517141695981261/.
88. Vicentini, F. (2017). La Robotica Collaborativa. Sicurezza e Flessibilità Delle Nuove Forme di Collaborazione Uomo-Robot, Tecniche Nuove.
89. Linsinger, M., Sudhoff, M., Lemmerz, K., Glogowski, P., and Kuhlenkötter, B. (2018). Task-based Potential Analysis for Human-Robot Collaboration within Assembly Systems. Tagungsband des 3. Kongresses Montage Handhabung Industrieroboter, Springer.10.1007/978-3-662-56714-2_1 ↗
90. Human Robot Interaction—Learning how to integrate collaborative robots into manual assembly lines10.1016/j.promfg.2019.03.005 ↗
91. Collaborative Robots in e-waste Management10.1016/j.promfg.2017.07.133 ↗
92. Gesture-based human-robot interaction for human assistance in manufacturing10.1007/s00170-018-2788-x ↗
93. Spada, S., Ghibaudo, L., Gilotta, S., Gastaldi, L., and Cavatorta, M.P. (2017). Analysis of exoskeleton introduction in industrial reality: Main issues and EAWS risk assessment. International Conference on Applied Human Factors and Ergonomics, Springer.10.1007/978-3-319-60825-9_26 ↗
94. Fleck, P. (2022, July 29). Collaborative Robot. Available online: https://www.researchgate.net/publication/313665204_Collaborative_Robots?channel=doi&linkId=58a21ad545851598babae8ff&showFulltext=true.
95. Manso, L., Bustos, P., Bandera, J., Romero-Garcés, A., Calderita, L., Marfil, R., and Bandera, A. (2016). Deep Representations for Collaborative Robotics, Springer.10.1007/978-3-319-50862-7_14 ↗
96. Lambrecht, J., and Kruger, J. (2012, January 7–12). Spatial programming for industrial robots based on gestures and Augmented Reality. Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.10.1109/IROS.2012.6385900 ↗
97. Landi, C.T., Ferraguti, F., Secchi, C., and Fantuzzi, C. (2016, January 23–26). Tool compensation in walk-through programming for admittance-controlled robots. Proceedings of the IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society, Florence, Italy.10.1109/IECON.2016.7793038 ↗
98. Siciliano, B., and Khatib, O. (2016). Learning from Humans. Springer Handbook of Robotics, Springer International Publishing.10.1007/978-3-319-32552-1 ↗
99. Control Techniques for Safe, Ergonomic, and Efficient Human-Robot Collaboration in the Digital Industry: A Survey10.1109/TASE.2021.3131011 ↗
100. Collaborating robots in Industry 4.0 conception10.1088/1757-899X/448/1/012023 ↗