The Sense of Agency in Human–Machine Interaction Systems
Source abstract
Abstract
Human–Machine Interaction (HMI) systems are integral to various domains and rely on human operators for effective performance. The sense of agency (SoA) is crucial in these systems, as it influences the operator’s concentration and overall efficiency. This review explores the SoA in HMI systems, analyzing its definition, key influencing factors, and methods for enhancement. We provide a comprehensive examination of SoA-related research and suggest strategies for measuring and improving the SoA. Two key research directions are highlighted: the impact of user experience on the SoA, and the role of the SoA in enabling unconscious communication between humans and machines. We propose a development route for HMI systems, outlining a progressive structure across three stages: machine-centric, human-centric, and human–machine integration. Finally, we discuss the potential of gaming platforms as tools for advancing SoA research in HMI systems. Our findings aim to enhance the design and functionality of HMI systems, ensuring improved operator engagement and system performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Free Will and Agency · Neural and Behavioral Psychology Studies · Embodied and Extended Cognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical Engineering, Tshwane University of Technology, Pretoria 0001, South Africa
School of Mechanical and Electrical Engineering, Kunming University, Kunming 650214, China
Funding
National Research Foundations of South Afric · Foundation of Yunnan Province Science and Technology Department · Kunming University Foundation
References (100+)
Show source-provided references
1. MABA-MABA or abracadabra? Progress on human–automation co-ordination10.1007/s101110200022 ↗
2. Bütepage, J., and Kragic, D. (2017). Human-robot collaboration: From psychology to social robotics. arXiv.
3. The out-of-the-loop performance problem and level of control in automation10.1518/001872095779064555 ↗
4. Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety10.1002/prs.680160304 ↗
5. Ibrahim, M.A., Assaad, Z., and Williams, E. (2022). Trust and communication in human-machine teaming. Front. Phys., 10.10.3389/fphy.2022.942896 ↗
6. (2024, July 27). CNN Business Safety Driver in Fatal 2018 Uber Self-Driving Car Crash Found Guilty of Endangerment. Available online: https://edition.cnn.com/2023/07/29/business/uber-self-driving-car-death-guilty/index.html.
7. The out-of-the-loop Brain: A neuroergonomic approach of the human automation interaction10.1016/j.arcontrol.2017.09.010 ↗
8. The sense of control and the sense of agency
9. Bigenwald, A., and Chambon, V. (2019). Criminal responsibility and neuroscience: No revolution yet. Front. Psychol., 10.10.3389/fpsyg.2019.01406 ↗
10. Try and try again: Post-error boost of an implicit measure of agency10.1080/17470218.2017.1350871 ↗
11. A steward robot for human-friendly human-machine interaction in a smart house environment10.1109/TASE.2007.911674 ↗
12. Strausser, K.A., and Kazerooni, H. (2011, January 25–30). The development and testing of a human machine interface for a mobile medical exoskeleton. Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.10.1109/IROS.2011.6095025 ↗
13. Kinematic design to improve ergonomics in human machine interaction10.1109/TNSRE.2006.881565 ↗
14. Current work in the human-machine interface for ergonomic intervention with exoskeletons
15. Kasahara, S., Konno, K., Owaki, R., Nishi, T., Takeshita, A., Ito, T., Kasuga, S., and Ushiba, J. (2017, January 6–11). Malleable embodiment: Changing sense of embodiment by spatial-temporal deformation of virtual human body. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.10.1145/3025453.3025962 ↗
16. Gonzalez-Franco, M., Cohn, B., Ofek, E., Burin, D., and Maselli, A. (2020, January 22–26). The self-avatar follower effect in virtual reality. Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Atlanta, GA, USA.10.1109/VR46266.2020.1580500165557 ↗
17. Barlas, Z., and Obhi, S.S. (2013). Freedom, choice, and the sense of agency. Front. Hum. Neurosci., 7.10.3389/fnhum.2013.00514 ↗
18. Battaglia, F. (2021). Agency, Responsibility, Selves, and the Mechanical Mind. Philosophies, 6.10.3390/philosophies6010007 ↗
19. Coercion changes the sense of agency in the human brain10.1016/j.cub.2015.12.067 ↗
20. Sense of agency in the human brain10.1038/nrn.2017.14 ↗
21. Power to the will: How exerting physical effort boosts the sense of agency10.1016/j.cognition.2013.08.020 ↗
22. Unravelling intention: Distal intentions increase the subjective sense of agency10.1016/j.concog.2013.05.003 ↗
23. Non-motor cues do not generate the perception of self-agency: A critique of cue-integration10.1016/j.concog.2022.103359 ↗
24. Was it me when it happened too early? Experience of delayed effects shapes sense of agency10.1016/j.cognition.2014.11.012 ↗
25. The sense of agency in perception, behaviour and human–machine interactions10.1038/s44159-022-00030-6 ↗
26. Cornelio, P., Haggard, P., Hornbaek, K., Georgiou, O., Bergström, J., Subramanian, S., and Obrist, M. (2022). The sense of agency in emerging technologies for human–computer integration: A review. Front. Neurosci., 16.10.3389/fnins.2022.949138 ↗
27. Limerick, H., Coyle, D., and Moore, J.W. (2014). The experience of agency in human-computer interactions: A review. Front. Hum. Neurosci., 8.10.3389/fnhum.2014.00643 ↗
28. Sahaï, A., Pacherie, E., Grynszpan, O., and Berberian, B. (2017). Predictive mechanisms are not involved the same way during human-human vs. human-machine interactions: A review. Front. Neurorobotics, 11.10.3389/fnbot.2017.00052 ↗
29. Hafner, V.V., Loviken, P., Pico Villalpando, A., and Schillaci, G. (2020). Prerequisites for an artificial self. Front. Neurorobotics, 14.10.3389/fnbot.2020.00005 ↗
30. A bibliometric analysis of materials research in Indonesian journal using VOSviewer
31. Yu, L., and Yu, Z. (2023). Qualitative and quantitative analyses of artificial intelligence ethics in education using VOSviewer and CitNetExplorer. Front. Psychol., 14.10.3389/fpsyg.2023.1061778 ↗
32. Chatila, R., Renaudo, E., Andries, M., Chavez-Garcia, R.O., Luce-Vayrac, P., Gottstein, R., Alami, R., Clodic, A., Devin, S., and Girard, B. (2018). Toward self-aware robots. Front. Robot. AI, 5.10.3389/frobt.2018.00088 ↗
33. Philosophical conceptions of the self: Implications for cognitive science10.1016/S1364-6613(99)01417-5 ↗
34. The sense of agency in joint action: An integrative review10.3758/s13423-021-02051-3 ↗
35. Does delay in feedback diminish sense of agency? A review10.1016/j.concog.2019.05.007 ↗
36. Wen, W., Kuroki, Y., and Asama, H. (2019). The sense of agency in driving automation. Front. Psychol., 10.10.3389/fpsyg.2019.02691 ↗
37. Physical and mental effort disrupts the implicit sense of agency10.1016/j.cognition.2016.08.018 ↗
38. Central cancellation of self-produced tickle sensation10.1038/2870 ↗
39. The cerebellum is involved in predicting the sensory consequences of action10.1097/00001756-200107030-00023 ↗
40. Abnormalities in the awareness of action10.1016/S1364-6613(02)01907-1 ↗
41. Vicarious agency: Experiencing control over the movements of others10.1037/0022-3514.86.6.838 ↗
42. Is the sense of agency in schizophrenia influenced by resting-state variation in self-referential regions of the brain?10.1093/schbul/sbv102 ↗
43. Sense of agency in health and disease: A review of cue integration approaches10.1016/j.concog.2011.08.010 ↗
44. Misattributions of agency in schizophrenia are based on imprecise predictions about the sensory consequences of one’s actions10.1093/brain/awp291 ↗
45. Me or not me—An optimal integration of agency cues?10.1016/j.concog.2009.07.007 ↗
46. Synofzik, M., Vosgerau, G., and Voss, M. (2013). The experience of agency: An interplay between prediction and postdiction. Front. Psychol., 4.10.3389/fpsyg.2013.00127 ↗
47. Gazzaniga, M. (2004). Authorship Processing. Cognitive Neurosciences III, MIT Press.
48. Modulating the sense of agency with external cues10.1016/j.concog.2009.05.004 ↗
49. Neurology of volition10.1007/s00221-013-3399-2 ↗
50. Conscious intention and motor cognition10.1016/j.tics.2005.04.012 ↗
51. Wen, W., Muramatsu, K., Hamasaki, S., An, Q., Yamakawa, H., Tamura, Y., Yamashita, A., and Asama, H. (2016). Goal-directed movement enhances body representation updating. Front. Hum. Neurosci., 10.10.3389/fnhum.2016.00329 ↗
52. Minohara, R., Wen, W., Hamasaki, S., Maeda, T., Kato, M., Yamakawa, H., Yamashita, A., and Asama, H. (2016). Strength of intentional effort enhances the sense of agency. Front. Psychol., 7.10.3389/fpsyg.2016.01165 ↗
53. Caspar, E.A., Cleeremans, A., and Haggard, P. (2018). Only giving orders? An experimental study of the sense of agency when giving or receiving commands. PLoS ONE, 13.10.1371/journal.pone.0204027 ↗
54. Preoccupied minds feel less control: Sense of agency is modulated by cognitive load10.1016/j.concog.2013.03.004 ↗
55. Beyond one’s body parts: Remote object movement with sense of agency involuntarily biases spatial attention10.3758/s13423-018-1552-4 ↗
56. Attention and the control of posture and gait: A review of an emerging area of research10.1016/S0966-6362(01)00156-4 ↗
57. Posture control, aging, and attention resources: Models and posture-analysis methods10.1016/j.neucli.2008.09.005 ↗
58. Kannape, O.A., Barré, A., Aminian, K., and Blanke, O. (2014). Cognitive loading affects motor awareness and movement kinematics but not locomotor trajectories during goal-directed walking in a virtual reality environment. PLoS ONE, 9.10.1371/journal.pone.0085560 ↗
59. Conscious cognitive effort in cognitive control10.1002/wcs.1629 ↗
60. Functional neuroanatomy of altered states of consciousness: The transient hypofrontality hypothesis10.1016/S1053-8100(02)00046-6 ↗
61. Endurance exercise selectively impairs prefrontal-dependent cognition10.1016/j.bandc.2004.03.002 ↗
62. Flexible cognitive resources: Competitive content maps for attention and memory10.1016/j.tics.2013.01.010 ↗
63. Howard, E. (2016). The Effect of Effort and Individual Differences on the Implicit Sense of Agency. [Ph.D. Thesis, University of East Anglia].
64. How cognitive load affects duration judgments: A meta-analytic review10.1016/j.actpsy.2010.03.006 ↗
65. Nataraj, R., Hollinger, D., Liu, M., and Shah, A. (2020). Disproportionate positive feedback facilitates sense of agency and performance for a reaching movement task with a virtual hand. PLoS ONE, 15.10.1371/journal.pone.0233175 ↗
66. Metacognition of agency10.1037/0096-3445.136.2.184 ↗
67. Wen, W., Yamashita, A., and Asama, H. (2015). The sense of agency during continuous action: Performance is more important than action-feedback association. PLoS ONE, 10.10.1371/journal.pone.0125226 ↗
68. Sense of agency in continuous action: Assistance-induced performance improvement is self-attributed even with knowledge of assistance10.1016/j.concog.2016.12.003 ↗
69. The sense of agency during skill learning in individuals and dyads10.1016/j.concog.2012.04.001 ↗
70. The ‘problem’ with automation: Inappropriate feedback and interaction, not ‘over-automation’10.1098/rstb.1990.0101 ↗
71. Berberian, B., Sarrazin, J.C., Le Blaye, P., and Haggard, P. (2012). Automation technology and sense of control: A window on human agency. PLoS ONE, 7.10.1371/journal.pone.0034075 ↗
72. Yun, S., Wen, W., An, Q., Hamasaki, S., Yamakawa, H., Tamura, Y., Yamashita, A., and Asama, H. (2018, January 16–20). Investigating the relationship between assisted driver’s SoA and EEG. Proceedings of the International Conference on NeuroRehabilitation, Pisa, Italy.10.1007/978-3-030-01845-0_208 ↗
73. The sense of agency in human-human vs human-robot joint action10.1016/j.concog.2019.102820 ↗
74. Action co-representation and the sense of agency during a joint Simon task: Comparing human and machine co-agents10.1016/j.concog.2018.11.008 ↗
75. Zanatto, D., Chattington, M., and Noyes, J. (2021). Sense of agency in human-machine interaction. Advances in Neuroergonomics and Cognitive Engineering: Proceedings of the AHFE 2021 Virtual Conferences on Neuroergonomics and Cognitive Engineering, Industrial Cognitive Ergonomics and Engineering Psychology, and Cognitive Computing and Internet of Things, New York, NY, USA, 25–29 July 2021, Springer.10.1007/978-3-030-80285-1_41 ↗
76. Albutihe, I. (2024, January 15). Sense of Agency and Automation: A Systematic Review. Available online: https://www.diva-portal.org/smash/record.jsf?pid=diva2:1791640.
77. Wen, W., Yun, S., Yamashita, A., Northcutt, B.D., and Asama, H. (2021). Deceleration assistance mitigated the trade-off between sense of agency and driving performance. Front. Psychol., 12.10.3389/fpsyg.2021.643516 ↗
78. Leveraging human agency to improve confidence and acceptability in human-machine interactions10.1016/j.cognition.2022.105020 ↗
79. Transparency in human-machine mutual action10.20965/jrm.2021.p0987 ↗
80. Stability and transparency in bilateral teleoperation10.1109/70.258054 ↗
81. Agency modulates interactions with automation technologies10.1080/00140139.2018.1468493 ↗
82. Defective recognition of one’s own actions in patients with schizophrenia10.1176/appi.ajp.158.3.454 ↗
83. Illusion of sense of self-agency: Discrepancy between the predicted and actual sensory consequences of actions modulates the sense of self-agency, but not the sense of self-ownership10.1016/j.cognition.2004.04.003 ↗
84. Nakashima, R., and Kumada, T. (2020). Explicit sense of agency in an automatic control situation: Effects of goal-directed action and the gradual emergence of outcome. Front. Psychol., 11.10.3389/fpsyg.2020.02062 ↗
85. When dyads act in parallel, a sense of agency for the auditory consequences depends on the order of the actions10.1016/j.concog.2012.12.004 ↗
86. Abnormalities in the awareness and control of action10.1098/rstb.2000.0734 ↗
87. Voluntary action and conscious awareness10.1038/nn827 ↗
88. The self in action effects: Selective attenuation of self-generated sounds10.1016/j.cognition.2011.06.011 ↗
89. Bays, P.M., Flanagan, J.R., and Wolpert, D.M. (2006). Attenuation of self-generated tactile sensations is predictive, not postdictive. PLoS Biol., 4.10.1371/journal.pbio.0040028 ↗
90. Internalizing agency of self-action: Perception of one’s own hand movements depends on an adaptable prediction about the sensory action outcome10.1152/jn.00104.2006 ↗
91. Action effect anticipation: Neurophysiological basis and functional consequences10.1016/j.neubiorev.2011.11.004 ↗
92. A preactivation account of sensory attenuation10.1016/j.neuropsychologia.2013.02.005 ↗
93. Body ownership determines the attenuation of self-generated tactile sensations10.1073/pnas.1703347114 ↗
94. Readiness-potentials preceding unrestricted “spontaneous” vs. pre-planned voluntary acts10.1016/0013-4694(82)90181-X ↗
95. Intentional binding coincides with explicit sense of agency10.1016/j.concog.2018.11.005 ↗
96. Scalar timing in memory10.1111/j.1749-6632.1984.tb23417.x ↗
97. Temporal discrimination and the indifference interval: Implications for a model of the “internal clock”10.1037/h0093864 ↗
98. Cortico-striatal circuits and interval timing: Coincidence detection of oscillatory processes10.1016/j.cogbrainres.2004.06.012 ↗
99. Saccadic eye movements cause compression of time as well as space10.1038/nn1488 ↗
100. How voluntary actions modulate time perception10.1007/s00221-009-1848-8 ↗