journal article·2026-07-09Open access

Adaptive Human Oversight for Maritime Agentic AI Systems: Balancing Operator Workload and Safety Through Risk-Aware Governance

Tymoteusz Miller, Irmina Durlik, Paweł Biczak

Applied Sciences· Volume 16· Issue 14

Abstract

The growing adoption of artificial intelligence and autonomous decision-making systems in maritime operations introduces significant challenges related to safety, accountability, and human oversight. Although Human-in-the-Loop approaches are commonly proposed to maintain human control over autonomous systems, continuous supervision is often impractical due to cognitive workload, operator fatigue, alert saturation, and scalability constraints. This study introduces an Adaptive Human Oversight framework for Maritime Agentic AI Systems, extending the Controlled Agentic AI Systems paradigm with a risk-aware, constraint-preserving, and auditable human governance layer. The framework employs a two-stage risk mechanism in which hard safety conditions, including critical loss of separation, boundary violations, infeasible actions, and excessive speed conditions, override the weighted composite risk score and trigger human oversight or fallback behavior independently of activation thresholds. Under elevated but non-critical conditions, a Composite Risk Assessment Module regulates human activation using interpretable indicators of separation, congestion, speed, and uncertainty. The framework also defines the behavioral semantics of human oversight actions, ensuring that approval, modification, override, and rejection remain compatible with governance constraints before execution. To reduce unnecessary supervisory burden, the framework incorporates hysteresis and trend-aware cooldown mechanisms that suppress redundant repeated requests while preserving responsiveness to critical safety events. Simulation experiments conducted using the MARIS-AI platform evaluated risk thresholds, CRAM weight sensitivity, cooldown diagnostics, operator reliability, intervention delay, and stress-test scenarios. Results show that threshold tuning primarily regulates non-critical elevated-risk states, while critical states are governed by hard safety overrides. Trend-aware cooldown reduced intervention frequency without suppressing critical safety events, whereas operator reliability and intervention timeliness strongly determined safety outcomes. The findings suggest that effective maritime human oversight should rely on hard safety constraints, interpretable risk assessment, timely human activation, workload-aware cooldown, and auditable decision traces rather than continuous monitoring alone. The proposed framework provides a pathway toward scalable, trustworthy, and accountable human–AI collaboration in maritime agentic AI systems.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Human-Automation Interaction and Safety · Maritime Navigation and Safety · Occupational Health and Safety Research

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Institute of Marine and Environmental Sciences, University of Szczecin, 70-364 Szczecin, Poland

Faculty of Data Science and Information, INTI International University, Nilai 71800, Negeri Sembilan, Malaysia

Faculty of Navigation, Maritime University of Szczecin, 70-500 Szczecin, Poland

Polish Society of Bioinformatics and Data Science Biodata, 71-214 Szczecin, Poland

References (37)

Show source-provided references
1. Miller, T. (2026). Controlled Agentic AI Systems: A Governance-Driven Architecture for Auditable and Reproducible Decision Pipelines. Mach. Learn. Knowl. Extr., 8.10.20944/preprints202603.1904.v1 ↗
2. A systematic review of human–AI interaction in autonomous ship systems10.1016/j.ssci.2022.105778 ↗
3. AIS-based maritime anomaly traffic detection: A review10.1016/j.eswa.2023.120561 ↗
4. Unsupervised maritime anomaly detection for intelligent situational awareness using AIS data10.1016/j.knosys.2023.111313 ↗
5. A model for types and levels of human interaction with automation10.1109/3468.844354 ↗
6. Improving decision transparency in autonomous maritime collision avoidance10.1007/s00773-024-01043-x ↗
7. Xue, J., Yang, P., Li, Q., Song, Y., Gelder, P.V., Papadimitriou, E., and Hu, H. (2025). Machine Learning in Maritime Safety for Autonomous Shipping: A Bibliometric Review and Future Trends. J. Mar. Sci. Eng., 13.10.3390/jmse13040746 ↗
8. Durlik, I., Miller, T., Kostecka, E., Kozlovska, P., and Ślączka, W. (2025). Enhancing Safety in Autonomous Maritime Transportation Systems with Real-Time AI Agents. Appl. Sci., 15.10.3390/app15094986 ↗
9. A state-of-the-art review of AI decision transparency for autonomous shipping
10. Human factor influences on supervisory control of remotely operated and autonomous vessels10.1016/j.oceaneng.2024.117257 ↗
11. Towards the development of a system-theoretic model for safety assessment of autonomous merchant vessels10.1016/j.ress.2018.05.019 ↗
12. Supporting human–AI teams: Transparency, explainability, and situation awareness10.1016/j.chb.2022.107574 ↗
13. The influence of agent transparency and complexity on situation awareness, mental workload, and task performance10.1177/15553434241240553 ↗
14. European Parliament, and Council of the European Union (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act)
15. European Parliament, and Council of the European Union (2024). Regulation
16. (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act); L 2024/1689; Official Journal of the European Union: Luxembourg. Available online: https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng.
17. Raza, M.S., Murtaza, M., Cheng, C.T., Muslam, M., and Albahlal, B.M. (2024). Systematic review of cognitive impairment in drivers through mental workload using physiological measures of heart rate variability. Front. Comput. Neurosci., 18.10.3389/fncom.2024.1475530 ↗
18. Cognitive task analysis and workload classification10.1016/j.mex.2021.101235 ↗
19. Xie, W., Gang, L., Zhang, M., Liu, T., and Lan, Z. (2024). Optimizing Multi-Vessel Collision Avoidance Decision Making for Autonomous Surface Vessels: A COLREGs-Compliant Deep Reinforcement Learning Approach. J. Mar. Sci. Eng., 12.10.3390/jmse12030372 ↗
20. Toward a theory of situation awareness in dynamic systems10.1518/001872095779049543 ↗
21. Zhang, L., Ge, J., Goerlandt, F., Du, L., Chen, T., Gu, T., Gan, L., and Li, X. (2025). A Method for Enhancing the Traffic Situation Awareness of Vessel Traffic Service Operators by Identifying High Risk Ships in Complex Navigation Conditions. J. Mar. Sci. Eng., 13.10.3390/jmse13020379 ↗
22. Toward COLREGs-aware ship collision avoidance with multi-agent PPO–LSTM in maritime IoT10.1016/j.jnca.2026.104511 ↗
23. Guan, W., Zhao, M., Zhang, C., and Xi, Z. (2023). Generalized behavior decision-making model for ship collision avoidance via reinforcement learning method. J. Mar. Sci. Eng., 11.10.3390/jmse11020273 ↗
24. Future of maritime autonomy: Cybersecurity, trust and mariner’s situational awareness10.1080/20464177.2024.2330176 ↗
25. Human–machine cooperation research for navigation of maritime autonomous surface ships: A review and consideration10.1016/j.oceaneng.2022.110555 ↗
26. Alert fatigue in security operations centres: Research challenges and opportunities
27. Accountability in artificial intelligence: What it is and how it works10.1007/s00146-023-01635-y ↗
28. What do we want from Explainable Artificial Intelligence (XAI)? A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research10.1016/j.artint.2021.103473 ↗
29. Langer, M., Lazar, V., and Baum, K. (2025). On the complexities of testing for compliance with human oversight requirements in AI regulation. arXiv.10.1007/978-3-032-07132-3_11 ↗
30. Sterz, S., Baum, K., Biewer, S., Hermanns, H., Lauber-Ronsberg, A., Meinel, P., and Langer, M. (2024). On the quest for effectiveness in human oversight: Interdisciplinary perspectives. Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24), Rio de Janeiro, Brazil, 3–6 June 2024, Association for Computing Machinery (ACM).10.1145/3630106.3659051 ↗
31. A trustable architecture over blockchain to facilitate maritime administration for MASS systems10.1016/j.ress.2021.108246 ↗
32. Lee, Y.-G., Lee, C.-H., Jeon, Y.-H., and Bae, J.-H. (2024). Transformative Impact of the EU AI Act on Maritime Autonomous Surface Ships. Laws, 13.10.3390/laws13050061 ↗
33. Miller, T. (2026, June 08). CAIS-pub: Simulation codebase for Controlled Agentic AI Systems Experiments. GitHub. Available online: https://github.com/TyMill/CAIS-pub.
34. Choi, J., and Kim, S. (2025). Predictive Risk-Aware Reinforcement Learning for Autonomous Vehicles Using Safety Potential. Electronics, 14.10.3390/electronics14224446 ↗
35. Fenton, A., and Chapsos, I. (2023). Ships without crews: IMO and UK responses to cybersecurity, technology, law and regulation of maritime autonomous surface ships (MASS). Front. Comput. Sci., 5.10.3389/fcomp.2023.1151188 ↗
36. Explainable AI for ship collision avoidance: Decoding decision-making processes and behavioral intentions10.1016/j.apor.2025.104471 ↗
37. Risk-aware autonomous search and rescue with multiagent reinforcement learning10.1111/risa.70067 ↗