journal article·2025-06-17Open access

Resilient AI in Therapeutic Rehabilitation: The Integration of Computer Vision and Deep Learning for Dynamic Therapy Adaptation

Egidia Cirillo, Claudia Conte, Alberto Moccardi, Mattia Fonisto

Applied Sciences· Volume 15· Issue 12

Abstract

Resilient artificial intelligence (Resilient AI) is relevant in many areas where technology needs to adapt quickly to changing and unexpected conditions, such as in the medical, environmental, security, and agrifood sectors. In the case study involving the therapeutic rehabilitation of patients with motor problems, the Resilient AI system is crucial to ensure that systems can effectively respond to changes, maintain high performance, cope with uncertainties and complex variables, and enable the dynamic monitoring and adaptation of therapy in real time. The proposed system integrates advanced technologies, such as computer vision and deep learning models, focusing on non-invasive solutions for monitoring and adapting rehabilitation therapies. The system combines the Microsoft Kinect v3 sensor with MoveNet Thunder – SinglePose, a state-of-the-art deep-learning model for human pose estimation. Kinect’s 3D skeletal tracking and MoveNet’s high-precision 2D keypoint detection together improve the accuracy and reliability of postural analysis. The main objective is to develop an intelligent system that captures and analyzes a patient’s movements in real time using Motion Capture techniques and artificial intelligence (AI) models to improve the effectiveness of therapies. Computer vision tracks human movement, identifying crucial biomechanical parameters and improving the quality of rehabilitation.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Stroke Rehabilitation and Recovery · Anomaly Detection Techniques and Applications · Medical Imaging and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical Engineering and Information Technology (DIETI), University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy

Department of Industrial Engineering (DII), University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy

References (81)

Show source-provided references
1. Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women with Breast Cancer10.1001/jama.2017.14585 ↗
2. Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks10.1038/nature21056 ↗
3. Silvestri, S., Gargiulo, F., and Ciampi, M. (2022). Iterative annotation of biomedical ner corpora with deep neural networks and knowledge bases. Appl. Sci., 12.10.3390/app12125775 ↗
4. Silvestri, S., Tricomi, G., Bassolillo, S.R., De Benedictis, R., and Ciampi, M. (2024). An Urban Intelligence Architecture for Heterogeneous Data and Application Integration, Deployment and Orchestration. Sensors, 24.10.3390/s24072376 ↗
5. TSUA (2025, February 28). Transforming Our World: The 2030 Agenda for Sustainable Development. Available online: https://sdgs.un.org/2030agenda.
6. Ramson, S.J., Vishnu, S., and Shanmugam, M. (2020, January 5–6). Applications of Internet of Things (IoT)—An overview. Proceedings of the 2020 5th International Conference on Devices, Circuits and Systems (ICDCS), Coimbatore, India.10.1109/ICDCS48716.2020.243556 ↗
7. Francesco, G., Stefano, S., and Mario, C. (2017, January 3–6). A big data architecture for knowledge discovery. Proceedings of the 2017 IEEE Symposium on Computers and Communications (ISCC), Heraklion, Greece.
8. Review of internet of medical things (IoMT)–based remote health monitoring through wearable sensors: A case study for diabetic patients10.11591/ijeecs.v20.i1.pp414-422 ↗
9. Cyber threat assessment and management for securing healthcare ecosystems using natural language processing10.1007/s10207-023-00769-w ↗
10. Firesmith, D. (2024, December 18). System Resilience: What Exactly Is It?. Available online: https://insights.sei.cmu.edu/blog/system-resilience-what-exactly-is-it/.
11. Basic concepts and taxonomy of dependable and secure computing10.1109/TDSC.2004.2 ↗
12. Reliable and Resilient AI and IoT-Based Personalised Healthcare Services: A Survey10.1109/ACCESS.2021.3137364 ↗
13. Mathis, A., Mamidanna, P., Cury, K.M., Abe, T., Murthy, V.N., Mathis, M.W., and Bethge, M. (2018). DeepLabCut: Markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci., 21.10.1038/s41593-018-0209-y ↗
14. Kanko, R.M., Laende, E.K., Strutzenberger, G., Brown, M., Selbie, W.S., DePaul, V., Scott, S.H., and Deluzio, K.J. (2021). Assessment of spatiotemporal gait parameters using a deep learning algorithm-based markerless Motion Capture system. J. Biomech., 122.10.1016/j.jbiomech.2021.110414 ↗
15. Drazan, J.F., Phillips, W.T., Seethapathi, N., Hullfish, T.J., and Baxter, J.R. (2021). Moving outside the lab: Markerless Motion Capture accurately quantifies sagittal plane kinematics during the vertical jump. J. Biomech., 125.10.1016/j.jbiomech.2021.110547 ↗
16. Mündermann, L., Corazza, S., and Andriacchi, T.P. (2006). The evolution of methods for the capture of human movement leading to markerless Motion Capture for biomechanical applications. J. NeuroEng. Rehabil., 3.10.1186/1743-0003-3-6 ↗
17. Markerless Motion Capture through Visual Hull, Articulated ICP and Subject Specific Model Generation10.1007/s11263-009-0284-3 ↗
18. Studies of human locomotion: Past, present and future10.1016/S0021-9290(00)00061-0 ↗
19. Harris, G.F., and Smith, P.A. (1996). Human Motion Analysis: Current Applications and Future Directions, IEEE Press.
20. Potential strategies to reduce medial compartment loading in patients with knee OA of varying severity: Reduced walking speed10.1002/art.20132 ↗
21. Quantification of human motion: Gait analysis benefits and limitations to its application to clinical problems10.1016/j.jbiomech.2004.02.047 ↗
22. Recent Developments in Human Motion Analysis10.1016/S0031-3203(02)00100-0 ↗
23. A survey of computer vision-based human Motion Capture10.1006/cviu.2000.0897 ↗
24. Kanade, T., Collins, R., Lipton, A., Burt, P., and Wixson, L. (1998). Advances in cooperative multi-sensor video surveillance. Darpa Image Underst. Workshop, 1.
25. Determination of 3D human body posture from a single view10.1016/0734-189X(85)90094-5 ↗
26. Model-based vision: A program to see a walking person10.1016/0262-8856(83)90003-3 ↗
27. Gavrila, D., and Davis, L. (1996, January 18–20). 3-D model-based tracking of humans in action: A multi-view approach. Proceedings of the Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.1996.517056 ↗
28. Yamamoto, M., and Koshikawa, K. (1991, January 3–6). Human motion analysis based on a robot arm model. Proceedings of the Computer Vision and Pattern Recognition, Maui, HI, USA.
29. Darrel, T., Maes, P., Blumberg, B., and Pentl, A.P. (1994, January 21–23). A novel environment for situated vision and behavior. Proceedings of the Workshop for Visual Behaviors at CVPR, Seattle, WA, USA.
30. Bharatkumar, A.G., Daigle, K.E., Pandy, M.G., Cai, Q., and Aggarwal, J.K. (1994, January 11–12). Lower limb kinematics of human walking with transformation. Proceedings of the Workshop on Motion of Non-Rigid and Articulated Objects, Austin, TX, USA.
31. Motion-based recognition: A survey10.1016/0262-8856(95)93154-K ↗
32. A posture optimization algorithm for model-based Motion Capture of movement sequences10.1016/j.jneumeth.2003.11.013 ↗
33. Lanshammar, H., Persson, T., and Medved, V. (1994, January 10–15). Comparison between a marker-based and a marker-free method to estimate centre of rotation using video image analysis. Proceedings of the Second World Congress of Biomechanics, Amsterdam, The Netherlands.
34. A marker-free method for tracking human lower limb segments based on model matching10.1016/0020-7101(95)01161-7 ↗
35. Marker-less systems for tracking working postures—Results from two experiments10.1016/S0003-6870(01)00023-0 ↗
36. A marker-free system for the analysis of movement disabilities
37. A 3-D marker-free system for the analysis of movement disabilities—An application to the legs10.1109/4233.908371 ↗
38. A clustering based methodology to support the translation of medical specifications to software models10.1016/j.asoc.2018.03.057 ↗
39. A computerized recognition system for the home-based physiotherapy exercises using an RGBD camera10.1109/TNSRE.2014.2326254 ↗
40. Risk factors for falls among elderly persons living in the community10.1056/NEJM198812293192604 ↗
41. Nalci, A., Khodamoradi, A., Balkan, O., Nahab, F., and Garudadri, H. (2015, January 25–29). A computer vision based candidate for functional balance test. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.10.1109/EMBC.2015.7319148 ↗
42. Effectiveness of exercise in patients with osteoarthritis of hip or knee: Nine months’ follow up10.1136/ard.60.12.1123 ↗
43. Leo, M., and Farinella, G.M. (2018). Real-time 3D tracker in robot-based neurorehabilitation. Computer Vision and Pattern Recognition, Computer Vision for Assistive Healthcare, Academic Press.
44. Home self-training: Visual feedback for assisting physical activity for stroke survivors10.1016/j.cmpb.2019.04.019 ↗
45. Wijekoon, A. (2018, January 9–12). Reasoning with multi-modal sensor streams for m-health applications. Proceedings of the Workshop Proceedings for the 26th International Conference on Case-Based Reasoning (ICCBR 2018), Stockholm, Sweden.
46. Rybarczyk, Y., Medina, J.L.P., Leconte, L., Jimenes, K., González, M., and Esparza, D. (2019). Implementation and assessment of an intelligent motor tele-rehabilitation platform. Electronics, 8.10.3390/electronics8010058 ↗
47. Dorado, J., del Toro, X., Santofimia, M.J., Parreno, A., Cantarero, R., Rubio, A., and Lopez, J.C. (2019). A computer-vision-based system for at-home rheumatoid arthritis rehabilitation. Int. J. Distrib. Sens. Netw., 15.10.1177/1550147719875649 ↗
48. A computer vision-based system for a rehabilitation of a human hand
49. Salisbury, J.P., Liu, R., Minahan, L.M., Shin, H.Y., Karnati, S.V.P., and Duffy, S.E. (2018, January 4–7). Patient engagement platform for remote monitoring of vestibular rehabilitation with applications in concussion management and elderly fall prevention. Proceedings of the 2018 IEEE International Conference on Healthcare Informatics (ICHI), New York, NY, USA.10.1109/ICHI.2018.00082 ↗
50. Internationale Vojta Gesellschaft e.V (2020, November 23). Vojta Therapy. Available online: https://www.vojta.com/en/the-vojta-principle/vojta-therapy.
51. A computer vision-based system for monitoring Vojta therapy10.1016/j.ijmedinf.2018.02.010 ↗
52. Chen, Y.L., Liu, C.H., Yu, C.W., Lee, P., and Kuo, Y.W. (2018). An upper extremity rehabilitation system using efficient vision-based action identification techniques. Appl Sci., 8.10.3390/app8071161 ↗
53. Rammer, J., Slavens, B., Krzak, J., Winters, J., Riedel, S., and Harris, G. (2018). Assessment of a marker-less motion analysis system for manual wheelchair application. J. Neuroeng. Rehabil., 15.10.1186/s12984-018-0444-1 ↗
54. (2020, November 24). OpenSim. Available online: https://simtk.org/projects/opensim.
55. Buonaiuto, G., Guarasci, R., Minutolo, R., De Pietro, G., and Esposito, M. (2024). Quantum Transfer Learning for Acceptability Judgements. Quantum Mach. Intell., 6.10.1007/s42484-024-00141-8 ↗
56. An ELECTRA-Based Model for Neural Coreference Resolution10.1109/ACCESS.2022.3189956 ↗
57. A multi-level methodology for the automated translation of a coreference resolution dataset: An application to the Italian language10.1007/s00521-022-07641-3 ↗
58. Maskeliūnas, R., Damaševičius, R., Blažauskas, T., Canbulut, C., Adomavičienė, A., and Griškevičius, J. (2023). BiomacVR: A Virtual Reality-Based System for Precise Human Posture and Motion Analysis in Rehabilitation Exercises Using Depth Sensors. Electronics, 12.10.3390/electronics12020339 ↗
59. Lim, J.H., He, K., Yi, Z., Hou, C., Zhang, C., Sui, Y., and Li, L. (2023, January 24–27). Adaptive Learning Based Upper-Limb Rehabilitation Training System with Collaborative Robot. Proceedings of the 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia.10.1109/EMBC40787.2023.10340313 ↗
60. Kinect-Based Integrated Physiotherapy Mentor Application for Shoulder Damage. Future Gener
61. Saratean, T., Antal, M., Pop, C., Cioara, T., Anghel, I., and Salomie, I. (2020, January 3–5). A Physiotheraphy Coaching System Based on Kinect Sensor. Proceedings of the 2020 IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.10.1109/ICCP51029.2020.9266178 ↗
62. Wagner, J., Szymański, M., Błażkiewicz, M., and Kaczmarczyk, K. (2023). Methods for Spatiotemporal Analysis of Human Gait Based on ˙ Data from Depth Sensors. Sensors, 23.10.3390/s23031218 ↗
63. HDL-PSR: Modelling Spatio-Temporal Features Using Hybrid Deep Learning Approach for Post-Stroke Rehabilitation10.1007/s11063-022-10744-6 ↗
64. Keller, A.V., Torres-Espin, A., Peterson, T.A., Booker, J., O’Neill, C., Lotz, J.C., Bailey, J.F., Ferguson, A.R., and Matthew, R.P. (2022). Unsupervised Machine Learning on Motion Capture Data Uncovers Movement Strategies in Low Back Pain. Front. Bioeng. Biotechnol., 10.10.3389/fbioe.2022.868684 ↗
65. Human Motion Capture for Movement Limitation Analysis Using an RGB-D Camera in Spondyloarthritis: A Validation Study10.1007/s11517-021-02406-x ↗
66. Hustinawaty, H., Rumambi, T., and Hermita, M. (2021). Skeletonization of the Straight Leg Raise Movement Using the Kinect SDK. Int. J. Adv. Comput. Sci. Appl., 12.10.14569/IJACSA.2021.0120683 ↗
67. Girase, H., Nyayapati, P., Booker, J., Lotz, J.C., Bailey, J.F., and Matthew, R.P. (2021). Automated Assessment and Classification of Spine, Hip, and Knee Pathologies from Sit-to-Stand Movements Collected in Clinical Practice. J. Biomech., 128.10.1016/j.jbiomech.2021.110786 ↗
68. Lim, D., Pei, W., Lee, J.W., Musselman, K.E., and Masani, K. (2024). Feasibility of Using a Depth Camera or Pressure Mat for Visual Feedback Balance Training with Functional Electrical Stimulation. Biomed. Eng. Online, 23.10.1186/s12938-023-01191-y ↗
69. LogRF: An Approach to Human Pose Estimation Using Skeleton Landmarks for Physiotherapy Fitness Exercise Correction10.1109/ACCESS.2023.3320144 ↗
70. Khan, M.A.A.H., Murikipudi, M., and Azmee, A.A. (2023, January 26–30). Post-Stroke Exercise Assessment Using Hybrid Quantum Neural Network. Proceedings of the 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), Torino, Italy.10.1109/COMPSAC57700.2023.00078 ↗
71. Using Sensors and Deep Learning to Enable On-Demand Balance Evaluation for Effective Physical Therapy10.1109/ACCESS.2020.2997341 ↗
72. Home Physiotherapy Rehabilitation Based on RGB-D Sensors: A Hybrid Approach to the Joints Angular Range of Motion Estimation10.1007/s12008-020-00728-y ↗
73. Trinidad-Fernández, M., Beckwée, D., Cuesta-Vargas, A., González-Sánchez, M., Moreno, F.A., González-Jiménez, J., Joos, E., and Vaes, P. (2020). Validation, Reliability, and Responsiveness Outcomes of Kinematic Assessment with an RGB-D Camera to Analyze Movement in Subacute and Chronic Low Back Pain. Sensors, 20.10.3390/s20030689 ↗
74. Random Forests10.1023/A:1010933404324 ↗
75. The regression analysis of binary sequences10.1111/j.2517-6161.1958.tb00292.x ↗
76. Support-vector networks10.1007/BF00994018 ↗
77. Principal component analysis10.1016/0169-7439(87)80084-9 ↗
78. Černek, A., Sedmidubsky, J., and Budikova, P. (2024, January 4–6). REHAB24-, Physical Therapy Dataset for Analyzing Pose Estimation Methods. Proceedings of the 17th International Conference on Similarity Search and Applications (SISAP), Providence, RI, USA.10.1007/978-3-031-75823-2_2 ↗
79. Voigt, P., and von dem Bussche, A. (2017). The EU General Data Protection Regulation (GDPR): A Practical Guide, Springer.10.1007/978-3-319-57959-7 ↗
80. European Commission (2020, November 24). Proposal for a Regulation Laying Down Harmonized Rules on Artificial Intelligence (Artificial Intelligence Act). Available online: https://digital-strategy.ec.europa.eu/en/library/proposal-regulation-laying-down-harmonised-rules-artificial-intelligence.
81. Ciampi, M., Sicuranza, M., and Silvestri, S. (2022). A Privacy-Preserving and Standard-Based Architecture for Secondary Use of Clinical Data. Information, 13.10.3390/info13020087 ↗