Modelling Proper and Improper Sitting Posture of Computer Users Using Machine Vision for a Human–Computer Intelligent Interactive System during COVID-19
Source abstract
Abstract
Human posture recognition is one of the most challenging tasks due to the variation in human appearance, changes in the background and illumination, additional noise in the frame, and diverse characteristics and amount of data generated. Aside from these, generating a high configuration for recognition of human body parts, occlusion, nearly identical parts of the body, variations of colors due to clothing, and other various factors make this task one of the hardest in computer vision. Therefore, these studies require high-computing devices and machines that could handle the computational load of this task. This study used a small-scale convolutional neural network and a smartphone built-in camera to recognize proper and improper sitting posture in a work-from-home setup. Aside from the recognition of body points, this study also utilized points’ distances and angles to help in recognition. Overall, the study was able to develop two objective datasets capturing the left and right side of the participants with the supervision and guidance of licensed physical therapists. The study shows accuracies of 85.18% and 92.07%, and kappas of 0.691 and 0.838, respectively. The system was developed, implemented, and tested in a work-from-home environment.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hand Gesture Recognition Systems · Ergonomics and Musculoskeletal Disorders · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Technology, Mapua University, Makati 1200, Philippines
College of Information Technology Education, Technological Institute of the Philippines-Manila, Manila 1001, Philippines
References (56)
Show source-provided references
1. Kreiss, S., Bertoni, L., and Alahi, A. (2019, January 15–20). PifPaf: Composite fields for human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01225 ↗
2. Babu, S.C. (2020, June 01). A 2019 Guide to Human Pose Estimation With Deep Learning. Available online: https://nanonets.com/blog/humanpose-estimation-2d-guide/.
3. Chen, X., and Yuille, A.L. (2014, January 8–13). Articulated pose estimation by a graphical model with image dependent pairwise relations. Proceedings of the Annual Conference on Neural Information Processing Systems, NIPS, Montreal, QC, Canada.
4. Mwiti, D. (2020, June 01). A 2019 Guide to Human Pose Estimation. Available online: https://heartbeat.fritz.ai/a-2019-guide-to-human-poseestimation-c10b79b64b73.
5. Andriluka, M., Roth, S., and Schiele, B. (2009, January 20–25). Pictorial structures revisited: People detection and articulated pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, USA.10.1109/CVPR.2009.5206754 ↗
6. Andriluka, M., Roth, S., and Schiele, B. (2010, January 13–18). Monocular 3D pose estimation and tracking by detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, USA.10.1109/CVPR.2010.5540156 ↗
7. Johnson, S., and Everingham, M. (September, January 31). Clustered pose and nonlinear appearance models for human pose estimation. Proceedings of the British Machine Vision Conference, Aberystwyth, UK.
8. Pishchulin, L., Andriluka, M., Gehler, P., and Schiele, B. (2013, January 23–28). Poselet conditioned pictorial structures. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.10.1109/CVPR.2013.82 ↗
9. Yang, Y., and Ramanan, D. (2011, January 20–25). Articulated pose estimation with flexible mixtures-of-parts. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.10.1109/CVPR.2011.5995741 ↗
10. Articulated human detection with flexible mixtures of parts10.1109/TPAMI.2012.261 ↗
11. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the ECCV, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
12. Andriluka, M., Pishchulin, L., Gehler, P., and Schiele, B. (2014, January 23–28). MPII human pose dataset. Proceedings of the CVPR, Columbus, OH, USA.
13. Toshev, A., and Szegedy, C. (2014, January 23–28). DeepPose: Human pose estimation via deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA.10.1109/CVPR.2014.214 ↗
14. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). AlexNet: ImageNet classification with deep convolutional neural networks. Proceedings of the NIPS, Lake Tahoe, NV, USA.
15. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
16. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
17. Simonyan, K., and Zisserman, A. (2015). VGG: Very deep convolutional networks for large-scale image recognition. arXiv.
18. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). ResNet: Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
19. Coupled action recognition and pose estimation from multiple views10.1007/s11263-012-0532-9 ↗
20. Reliability and validity of a domain-specific last 7-d sedentary time questionnaire10.1249/MSS.0000000000000214 ↗
21. Study of physical activity and its effect on body composition and quality of life in female employees of Kermanshah University of Medical Sciences in 2013
22. Homer, A.R., Owen, N., Sethi, P., Clark, B.K., Healy, G.N., Dempsey, P.C., and Dunstan, D.W. (2020). Differing context-specific sedentary behaviors in Australian adults with higher and lower diabetes risk. Res. Sq.10.21203/rs.3.rs-71442/v1 ↗
23. Occupational sitting time, its determinants and intervention strategies in Malaysian office workers: A mixed-methods study10.1093/heapro/daab149 ↗
24. Association of Sitting Time with Mortality and Cardiovascular Events in High-Income, Middle-Income, and Low-Income Countries10.1001/jamacardio.2022.1581 ↗
25. Temporal trends in sitting time by domain in a cohort of mid-age Australian men and women10.1016/j.maturitas.2018.08.006 ↗
26. Amount of time spent in sedentary behaviors in the United States, 2003–200410.1093/aje/kwm390 ↗
27. Association between sitting and occupational LBP10.1007/s00586-006-0143-7 ↗
28. The burden and determinants of neck pain in workers: Results of the Bone and Joint Decade 2000–2010 Task Force on Neck Pain and Its Associated Disorders10.1097/BRS.0b013e3181643ee4 ↗
29. Prevalence of self-reported musculoskeletal symptoms among office workers10.1093/occmed/kqn072 ↗
30. Computer users’ risk factors for developing shoulder, elbow and back symptoms10.5271/sjweh.827 ↗
31. Prevalence of neck pain and disability in computer users
32. Back Pain-Are Health Care Undergraduates at Risk?
33. The Global Spine Care Initiative: A summary of the global burden of low back and neck pain studies10.1007/s00586-017-5432-9 ↗
34. Burden of disability due to musculoskeletal (MSK) disorders10.1016/j.berh.2014.08.002 ↗
35. US Health Care Spending by Payer and Health Condition, 1996–201610.1001/jama.2020.0734 ↗
36. Wei, S.-E., Ramakrishna, V., Kanade, T., and Sheikh, Y. (2016, January 27–30). Convolutional pose machines. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.10.1109/CVPR.2016.511 ↗
37. Liu, Y., and Weinshall, D. (2017, January 23–27). Recognizing dance gestures with pose-based RNNs. Proceedings of the on Thematic Workshops of ACM Multimedia, Mountain View, CA, USA.
38. Chen, J., Li, Y., Liu, Y., and Weinshall, D. (2018, January 18–23). Hybrid CNN-RNN for action recognition from video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
39. Sun, S., Liu, Y., and Weinshall, D. (2018, January 18–23). Multi-task single-person pose estimation and action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
40. Su, Z., Ye, M., Zhang, G., Dai, L., and Sheng, J. (2019). Cascade feature aggregation for human pose estimation. arXiv.
41. Newell, A., Yang, K., and Deng, J. (2016, January 8–16). Stacked hourglass networks for human pose estimation. Proceedings of the ECCV, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_29 ↗
42. Zhang, H., Ouyang, H., Liu, S., Qi, X., Shen, X., Yang, R., and Jia, J. (2019, January 15–20). Human pose estimation with spatial contextual information. Proceedings of the CVPR, Long Beach, CA, USA.
43. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018, January 8–14). Simple baselines for human pose estimation and tracking. Proceedings of the European Conference on Computer Vision—ECCV, Munich, Germany.10.1007/978-3-030-01225-0 ↗
44. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., and Sun, J. (2018, January 18–23). Cascaded pyramid network for multi-person pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00742 ↗
45. Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., and Schiele, B. (2016, January 8–16). DeeperCut: A deeper, stronger, and faster multi-person pose estimation model. Proceedings of the ECCV, Amsterdam, The Netherlands.10.1007/978-3-319-46466-4_3 ↗
46. Cao, Z., Simon, T., Wei, S.-E., and Sheikh, Y. (2017, January 21–26). Realtime multi-person 2D pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.143 ↗
47. Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15–20). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00584 ↗
48. Estrada, J., and Vea, L. (2017, January 5–8). Sitting posture recognition for computer users using smartphones and a web camera. Proceedings of the TENCON 2017 IEEE Region 10 Conference, Penang, Malaysia.10.1109/TENCON.2017.8228098 ↗
49. Ghazal, S., and Khan, U.S. (2018, January 3–4). Human Posture Classification Using Skeleton Information. Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies: Invent, Innovate and Integrate for Socioeconomic Development, iCoMET 2018, Sukkur, Pakistan.10.1109/ICOMET.2018.8346407 ↗
50. A real-time webcam-based method for assessing upper-body postures10.1007/s00138-019-01033-9 ↗
51. Kappattanavar, A.M., da Cruz, H.F., Arnrich, B., and Böttinger, E. (December, January 30). Position Matters: Sensor Placement for Sitting Posture Classification. Proceedings of the 2020 IEEE International Conference on Healthcare Informatics (ICHI), Oldenburg, Germany.10.1109/ICHI48887.2020.9374328 ↗
52. Katayama, H., Mizomoto, T., Rizk, H., and Yamaguchi, H. (2022, January 21–25). You Work We Care: Sitting Posture Assessment Based on Point Cloud Data. Proceedings of the 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Pisa, Italy.10.1109/PerComWorkshops53856.2022.9767292 ↗
53. Jolly, V., Jain, R., Shah, J., and Dhage, S. (2022, January 21–22). Posture Correction and Detection using 3-D Image Classification. Proceedings of the 2022 International Conference for Advancement in Technology (ICONAT), Goa, India.10.1109/ICONAT53423.2022.9725833 ↗
54. Estrada, J.E., and Vea, L.A. (2016, January 9–11). Real-time human sitting posture detection using mobile devices. Proceedings of the 2016 IEEE Region 10 Symposium (TENSYMP), Bali, Indonesia.10.1109/TENCONSpring.2016.7519393 ↗
55. (2021, December 10). Pose Landmarks Detection Task Guide. Available online: https://google.github.io/mediapipe/solutions/pose.html.
56. Ma, J., Ma, L., Ruan, W., Chen, H., and Feng, J. (2022, January 24–26). A Wushu Posture Recognition System Based on MediaPipe. Proceedings of the 2022 2nd International Conference on Information Technology and Contemporary Sports (TCS), Guangzhou, China.10.1109/TCS56119.2022.9918744 ↗