Application of Texture Descriptors to Facial Emotion Recognition in Infants
Source abstract
Abstract
The recognition of facial emotions is an important issue in computer vision and artificial intelligence due to its important academic and commercial potential. If we focus on the health sector, the ability to detect and control patients’ emotions, mainly pain, is a fundamental objective within any medical service. Nowadays, the evaluation of pain in patients depends mainly on the continuous monitoring of the medical staff when the patient is unable to express verbally his/her experience of pain, as is the case of patients under sedation or babies. Therefore, it is necessary to provide alternative methods for its evaluation and detection. Facial expressions can be considered as a valid indicator of a person’s degree of pain. Consequently, this paper presents a monitoring system for babies that uses an automatic pain detection system by means of image analysis. This system could be accessed through wearable or mobile devices. To do this, this paper makes use of three different texture descriptors for pain detection: Local Binary Patterns, Local Ternary Patterns, and Radon Barcodes. These descriptors are used together with Support Vector Machines (SVM) for their classification. The experimental results show that the proposed features give a very promising classification accuracy of around 95% for the Infant COPE database, which proves the validity of the proposed method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Pediatric Pain Management Techniques · Infant Health and Development · Infant Development and Preterm Care
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Technology, University of Alicante, 03690 San Vicente del Raspeig-Alicante, Spain
Funding
Ministerio de Economía y Competitividad · Conselleria d'Educació, Investigació, Cultura i Esport
References (55)
Show source-provided references
1. Facial expression and emotion10.1037/0003-066X.48.4.384 ↗
2. Pujol, F.A., Mora, H., and Martínez, A. (2019, January 24–26). Emotion Recognition to Improve e-Healthcare Systems in Smart Cities. Proceedings of the Research & Innovation Forum 2019, Rome, Italy.10.1007/978-3-030-30809-4_23 ↗
3. Machine assessment of neonatal facial expressions of acute pain10.1016/j.dss.2006.02.004 ↗
4. An Approach for Automatic Pain Detection through Facial Expression10.1016/j.procs.2016.04.072 ↗
5. Lucey, P., Cohn, J.F., Prkachin, K.M., Solomon, P.E., and Matthews, I. (2011). Painful data: The UNBC-McMaster shoulder pain expression archive database. Face and Gesture 2011, IEEE.10.1109/FG.2011.5771462 ↗
6. Hammal, Z., and Cohn, J.F. (2012, January 22–26). Automatic detection of pain intensity. Proceedings of the 14th ACM International Conference on Multimodal Interaction—ICMI ’12, Santa Monica, CA, USA.10.1145/2388676.2388688 ↗
7. Automatically Detecting Pain in Video Through Facial Action Units10.1109/TSMCB.2010.2082525 ↗
8. Rodriguez, P., Cucurull, G., Gonzàlez, J., Gonfaus, J.M., Nasrollahi, K., Moeslund, T.B., and Roca, F.X. (2017). Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification. IEEE Trans. Cybern., 1–11.
9. Ilyas, C.M.A., Haque, M.A., Rehm, M., Nasrollahi, K., and Moeslund, T.B. (2018, January 27–29). Facial Expression Recognition for Traumatic Brain Injured Patients. Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Funchal, Madeira, Portugal.10.5220/0006721305220530 ↗
10. Machine recognition and representation of neonatal facial displays of acute pain10.1016/j.artmed.2004.12.003 ↗
11. Naufal Mansor, M., and Rejab, M.N. (December, January 29). A computational model of the infant pain impressions with Gaussian and Nearest Mean Classifier. Proceedings of the 2013 IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia.10.1109/ICCSCE.2013.6719968 ↗
12. Local binary patterns variants as texture descriptors for medical image analysis10.1016/j.artmed.2010.02.006 ↗
13. Celona, L., and Manoni, L. (, January 11–15). Neonatal Facial Pain Assessment Combining Hand-Crafted and Deep Features. Proceedings of the New Trends in Image Analysis and Processing—ICIAP 2017, Catania, Italy.10.1007/978-3-319-70742-6_19 ↗
14. Zamzmi, G., Goldgof, D., Kasturi, R., and Sun, Y. (2018). Neonatal Pain Expression Recognition Using Transfer Learning. arXiv.
15. Zamzmi, G., Paul, R., Goldgof, D., Kasturi, R., and Sun, Y. (2019, January 14–19). Pain assessment from facial expression: Neonatal convolutional neural network (N-CNN). Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.10.1109/IJCNN.2019.8851879 ↗
16. Video-based discomfort detection for infants10.1007/s00138-018-0968-1 ↗
17. Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals10.1155/S1110865704406192 ↗
18. Affective computing in virtual reality: Emotion recognition from brain and heartbeat dynamics using wearable sensors
19. Miranda Calero, J.A., Marino, R., Lanza-Gutierrez, J.M., Riesgo, T., Garcia-Valderas, M., and Lopez-Ongil, C. (2018, January 14–16). Embedded Emotion Recognition within Cyber-Physical Systems using Physiological Signals. Proceedings of the 2018 Conference on Design of Circuits and Integrated Systems (DCIS), Lyon, France.10.1109/DCIS.2018.8681496 ↗
20. Wearable 2.0: Enabling Human-Cloud Integration in Next, Generation Healthcare Systems10.1109/MCOM.2017.1600410CM ↗
21. Kwon, J., Kim, D.H., Park, W., and Kim, L. (2016, January 16–20). A wearable device for emotional recognition using facial expression and physiological response. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.10.1109/EMBC.2016.7592037 ↗
22. Washington, P., Voss, C., Haber, N., Tanaka, S., Daniels, J., Feinstein, C., Winograd, T., and Wall, D. (2016, January 7–12). A Wearable Social Interaction Aid for Children with Autism. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA. CHI EA ’16.10.1145/2851581.2892282 ↗
23. Kwon, J., and Kim, L. (2019). Emotion recognition using a glasses-type wearable device via multi-channel facial responses. arXiv.
24. Dias, D., and Paulo Silva Cunha, J. (2018). Wearable Health Devices—Vital Sign Monitoring, Systems and Technologies. Sensors, 18.10.3390/s18082414 ↗
25. Chen, W., Dols, S., Oetomo, S.B., and Feijs, L. (2010, January 10–12). Monitoring Body Temperature of Newborn Infants at Neonatal Intensive Care Units Using Wearable Sensors. Proceedings of the Fifth International Conference on Body Area Networks, BodyNets ’10, Corfu Island, Greece.10.1145/2221924.2221960 ↗
26. Mahmud, M.S., Wang, H., and Fang, H. (2017, January 13). Design of a Wireless Non-Contact Wearable System for Infants Using Adaptive Filter. Proceedings of the 10th EAI International Conference on Mobile Multimedia Communications, Chongqing, China.10.4108/eai.13-7-2017.2270652 ↗
27. Wearables for Pediatric Rehabilitation: How to Optimally Design and Use Products to Meet the Needs of Users10.1093/ptj/pzz024 ↗
28. Cattani, L., Saini, H.P., Ferrari, G., Pisani, F., and Raheli, R. (2016, January 15–18). SmartCED: An Android application for neonatal seizures detection. Proceedings of the 2016 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Benevento, Italy.10.1109/MeMeA.2016.7533708 ↗
29. The Emerging Market of Smartphone-Integrated Infant Physiologic Monitors10.1001/jama.2016.19137 ↗
30. King, D. (2014). Marketing wearable home baby monitors: Real peace of mind?. BMJ, 349.10.1136/bmj.g6639 ↗
31. Quantified Baby: Parenting and the Use of a Baby Wearable in the Wild
32. Eight Years Later, Are We Still Hurting Newborn Infants?10.1159/000357207 ↗
33. Epidemiology of painful procedures performed in neonates: A systematic review of observational studies10.1002/ejp.757 ↗
34. fMRI reveals neural activity overlap between adult and infant pain10.7554/eLife.06356 ↗
35. Assessment of pain in newborn infants10.1016/j.siny.2019.04.003 ↗
36. Neurophysiological and behavioral measures of pain during neonatal hip examination10.1002/pne2.12006 ↗
37. Premature Infant Pain Profile: Development and Initial Validation10.1097/00002508-199603000-00004 ↗
38. CRIES: A new neonatal postoperative pain measurement score. Initial testing of validity and reliability10.1111/j.1460-9592.1995.tb00242.x ↗
39. The development of a tool to assess neonatal pain
40. Pain expression in neonates: Facial action and cry10.1016/0304-3959(87)90073-X ↗
41. Neonatal facial and cry responses to invasive and non-invasive procedures10.1016/0304-3959(90)91142-6 ↗
42. Neonatal Facial Coding System for Assessing Postoperative Pain in Infants: Item Reduction is Valid and Feasible10.1097/00002508-200311000-00003 ↗
43. Automatic Facial Expression Analysis A Survey10.5121/ijcses.2012.3604 ↗
44. Facial Expression Recognition: A Survey10.1016/j.procs.2015.08.011 ↗
45. Differences between uni-and multidimensional scales for assessing pain in term newborn infants at the bedside10.6061/clinics/2012(10)08 ↗
46. A Guide to Pain Assessment and Management in the Neonate10.1007/s40138-016-0089-y ↗
47. Ahmed, M., Shaukat, A., and Akram, M.U. (2016, January 4–6). Comparative analysis of texture descriptors for classification. Proceedings of the 2016 IEEE International Conference on Imaging Systems and Techniques (IST), Chania, Greece.10.1109/IST.2016.7738192 ↗
48. Pajdla, T., and Matas, J. (2004). Face Recognition with Local Binary Patterns. Computer Vision—ECCV 2004, Springer.10.1007/b97873 ↗
49. Shan, C., Gong, S., and McOwan, P. (2005, January 11–14). Robust facial expression recognition using local binary patterns. Proceedings of the IEEE International Conference on Image Processing 2005, Genoa, Italy.
50. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Evaluation of LBP and Deep Texture Descriptors with a New Robustness Benchmark. Lecture Notes in Computer Science, Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016, Springer International Publishing.10.1007/978-3-319-46454-1 ↗
51. Tan, X., and Triggs, B. (2007). Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions. Analysis and Modeling of Faces and Gestures, Springer.10.1007/978-3-540-75690-3_13 ↗
52. Invariant pattern recognition using the RFM descriptor10.1016/j.patcog.2011.06.020 ↗
53. Tizhoosh, H.R. (2015, January 27–30). Barcode annotations for medical image retrieval: A preliminary investigation. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.10.1109/ICIP.2015.7350913 ↗
54. Prediction of brain maturity in infants using machine-learning algorithms10.1016/j.neuroimage.2016.05.029 ↗
55. Support vector machines10.1177/1536867X1601600407 ↗