Real-time deep learning-assisted mechano-acoustic system for respiratory diagnosis and multifunctional classification
Source abstract
Abstract
Epidermally mounted sensors using triaxial accelerometers have been previously used to monitor physiological processes with the implementation of machine learning (ML) algorithm interfaces. The findings from these previous studies have established a strong foundation for the analysis of high-resolution, intricate signals, typically through frequency domain conversion. In this study we integrate a wireless mechano-acoustic sensor with a multi-modal deep learning system for the real-time analysis of signals emitted by the laryngeal prominence area of the thyroid cartilage at frequency ranges up to 1 kHz. This interface provides real-time data visualization and communication with the ML server, creating a system that assesses severity of chronic obstructive pulmonary disease and analyzes the user’s speech patterns.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Phonocardiography and Auscultation Techniques · Respiratory and Cough-Related Research · COVID-19 diagnosis using AI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (53)
Show source-provided references
1. Gupta, P. et al. Precision wearable accelerometer contact microphones for longitudinal monitoring of mechano-acoustic cardiopulmonary signals. NPJ Digital Med. 3, 19 (2020).10.1038/s41746-020-0225-7 ↗
2. Cook, J., Umar, M., Khalili, F. & Taebi, A. Body acoustics for the non-invasive diagnosis of medical conditions. Bioengineering 9, 149 (2022).10.3390/bioengineering9040149 ↗
3. Kang, Y. J. et al. Soft skin-interfaced mechano-acoustic sensors for real-time monitoring and patient feedback on respiratory and swallowing biomechanics. NPJ Digital Med. 5, 147 (2022).10.1038/s41746-022-00691-w ↗
4. Liu, Y. et al. Epidermal mechano-acoustic sensing electronics for cardiovascular diagnostics and human-machine interfaces. Sci. Adv. 2, e1601185 (2016).10.1126/sciadv.1601185 ↗
5. Badshah, A. M., Ahmad, J., Rahim, N. & Baik, S. W. Speech emotion recognition from spectrograms with deep convolutional neural network. in 2017 international conference on platform technology and service (PlatCon). 1–5 (IEEE).10.1109/PlatCon.2017.7883728 ↗
6. Singh, A., Kaur, N., Kukreja, V., Kadyan, V. & Kumar, M. Computational intelligence in processing of speech acoustics: a survey. Complex Intell. Syst. 8, 2623–2661 (2022).10.1007/s40747-022-00665-1 ↗
7. Srivastava, A. et al. Deep learning based respiratory sound analysis for detection of chronic obstructive pulmonary disease. PeerJ Comput. Sci. 7, e369 (2021).10.7717/peerj-cs.369 ↗
8. Zhang, X. et al. Understanding the learning mechanism of convolutional neural networks in spectral analysis. Analytica Chim. Acta 1119, 41–51 (2020).10.1016/j.aca.2020.03.055 ↗
9. Shin, D., Shin, D. & Shin, D. Development of emotion recognition interface using complex EEG/ECG bio-signal for interactive contents. Multimed. Tools Appl. 76, 11449–11470 (2017).10.1007/s11042-016-4203-7 ↗
10. Torok, A. From human-computer interaction to cognitive infocommunications: a cognitive science perspective. in 2016 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). 000433-000438 (IEEE).10.1109/CogInfoCom.2016.7804588 ↗
11. Lee, K. et al. Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch. Nat. Biomed. Eng. 4, 148–158 (2020).10.1038/s41551-019-0480-6 ↗
12. World Health Organization. (Accessed October 20, 2023). Global Health Estimates. https://www.who.int/data/global-health-estimates
13. Agustí, A. et al. Global initiative for chronic obstructive lung disease 2023 report: GOLD executive summary. Am. J. respiratory Crit. care Med. 207, 819–837 (2023).10.1164/rccm.202301-0106PP ↗
14. Lamprecht, B. et al. Determinants of underdiagnosis of COPD in national and international surveys. Chest 148, 971–985 (2015).10.1378/chest.14-2535 ↗
15. Bolton, C. et al. Attaining a correct diagnosis of COPD in general practice. Respiratory Med. 99, 493–500 (2005).10.1016/j.rmed.2004.09.015 ↗
16. Lei, Z. et al. Wearable Neck Surface Accelerometers for Occupational Vocal Health Monitoring: Instrument and Analysis Validation Study. JMIR Form. Res. 6, e39789 (2022).10.2196/39789 ↗
17. Z, N., M, F. & Abdeghani, R. Analysis of Acoustic Parameters from Respiratory Signal in COPD and Pneumonia patients. in 2018 International Conference on Signal, Image, Vision and their Applications (SIVA). 1-4.10.1109/SIVA.2018.8661111 ↗
18. Mohamed, E. E. Voice changes in patients with chronic obstructive pulmonary disease. Egypt. J. Chest Dis. Tuberculosis 63, 561–567 (2014).10.1016/j.ejcdt.2014.03.006 ↗
19. Gupta, S., Chang, P., Anyigbo, N. & Sabharwal, A. mobileSpiro: accurate mobile spirometry for self-management of asthma. in Proceedings of the First ACM Workshop on Mobile Systems, Applications, and Services for Healthcare Article 1 (Association for Computing Machinery, Seattle, Washington, 2011).10.1145/2064942.2064944 ↗
20. Hu, Y., Kim, E. G., Cao, G., Liu, S. & Xu, Y. Physiological acoustic sensing based on accelerometers: a survey for mobile healthcare. Ann. Biomed. Eng. 42, 2264–2277 (2014).10.1007/s10439-014-1111-8 ↗
21. Pasterkamp, H., Kraman, S. S. & Wodicka, G. R. Respiratory sounds. Advances beyond the stethoscope. Am. J. Respir. Crit. Care Med. 156, 974–987 (1997).10.1164/ajrccm.156.3.9701115 ↗
22. Gupta, P., Wen, H., Di Francesco, L. & Ayazi, F. Detection of pathological mechano-acoustic signatures using precision accelerometer contact microphones in patients with pulmonary disorders. Sci. Rep. 11, 13427 (2021).10.1038/s41598-021-92666-2 ↗
23. Tayler, N., Grainge, C., Gove, K., Howarth, P. & Holloway, J. Clinical assessment of speech correlates well with lung function during induced bronchoconstriction. NPJ Prim. care respiratory Med. 25, 1–3 (2015).
24. Shastry, A., Balasubramanium, R. K. & Acharya, P. R. Voice analysis in individuals with chronic obstructive pulmonary disease. Int. J. Phonosurgery Laryngol. 4, 45–49 (2011).10.5005/jp-journals-10023-1081 ↗
25. Nathan, V. et al. Extraction of voice parameters from continuous running speech for pulmonary disease monitoring. in 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). 859-864 (IEEE).10.1109/BIBM47256.2019.8983115 ↗
26. Johnston, I. D., McCluskey, D. K., Tan, C. K. & Tracey, M. C. Mechanical characterization of bulk Sylgard 184 for microfluidics and microengineering. J. Micromech. Microeng. 24, 035017 (2014).10.1088/0960-1317/24/3/035017 ↗
27. Liu, Y. et al. Electronic skin as wireless human-machine interfaces for robotic VR. Sci. Adv. 8, eabl6700 (2022).10.1126/sciadv.abl6700 ↗
28. Chen, Z. et al. Sandwich-structured flexible PDMS@graphene multimodal sensors capable of strain and temperature monitoring with superlative temperature range and sensitivity. Compos. Sci. Technol. 232, 109881 (2023).10.1016/j.compscitech.2022.109881 ↗
29. Deutz, D. B. et al. Flexible Piezoelectric Touch Sensor by Alignment of Lead-Free Alkaline Niobate Microcubes in PDMS. Adv. Funct. Mater. 27, 1700728 (2017).10.1002/adfm.201700728 ↗
30. Callister Jr, W. D. Materials science and engineering an introduction. (2007).
31. Kim, D. et al. Optimal bilayer composites for temperature-tracking wireless electronics. Nanoscale 16, 5613–5623 (2024).10.1039/D3NR05784D ↗
32. Hecht, A., Ma, S., Porszasz, J. & Casaburi, R. Methodology for using long-term accelerometry monitoring to describe daily activity patterns in COPD. Copd 6, 121–129 (2009).10.1080/15412550902755044 ↗
33. Liu, Y., Pharr, M. & Salvatore, G. A. Lab-on-skin: a review of flexible and stretchable electronics for wearable health monitoring. ACS nano 11, 9614–9635 (2017).10.1021/acsnano.7b04898 ↗
34. Hepokoski, M., Curran, A., Viola, T. & Ockfen, A. Thermal Acceptability Limits for Wearable Electronic Devices. in 2021 37th Semiconductor Thermal Measurement, Modeling & Management Symposium (SEMI-THERM). 16-19.
35. Kano, S. & Mekaru, H. Preliminary comparison of respiratory signals using acceleration on neck and humidity in exhaled air. Microsyst. Technol. 27, 1–9 (2021).10.1007/s00542-020-04889-4 ↗
36. Preeti, M., Koushik, G., Baishnab, K. L., Dusarlapudi, K. & Narasimha Raju, K. Low frequency MEMS accelerometers in health monitoring – A review based on material and design aspects. Mater. Today.: Proc. 18, 2152–2157 (2019).
37. Rajan, R., Johnson, J. & Abdul Kareem, N. Bird call classification using dnn-based acoustic modelling. Circuits, Syst., Signal Process. 41, 1–12 (2022).10.1007/s00034-021-01896-2 ↗
38. Tursunov, A., Mustaqeem, Choeh, J. Y. & Kwon, S. Age and gender recognition using a convolutional neural network with a specially designed multi-attention module through speech spectrograms. Sensors 21, 5892 (2021).10.3390/s21175892 ↗
39. Alnuaim, A. A. et al. Speaker gender recognition based on deep neural networks and ResNet50. Wirel. Commun. Mob. Comput. 2022, 1–13 (2022).10.1155/2022/4444388 ↗
40. Ma, E. P.-M. & Love, A. L. Electroglottographic evaluation of age and gender effects during sustained phonation and connected speech. J. voice 24, 146–152 (2010).10.1016/j.jvoice.2008.08.004 ↗
41. Fllho, J. A. X., Bohadana, S. C., Tsuji, D. H., Perázzio, A. F. & Sennes, L. U. Anatomy of the cricothyroid articulation: differences between men and women. Ann. Otol., Rhinol. Laryngol. 114, 250–252 (2005).10.1177/000348940511400316 ↗
42. Wittenberg, T., Tigges, M., Mergell, P. & Eysholdt, U. Functional imaging of vocal fold vibration: digital multislice high-speed kymography. J. Voice 14, 422–442 (2000).10.1016/S0892-1997(00)80087-9 ↗
43. Titze, I. R. & Hunter, E. J. Normal vibration frequencies of the vocal ligament. J. Acoustical Soc. Am. 115, 2264–2269 (2004).10.1121/1.1698832 ↗
44. Ranipa, K., Zhu, W.-P. & Swamy, M. Multimodal CNN fusion architecture with multi-features for heart sound classification. in 2021 IEEE International symposium on circuits and systems (ISCAS). 1-5 (IEEE).10.1109/ISCAS51556.2021.9401551 ↗
45. Paliwal, K. K., Lyons, J. G. & Wójcicki, K. K. Preference for 20-40 ms window duration in speech analysis. in 2010 4th International Conference on Signal Processing and Communication Systems. 1-4 (IEEE).10.1109/ICSPCS.2010.5709770 ↗
46. Montesinos López, O. A., Montesinos López, A. & Crossa, J. in Multivariate Statistical Machine Learning Methods for Genomic Prediction (eds O. A. Montesinos López, A. Montesinos López, & J. Crossa) 109-139 (Springer International Publishing, 2022).10.1007/978-3-030-89010-0_4 ↗
47. Park, Y.-B. et al. Revised (2018) COPD clinical practice guideline of the Korean Academy of Tuberculosis and Respiratory Disease: a summary. Tuberculosis Respiratory Dis. 81, 261–273 (2018).10.4046/trd.2018.0029 ↗
48. Williams, N. The MRC breathlessness scale. Occup. Med. 67, 496–497 (2017).10.1093/occmed/kqx086 ↗
49. Kissner, S. & Bitzer, J. Analysis of current MEMS microphones for cost-effective microphone arrays—a practical approach. in Audio Engineering Society Convention 140. (Audio Engineering Society).
50. Moura, B. A. B. et al. Neck and waist circumference values according to sex, age, and body-mass index: Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). Braz. J. Med Biol. Res. 53, e9815 (2020).10.1590/1414-431x20209815 ↗
51. Yeo, W. H. et al. Multifunctional epidermal electronics printed directly onto the skin. Adv. Mater. 25, 2773–2778 (2013).10.1002/adma.201204426 ↗
52. Choi, J. K., Paek, D. & Lee, J. O. Normal predictive values of spirometry in Korean population. Tuberculosis Respiratory Dis. 58, 230–242 (2005).10.4046/trd.2005.58.3.230 ↗
53. Jones, C. J., Rikli, R. E. & Beam, W. C. A 30-s chair-stand test as a measure of lower body strength in community-residing older adults. Res. Q. Exerc. sport 70, 113–119 (1999).10.1080/02701367.1999.10608028 ↗