Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning
Source abstract
Abstract
Smartness, which underpins smart cities and societies, is defined by our ability to engage with our environments, analyze them, and make decisions, all in a timely manner. Healthcare is the prime candidate needing the transformative capability of this smartness. Social media could enable a ubiquitous and continuous engagement between healthcare stakeholders, leading to better public health. Current works are limited in their scope, functionality, and scalability. This paper proposes Sehaa, a big data analytics tool for healthcare in the Kingdom of Saudi Arabia (KSA) using Twitter data in Arabic. Sehaa uses Naive Bayes, Logistic Regression, and multiple feature extraction methods to detect various diseases in the KSA. Sehaa found that the top five diseases in Saudi Arabia in terms of the actual afflicted cases are dermal diseases, heart diseases, hypertension, cancer, and diabetes. Riyadh and Jeddah need to do more in creating awareness about the top diseases. Taif is the healthiest city in the KSA in terms of the detected diseases and awareness activities. Sehaa is developed over Apache Spark allowing true scalability. The dataset used comprises 18.9 million tweets collected from November 2018 to September 2019. The results are evaluated using well-known numerical criteria (Accuracy and F1-Score) and are validated against externally available statistics.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Data-Driven Disease Surveillance · Artificial Intelligence in Healthcare · Sentiment Analysis and Opinion Mining
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
High-Performance Computing Center, King Abdulaziz University, Jeddah 21589, Saudi Arabia
School of Computer Science, Cardiff University, Cardiff CF10 3AT, UK
Funding
King Abdulaziz University
References (97)
Show source-provided references
1. Mehmood, R., Katib, S.S.I., and Chlamtac, I. (2020). Smart Infrastructure and Applications: Foundations for Smarter Cities and Societies, EAI/Springer Innovations in Communication and Computing, Springer International Publishing, Springer Nature Switzerland AG.10.1007/978-3-030-13705-2 ↗
2. (2020, February 08). Just How Big Is the Healthcare Industry? Here’s What You Need to Know—Dreamit Ventures. Available online: https://www.dreamit.com/journal/2018/4/24/size-healthcare-industry.
3. (2020, February 08). Getting the Right Care to the Right People at the Right Cost: An Interview With Ron Walls | McKinsey. Available online: https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/getting-the-right-care-to-the-right-people-at-the-right-cost-an-interview-with-ron-walls.
4. Sherman, E. (2020, January 12). U.S. Health Care Spending Hit $3.65 Trillion in 2018. Available online: https://fortune.com/2019/02/21/us-health-care-costs-2/.
5. (2020, February 08). Finding the Future of Care Provision: The Role of Smart Hospitals | McKinsey. Available online: https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/finding-the-future-of-care-provision-the-role-of-smart-hospitals.
6. Kemp, S. (2020, January 10). Digital Trends 2019: Every Single Stat You Need to Know about the Internet. Available online: https://thenextweb.com/contributors/2019/01/30/digital-trends-2019-every-single-stat-you-need-to-know-about-the-internet/.
7. Statista (2019, April 19). Countries with Most Twitter Users 2019 | Statistic. Available online: https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/.
8. Lin, Y. (2020, January 11). 10 Twitter Statistics Every Marketer Should Know in 2020. Available online: https://www.oberlo.com/blog/twitter-statistics.
9. (2020, January 11). witter by the Numbers (2019): Stats, Demographics & Fun Facts. Available online: https://www.omnicoreagency.com/twitter-statistics/.
10. Alotaibi, S., Mehmood, R., and Katib, I. (2019, January 10–13). Sentiment Analysis of Arabic Tweets in Smart Cities: A Review of Saudi Dialect. Proceedings of the 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC), Rome, Italy.10.1109/FMEC.2019.8795331 ↗
11. Sentiment analysis of health care tweets: Review of the methods used
12. Investigating the Use of Machine Learning Algorithms in Detecting Gender of the Arabic Tweet Author
13. Lexicon-based Approach to Build Service Provider Reputation from Arabic Tweets in Twitter
14. A comprehensive survey of arabic sentiment analysis10.1016/j.ipm.2018.07.006 ↗
15. Alayba, A.M., Palade, V., England, M., and Iqbal, R. (2017, January 3–5). Arabic Language Sentiment Analysis on Health Services. Proceedings of the International Workshop on Arabic and derived Script Analysis and Recognition, Nancy, France.10.1109/ASAR.2017.8067771 ↗
16. Alkouz, B., and Al Aghbari, Z. (2018, January 9–12). Analysis and prediction of influenza in the UAE based on Arabic tweets. Proceedings of the 2018 IEEE 3rd International Conference on Big Data Analysis (ICBDA 2018), Shanghai, China.10.1109/ICBDA.2018.8367652 ↗
17. Ilyas, M.U., and Alowibdi, J.S. (2018, January 13–17). Disease Tracking in GCC Region Using Arabic Language Tweets. Proceedings of the Companion of the Web Conference 2018—WWW’18, Lyon, France.10.1145/3184558.3186357 ↗
18. Alomari, E., Mehmood, R., and Katib, I. (2020). Sentiment Analysis of Arabic Tweets for Road Traffic Congestion and Event Detection. Smart Infrastructure and Applications, Springer.10.1007/978-3-030-13705-2_2 ↗
19. Suma, S., Mehmood, R., and Albeshri, A. (2019). Automatic Detection and Validation of Smart City Events Using HPC and Apache Spark Platforms. Smart Infrastructure and Applications: Foundations for Smarter Cities and Societies, Springer.10.1007/978-3-030-13705-2_3 ↗
20. Alomari, E., Mehmood, R., and Katib, I. (2019, January 19–23). Road Traffic Event Detection Using Twitter Data, Machine Learning, and Apache Spark. Proceedings of the 3rd IEEE International Conference on Smart City Innovations (SCI 2019), Leicester, UK.10.1109/SmartWorld-UIC-ATC-SCALCOM-IOP-SCI.2019.00332 ↗
21. Lau, R.Y. (2017, January 12–15). Toward a social sensor based framework for intelligent transportation. Proceedings of the 2017 IEEE 18th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), Macau, China.10.1109/WoWMoM.2017.7974354 ↗
22. Pandhare, K.R., and Shah, M.A. (2017, January 10–11). Real time road traffic event detection using Twitter and spark. Proceedings of the 2017 International Conference on Inventive Communication and Computational Technologies (ICICCT), Coimbatore, India.10.1109/ICICCT.2017.7975237 ↗
23. Salas, A., Georgakis, P., Nwagboso, C., Ammari, A., and Petalas, I. (2017, January 23–26). Traffic Event Detection Framework Using Social Media. Proceedings of the IEEE International Conference on Smart Grid and Smart Cities, Singapore.10.1109/ICSGSC.2017.8038595 ↗
24. Big data: A survey10.1007/s11036-013-0489-0 ↗
25. Information Resources Management Association Future Networked Healthcare Systems: A Review and Case Study. Big Data: Concepts, Methodologies, Tools, and Applications, IGI Global.
26. (2019, December 28). “Apache SparkTM - Unified Analytics Engine for Big Data.” [Online]. Available online: https://spark.apache.org/.
27. Shalev-Shwartz, S., and Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press.10.1017/CBO9781107298019 ↗
28. (2019, December 29). Logistic Regression — ML Glossary documentation. Available online: https://ml-cheatsheet.readthedocs.io/en/latest/logistic_regression.html.
29. (2020, January 02). Graphical Models Lecture 2: Bayesian Network Representatioon. Available online: https://people.cs.umass.edu/~mccallum/courses/gm2011/02-bn-rep.pdf.
30. (2020, February 07). Extracting, Transforming and Selecting Features—Spark 2.4.4 Documentation. Available online: https://spark.apache.org/docs/latest/mL-features#tf-idf.
31. Mehmood, R., Bhaduri, B., Katib, I., and Chlamtac, I. (2017, January 27–29). Smart Societies, Infrastructure, Technologies and Applications. Proceedings of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering (LNICST), Jeddah, Saudi Arabia.10.1007/978-3-319-94180-6 ↗
32. Enabling reliable and resilient IoT based smart city applications10.1007/978-3-319-94180-6_17 ↗
33. Data Fusion and IoT for Smart Ubiquitous Environments: A Survey10.1109/ACCESS.2017.2697839 ↗
34. UbeHealth: A personalized ubiquitous cloud and edge-enabled networked healthcare system for smart cities10.1109/ACCESS.2018.2846609 ↗
35. Muhammed, T., Mehmood, R., Albeshri, A., and Alzahrani, A. (2020). HCDSR: A Hierarchical Clustered Fault Tolerant Routing Technique for IoT-Based Smart Societies. Smart Infrastructure and Applications, Springer.10.1007/978-3-030-13705-2_25 ↗
36. UTiLearn: A Personalised Ubiquitous Teaching and Learning System for Smart Societies10.1109/ACCESS.2017.2668840 ↗
37. Alomari, K.M., ElSherif, H.M., and Shaalan, K. (2017). Arabic Tweets Sentimental Analysis Using Machine Learning. Lecture Notes in Computer Science, Springer.10.1007/978-3-319-60042-0_66 ↗
38. Alomari, E., and Mehmood, R. (2018). Analysis of Tweets in Arabic Language for Detection of Road Traffic Conditions, Springer.10.1007/978-3-319-94180-6_12 ↗
39. Big Data Logistics: A health-care Transport Capacity Sharing Model10.1016/j.procs.2015.08.566 ↗
40. Exploring the influence of big data on city transport operations: A Markovian approach10.1108/IJOPM-03-2015-0179 ↗
41. Arfat, Y., Usman, S., Mehmood, R., and Katib, I. (2020). Big Data Tools, Technologies, and Applications: A Survey, Springer.10.1007/978-3-030-13705-2_19 ↗
42. Arfat, Y., Usman, S., Mehmood, R., and Katib, I. (2020). Big Data for Smart Infrastructure Design: Opportunities and Challenges, Springer.10.1007/978-3-030-13705-2_20 ↗
43. Arfat, Y., Suma, S., Mehmood, R., and Albeshri, A. (2020). Parallel Shortest Path Big Data Graph. Computations of US Road Network Using Apache Spark: Survey, Architecture, and Evaluation, Springer.10.1007/978-3-030-13705-2_8 ↗
44. Usman, S., Mehmood, R., and Katib, I. (2020). Big Data and HPC Convergence for Smart Infrastructures: A Review and Proposed Architecture, Springer.10.1007/978-3-030-13705-2_23 ↗
45. Muhammed, T., Mehmood, R., Albeshri, A., and Katib, I. (2019). SURAA: A Novel Method and Tool for Loadbalanced and Coalesced SpMV Computations on GPUs. Appl. Sci., 9.10.3390/app9050947 ↗
46. Alyahya, H., Mehmood, R., and Katib, I. (2020). Parallel Iterative Solution of Large Sparse Linear Equation Systems on the Intel MIC Architecture. Smart Infrastructure and Applications, Springer.10.1007/978-3-030-13705-2_16 ↗
47. Usman, S., Mehmood, R., Katib, I., Albeshri, A., and Altowaijri, S.M. (2019). ZAKI: A Smart Method and Tool for Automatic Performance Optimization of Parallel SpMV Computations on Distributed Memory Machines. Mob. Netw. Appl., 1–20.10.1007/s11036-019-01318-3 ↗
48. ZAKI+: A Machine Learning Based Process Mapping Tool for SpMV Computations on Distributed Memory Architectures10.1109/ACCESS.2019.2923565 ↗
49. Enabling Smarter Societies through Mobile Big Data Fogs and Clouds10.1016/j.procs.2017.05.439 ↗
50. Boucadair, M., and Jacquenet, C. (2015). Future Networked Healthcare Systems: A Review and Case Study. Handbook of Research on Redesigning the Future of Internet Architectures, IGI Global.10.4018/978-1-4666-8371-6 ↗
51. Lo’ai, A.T., Bakhader, W., Mehmood, R., and Song, H. (2016, January 4–8). Cloudlet-Based Mobile Cloud Computing for Healthcare Applications. Proceedings of the 2016 IEEE Global Communications Conference (GLOBECOM), Washington, DC, USA.
52. Schlingensiepen, J., Mehmood, R., Nemtanu, F.C., and Niculescu, M. (2013, January 25–27). Increasing Sustainability of Road Transport in European Cities and Metropolitan Areas by Facilitating Autonomic Road Transport Systems (ARTS). Proceedings of the 2013 5th International Conference on Sustainable Automotive Technologies (ICSAT 2013), Ingolstadt, Germany.10.1007/978-3-319-01884-3_20 ↗
53. Alam, F., Mehmood, R., Katib, I., Altowaijri, S.M., and Albeshri, A. (2019). TAAWUN: A Decision Fusion and Feature Specific Road Detection Approach for Connected Autonomous Vehicles. Mob. Netw. Appl., 1–17.10.1007/s11036-019-01319-2 ↗
54. Alotaibi, S., Mehmood, R., and Katib, I. (2020). The Role of Big Data and Twitter Data Analytics in Healthcare Supply Chain Management. Smart Infrastructure and Applications, Springer.10.1007/978-3-030-13705-2_11 ↗
55. Alamoudi, E., Mehmood, R., Albeshri, A., and Gojobori, T. (2020). A Survey of Methods and Tools for Large-Scale DNA Mixture Profiling. Smart Infrastructure and Applications, Springer.10.1007/978-3-030-13705-2_9 ↗
56. Big data enabled healthcare supply chain management: Opportunities and challenges10.1007/978-3-319-94180-6_21 ↗
57. Aqib, M., Mehmood, R., Alzahrani, A., Katib, I., Albeshri, A., and Altowaijri, S.M. (2019). Altowaijri. Smarter Traffic Prediction Using Big Data, In-Memory Computing, Deep Learning and GPUs. Sensors, 19.10.3390/s19092206 ↗
58. Aqib, M., Mehmood, R., Alzahrani, A., Katib, I., Albeshri, A., and Altowaijri, S.M. (2019). Rapid Transit Systems: Smarter Urban Planning Using Big Data, In-Memory Computing, Deep Learning, and GPUs. Sustainability, 11.10.3390/su11102736 ↗
59. Location Privacy in Smart Cities Era10.1007/978-3-319-94180-6_14 ↗
60. Towards a semantically enriched computational intelligence (SECI) framework for smart farming10.1007/978-3-319-94180-6_24 ↗
61. Sentiment Analysis and Opinion Mining
62. Big Data for Health10.1109/JBHI.2015.2450362 ↗
63. Health-related hypothesis generation using social media data10.1007/s13278-014-0239-8 ↗
64. A model for mining public health topics from Twitter
65. Paul, M.J., and Dredze, M. (2011, January 17–21). You are what you Tweet: Analyzing Twitter for public health. Proceedings of the Fifth International Conference on Weblogs and Social Media (ICWSM-2011), Barcelona, Spain.
66. Twitter Catches the Flu: Detecting Influenza Epidemics Using Twitter
67. Twitter-based influenza detection after flu peak via tweets with indirect information: Text mining study
68. Tweet classification toward twitter-based disease surveillance: New data, methods, and evaluations10.2196/12783 ↗
69. Lamb, A., Paul, M., and Dredze, M. (2013, January 9–14). Separating fact from fear: Tracking flu infections on Twitter. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA.
70. Smith, M., Broniatowski, D.A., Paul, M.J., and Dredze, M. (2015, January 25–26). Towards Real-Time Measurement of Public Epidemic Awareness: Monitoring Influenza Awareness through Twitter. Proceedings of the AAAI Workshop on World Wide Web and Public Health Intelligence, Austin, TX, USA.
71. Bian, J., Topaloglu, U., and Yu, F. (2012, January 29). Towards large-scale twitter mining for drug-related adverse events. Proceedings of the 2012 International Workshop on Smart Health and Wellbeing 2012, Maui, HI, USA.10.1145/2389707.2389713 ↗
72. Using Twitter to Examine Smoking Behavior and Perceptions of Emerging Tobacco Products10.2196/jmir.2534 ↗
73. Tracking Suicide Risk Factors through Twitter in the US10.1027/0227-5910/a000234 ↗
74. Achrekar, H., Gandhe, A., Lazarus, R., Yu, S.H., and Liu, B. (2012, January 1–4). Twitter Improves Seasonal Influenza Prediction. Proceedings of the International Conference on Health Informatics (HEALTHINF 2012), Vilamoura, Algarve.
75. Broniatowski, D.A., Paul, M.J., and Dredze, M. (2013). National and local influenza surveillance through twitter: An analysis of the 2012–2013 influenza epidemic. PLoS ONE, 8.10.1371/journal.pone.0083672 ↗
76. Predicting Asthma-Related Emergency Department Visits Using Big Data10.1109/JBHI.2015.2404829 ↗
77. Culotta, A. (2009). Detecting influenza outbreaks by analyzing Twitter messages. arXiv.10.1145/1964858.1964874 ↗
78. Enabling Next Generation Logistics and Planning for Smarter Societies10.1016/j.procs.2017.05.440 ↗
79. Automatic event detection in smart cities using big data analytics
80. (2019, November 06). Statistical Yearbook, Available online: https://www.moh.gov.sa/en/Ministry/Statistics/book/Pages/default.aspx.
81. Machine Learning Models and Algorithms for Big Data Classification: Thinking with Examples for Effective Learning
82. (2019, November 06). Saudi Arabia | Institute for Health Metrics and Evaluation. Available online: http://www.healthdata.org/saudi-arabia.
83. (2019, November 06). WHO | Saudi Arabia. Available online: https://www.who.int/countries/sau/en/.
84. (2019, November 26). CDC Global Health-Saudi Arabia, Available online: https://www.cdc.gov/globalhealth/countries/saudi_arabia/default.htm.
85. Arterial hypertension in Saudi Arabia10.5144/0256-4947.1997.170 ↗
86. Association between Hemoglobin Level and Severity of Chronic Periodontitis10.4197/med.17-1.6 ↗
87. (2020, January 09). Health Days 2017—World Hypertension Day, Available online: https://www.moh.gov.sa/en/HealthAwareness/healthDay/2017/Pages/HealthDay-2017-05-17.aspx.
88. (2019, December 17). حالات الدرن الرئوي حسب المنطقة وفئة العمر خلال عام 1439 هـ (2018م) - البيانات - البوابة السعودية للبيانات المفتوحة, Available online: https://data.gov.sa/Data/ar/dataset/pulmonary_tuberculosis_by_region-_age_group_during_1439h_-2018g-.
89. Enterprise systems and performance of future city logistics10.1080/09537287.2016.1147098 ↗
90. Ahmad, N., and Mehmood, R. (2020). Enterprise Systems for Networked Smart Cities. Smart Infrastructure and Applications, Springer.10.1007/978-3-030-13705-2_1 ↗
91. Enterprise systems: Are we ready for future sustainable cities10.1108/SCM-11-2014-0370 ↗
92. (2020, February 08). How Data Science Is Shaping the Modern NHS. Available online: https://www.newstatesman.com/science-tech/technology/2018/11/how-data-science-shaping-modern-nhs.
93. Using unsupervised clustering approach to train the Support Vector Machine for text classification10.1016/j.neucom.2015.10.137 ↗
94. Security and Privacy in Cyber-Physical Systems: A Survey of Surveys10.1109/MDAT.2017.2709310 ↗
95. LocPriS: A security and privacy preserving location based services development framework
96. Localization to enhance security and services in Wi-Fi networks under privacy constraints10.1007/978-3-642-11284-3_18 ↗
97. A framework for preserving location privacy for continuous queries10.1007/978-3-030-33582-3_77 ↗