Ensuring Inclusion and Diversity in Research and Research Output: A Case for a Language-Sensitive NLP Crowdsourcing Platform
Source abstract
Abstract
In the context of the debate on the need to place citizens at the center of the technological revolution, this paper makes a case for a natural language processing (NLP) crowdsourcing platform that ensures inclusion and diversity, thus making the research outcome relevant and applicable across issues and domains. This paper also makes the case that by enabling participation for a wide variety of stakeholders, this NLP crowdsourcing platform might ultimately prove useful in the decision- and policy-making processes at city, community, and country levels. Against the backdrop of the debates on artificial intelligence (AI) and NLP research, and considering substantial differentiation specific to the Arab language, this paper introduces and evaluates an Arab language-sensitive NLP crowdsourcing platform. The value of the platform and its accuracy are measured via the System Usability Scale (SUS), where it scores 72.5, i.e., above the accepted usability average. These findings are crucial for NLP research and the research community in general. They are equally promising in view of the practical application of the research findings.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Imbalanced Data Classification Techniques · Misinformation and Its Impacts · Hate Speech and Cyberbullying Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
Effat College of Business, Effat University, Jeddah 34689, Saudi Arabia
Funding
Deanship of Scientific Research, King Abdulaziz University, Jeddah, Saudi Arabia
References (37)
Show source-provided references
1. The trainer, the verifier, the imitator: Three ways in which human platform workers support artificial intelligence10.1177/2053951720919776 ↗
2. Tweeting and mining OECD-related microcontent in the post-truth era: A cloud-based app10.1016/j.chb.2019.03.022 ↗
3. Mora-Cantallops, M., Sánchez-Alonso, S., and Visvizi, A. (2019). The influence of external political events on social networks: The case of the Brexit Twitter Network. J. Ambient. Intell. Hum. Comput., 1–13.10.1007/s12652-019-01273-7 ↗
4. A hybrid Persian sentiment analysis framework: Integrating dependency grammar based rules and deep neural networks10.1016/j.neucom.2019.10.009 ↗
5. Towards the automatic detection and identification of English puns10.7592/EJHR2016.4.1.miller ↗
6. Using natural language processing to extract clinically useful information from Chinese electronic medical records10.1016/j.ijmedinf.2019.01.004 ↗
7. Crowston, K. (2012). Amazon mechanical turk: A research tool for organizations and information systems scholars. Shaping the Future of ICT Research. Methods and Approaches, Springer.10.1007/978-3-642-35142-6_14 ↗
8. (2020, March 07). Figure Eight. Available online: https://www.figure-eight.com/platform/.
9. (2020, March 07). Lionbridge. Available online: https://lionbridge.ai/services/crowdsourcing/.
10. Using CrowdFlower to study the relationship between self-reported violations and traffic accidents10.1016/j.promfg.2015.07.514 ↗
11. Making better use of the crowd: How crowdsourcing can advance machine learning research
12. The face of internet recruitment: Evaluating the labor markets of online crowdsourcing platforms in China
13. Startup Initiative Response Analysis (SIRA) Framework for Analyzing Startup Initiatives on Twitter10.1109/ACCESS.2020.2965181 ↗
14. Alshutayri, A., and Atwell, E. (2018). Creating an Arabic dialect text corpus by exploring Twitter, Facebook, and online newspapers. OSACT 3 Proceedings. OSACT 3 the 3rd Workshop on Open-Source Arabic Corpora and Processing Tools, Co-Located with LREC 2018, Miyazaki, Japan, 8 May 2018, LREC.
15. Alsarsour, I., Mohamed, E., Suwaileh, R., and Elsayed, T. (2018, January 7–12). Dart: A large dataset of dialectal Arabic tweets. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
16. Suar: Towards building a corpus for the Saudi dialect10.1016/j.procs.2018.10.462 ↗
17. Mubarak, H. (2017, January 9–11). Crowdsourcing Speech and Language Data for Resource-Poor Languages. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics, Cairo, Egypt.10.1007/978-3-319-64861-3_41 ↗
18. Alhelbawy, A., Massimo, P., and Kruschwitz, U. (2016, January 23–28). Towards a corpus of violence acts in arabic social media. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), Portorož, Slovenia.
19. Arabic dialect identification10.1162/COLI_a_00169 ↗
20. Investigating the effect of combining GRU neural networks with handcrafted features for religious hatred detection on Arabic Twitter space10.1007/s13278-019-0587-5 ↗
21. Altruistic crowdsourcing for arabic speech corpus annotation10.1016/j.procs.2017.10.102 ↗
22. Akila, G., El-Menisy, M., Khaled, O., Sharaf, N., Tarhony, N., and Abdennadher, S. (2015, January 14–20). Kalema: Digitizing arabic content for accessibility purposes using crowdsourcing. Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics, Cairo, Egypt.10.1007/978-3-319-18117-2_49 ↗
23. Almeman, K., and Lee, M. (2013, January 12–14). Automatic building of arabic multi dialect text corpora by bootstrapping dialect words. Proceedings of the 2013 1st International Conference on Communications, Signal Processing, and Their Applications (ICCSPA), Sharjah, UAE.10.1109/ICCSPA.2013.6487247 ↗
24. Baly, R., Khaddaj, A., Hajj, H., El-Hajj, W., and Shaban, K.B. (2019). Arsentd-lev: A multi-topic corpus for target-based sentiment analysis in arabic levantine tweets. arXiv.
25. Farra, N., McKeown, K., and Habash, N. (2015, January 30). Annotating targets of opinions in arabic using crowdsourcing. Proceedings of the Second Workshop on Arabic Natural Language Processing, Beijing, China.10.18653/v1/W15-3210 ↗
26. Al-Muzaini, H.A., Al-Yahya, T.N., and Benhidour, H. (2018). Automatic Arabic Image Captioning using RNN-LST M-Based Language Model and CNN. Int. J. Adv. Comput. Sci. Appl., 9.10.14569/IJACSA.2018.090610 ↗
27. (2020, September 03). Tashkeel Crowdsourcing Platform [تشكيل]. Available online: http://crowdsourcing.kauwld.com/.
28. Öhlen, J., Morse, J.M., and Niehaus, L. (2010). Mixed Method Design: Principles and Procedures. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, Routledge.
29. Denzin, N.K., and Lincoln, Y.S. (2008). Introduction: The discipline and practice of qualitative research. The Sage Handbook of Qualitative Research, Sage Publications Ltd.
30. Nested analysis as a mixed-method strategy for comparative research10.1017/S0003055405051762 ↗
31. Brooke, J. (1996). A Quick and Dirty Usability Scale.’Usability Evaluation in Industry, Taylor & Francis Ltd.
32. SUS: A retrospective
33. Sauro, J. (2020, September 06). Measuring Usability with the System Usability Scale (SUS). Available online: www.measuringu.com.
34. Lewis, J.R., and Sauro, J. (2009, January 19–24). The factor structure of the system usability scale. Proceedings of the International Conference on Human Centered Design, San Diego, CA, USA.10.1007/978-3-642-02806-9_12 ↗
35. (2020, July 15). Amazon Mechanical Turk. Available online: https://www.mturk.com/.
36. (2020, July 15). Appen. Available online: https://appen.com/.
37. (2019, December 16). Techopedia–IT Dictionary for Computer Terms and Tech Definitions. Available online: https://www.techopedia.com/dictionary.