Causality Mining in Natural Languages Using Machine and Deep Learning Techniques: A Survey
Source abstract
Abstract
The era of big textual corpora and machine learning technologies have paved the way for researchers in numerous data mining fields. Among them, causality mining (CM) from textual data has become a significant area of concern and has more attention from researchers. Causality (cause-effect relations) serves as an essential category of relationships, which plays a significant role in question answering, future events predication, discourse comprehension, decision making, future scenario generation, medical text mining, behavior prediction, and textual prediction entailment. While, decades of development techniques for CM are still prone to performance enhancement, especially for ambiguous and implicitly expressed causalities. The ineffectiveness of the early attempts is mainly due to small, ambiguous, heterogeneous, and domain-specific datasets constructed by manually linguistic and syntactic rules. Many researchers have deployed shallow machine learning (ML) and deep learning (DL) techniques to deal with such datasets, and they achieved satisfactory performance. In this survey, an effort has been made to address a comprehensive review of some state-of-the-art shallow ML and DL approaches in CM. We present a detailed taxonomy of CM and discuss popular ML and DL approaches with their comparative weaknesses and strengths, applications, popular datasets, and frameworks. Lastly, the future research challenges are discussed with illustrations of how to transform them into productive future research directions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Advanced Text Analysis Techniques · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Technology, Jilin University, Changchun 130012, China
Quaid-e-Azam College of Commerce, University of Peshawar, Peshawar 25000, Pakistan
Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, Johor, Parit Raja, Batu Pahat 86400, Malaysia
References (100+)
Show source-provided references
1. Extracting causation knowledge from natural language texts10.1002/int.20069 ↗
2. Luo, Z., Sha, Y., Zhu, K.Q., and Wang, Z. (2016, January 25–29). Commonsense Causal Reasoning between Short Texts. Proceedings of the Fifteenth International Conference on Principles of Knowledge Representation and Reasoning, KR’16, Cape Town, South Africa.
3. Khoo, C., Chan, S., and Niu, Y. (2002). The Many Facets of the Cause-Effect Relation. The Semantics of Relationships, Springer.10.1007/978-94-017-0073-3_4 ↗
4. Theodorson, G., and Theodorson, A. (1969). A Modern Dictionary of Sociology, Crowell.
5. Hassanzadeh, O., Bhattacharjya, D., and Feblowitz, M. (2019, January 10–16). Answering Binary Causal Questions Through Large-Scale Text Mining: An Evaluation Using Cause-Effect Pairs from Human Experts. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI), Macao, China.10.24963/ijcai.2019/695 ↗
6. Causal inference in statistics: An overview10.1214/09-SS057 ↗
7. Girju, R. (, January July). Automatic detection of causal relations for question answering. Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering, Sapporo, Japan.10.3115/1119312.1119322 ↗
8. Automatic extraction of cause-effect information from newspaper text without knowledge-based inferencing10.1093/llc/13.4.177 ↗
9. Radinsky, K., Davidovich, S., and Markovitch, S. (2012, January 16–20). Learning causality for news events prediction. Proceedings of the 21st International Conference on World Wide Web, Lyon, France.10.1145/2187836.2187958 ↗
10. Scalable techniques for mining causal structures10.1023/A:1009891813863 ↗
11. Riaz, M., and Girju, R. (2010, January 20–22). Another Look at Causality: Discovering Scenario-Specific Contingency Relationships with No Supervision. Proceedings of the 2010 IEEE Fourth International Conference on Semantic Computing, Pittsburgh, PA, USA.10.1109/ICSC.2010.19 ↗
12. Hashimoto, C., Torisawa, K., Kloetzer, J., and Sano, M. (2014, January 22–27). Toward future scenario generation: Extracting event causality exploiting semantic relation, coantext, and association features. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, MA, USA.10.3115/v1/P14-1093 ↗
13. Extracting a causal network of news topics
14. Causality patterns for detecting adverse drug reactions from social media: Text mining approach10.2196/publichealth.8214 ↗
15. Richardson, M., and Burges, C. (2013, January 18–21). Mctest: A challenge dataset for the open-domain machine comprehension of text. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
16. Berant, J., and Srikumar, V. (2014, January 25–29). Modeling biological processes for reading comprehension. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1159 ↗
17. Hassanzadeh, O., Bhattacharjya, D., Feblowitz, M., Srinivas, K., Perrone, M., Sohrabi, S., and Katz, M. (2020, January 7–12). Causal Knowledge Extraction through Large-Scale Text Mining. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i09.7092 ↗
18. Using cause-effect relations in text to improve information retrieval precision10.1016/S0306-4573(00)00022-4 ↗
19. Khoo, C., and Chan, S. (2000, January 3–6). Extracting causal knowledge from a medical database using graphical patterns. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, Hong Kong, China.10.3115/1075218.1075261 ↗
20. Causal protein-signaling networks derived from multiparameter single-cell data10.1126/science.1105809 ↗
21. Araúz, P.L., and Faber, P. (2012, January 22). Causality in the Specialized Domain of the Environment. Proceedings of the Semantic Relations-II, Enhancing Resources and Applications Workshop Programme Lütfi Kirdar, Istanbul Exhibition and Congress Centre, Istanbul, Turkey.
22. Representing causation
23. Talmy, L. (2000). Toward a Cognitive Semantics, MIT Press. Volume I: Concept Structuring Systems.10.7551/mitpress/6847.001.0001 ↗
24. Toward a useful concept of causality for lexical semantics10.1093/jos/ffh024 ↗
25. Ideas about causation in philosophy and psychology
26. Scaria, A., Berant, J., Wang, M., Clark, P., and Lewis, J. (2013, January 18–21). Learning biological processes with global constraints. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
27. Predicting COVID-19 incidence through analysis of google trends data in iran: Data mining and deep learning pilot study10.2196/18828 ↗
28. (2021, October 03). FAQ about Google Trends Data—Trends Help. Available online: https://support.google.com/trends/answer/4365533?hl=en#.
29. Blanco, E., Castell, N., and Moldovan, D. (2008, January 28–30). Causal Relation Extraction. Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco.
30. Hendrickx, I., Kim, S.N., Kozareva, Z., Nakov, P., Diarmuidó, D., Diarmuidó, S., Padó, S., Pennacchiotti, M., Romano, L., and Szpakowicz, S. (2019). SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals. arXiv.
31. Sorgente, A., Vettigli, G., and Mele, F. (2013, January 6). Automatic Extraction of Cause-Effect Relations in Natural Language Text. Proceedings of the 7th International Workshop on Information Filtering and Retrieval Co-Located with the 13th Conference of the Italian Association for Artificial Intelligence (AI*IA 2013), Turin, Italy.
32. Cresswell, M. (1981). Adverbs of causation. Words, Worlds, and Contexts: New Approaches in Word Semantics, De Gruyter.
33. Simpson, J. (1983). Resultatives. Papers in Lexical-Functional Grammar, Indiana University Linguistics Club.
34. Causal linking in spoken and written English10.1111/j.1467-9582.1984.tb00734.x ↗
35. Nastase, V. (2004). Semantic Relations across Syntactic Levels, University of Ottawa.
36. Sadek, J. (2013). Automatic detection of arabic causal relations. International Conference on Application of Natural Language to Information Systems (NLDB’13), Springer.10.1007/978-3-642-38824-8_48 ↗
37. Garcia, D. (1997, January 15–18). COATIS, an NLP system to locate expressions of actions connected by causality links. Proceedings of the Knowledge Acquisition, Modeling and Management: 10th European Workshop, EKAW’97, Sant Feliu de Guixols, Catalonia, Spain.10.1007/BFb0026799 ↗
38. Asghar, N. (2016). Automatic extraction of causal relations from natural language texts: A comprehensive survey. arXiv.
39. Causality and statistical learning10.1086/662659 ↗
40. Machine learning methods for estimating heterogeneous causal effects
41. Distinguishing cause from effect using observational data: Methods and benchmarks
42. Causal discovery and inference: Concepts and recent methodological advances10.1186/s40535-016-0018-x ↗
43. A Survey of Learning Causality with Data: Problems and Methods
44. Quinlan, J. (2014). C4.5: Programs for Machine Learning, Elsevier.
45. Charniak, E. (May, January 29). A maximum-entropy-inspired parser. Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics, Seattle, WA, USA.
46. Rosario, B., and On, M.H. (2001, January 3–4). Classifying the semantic relations in noun compounds via a domain-specific lexical hierarchy. Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing (EMNLP-01), Pittsburgh, PA, USA.
47. Chang, D., and KS, C. (2004, January 22–24). Causal relation extraction using cue phrase and lexical pair probabilities. Proceedings of the 1st International Joint Conference on Natural Language Processing (IJCNLP’04), Hainan, China.
48. Marcu, D., and Echihabi, A. (2002, January 6–12). An unsupervised approach to recognizing discourse relations. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073145 ↗
49. Rink, B., and On, S.H. (2010, January 15–16). Utd: Classifying semantic relations by combining lexical and semantic resources. Proceedings of the 5th International Workshop on Semantic Evaluation; Association for Computational Linguistics, Uppsala, Sweden.
50. Sil, A., Huang, F., and Series, A.Y. (2010, January 11–13). Extracting action and event semantics from web text. Proceedings of the 2010 AAAI Fall Symposium Series, Westin Arlington Gateway, Arlington, Virginia.
51. Pal, S., Pakray, P., and Das, D. (2010, January 15–16). JU: A supervised approach to identify semantic relations from paired nominals. Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010, Uppsala University, Uppsala, Sweden.
52. Li, Z., Ding, X., Liu, T., Hu, J.E., and Durme, B. (2020). Van Guided Generation of Cause and Effect. arXiv, 1–8.10.24963/ijcai.2020/502 ↗
53. Schank, R.C. (1983). Dynamic Memory: A Theory of Reminding and Learning in Computers and People, Cambridge University Press.
54. Szpakowicz, S., and Nastase, V. (2003, January 15–17). Exploring noun-modifier semantic relations. Proceedings of the Fifth International Workshop on Computational Semantics (IWCS-5), Tilburg University, Tilburg, The Netherlands.
55. Tapanainen, P., and Natural, T.J. (April, January 31). A non-projective dependency parser. Proceedings of the Fifth Conference on Applied Natural Language Processing, Washington, DC, USA.
56. A knowledge-rich approach to identifying semantic relations between nominals10.1016/j.ipm.2009.09.002 ↗
57. Girju, R., Nakov, P., Nastase, V., Szpakowicz, S., Turney, P., and Yuret, D. (2007, January 23). Semeval-2007 task 04: Classification of semantic relations between nominals. Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), Prague, Czech Republic.10.3115/1621474.1621477 ↗
58. Pakray, P., and Gelbukh, A. (2014). An open-domain cause-effect relation detection from paired nominals. Mexican International Conference on Artificial Intelligence, Springer.10.1007/978-3-319-13650-9_24 ↗
59. Bethard, S., and HLT, J.M. (2008, January 16–17). Learning semantic links from a corpus of parallel temporal and causal relations. Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, Columbus, OH, USA.10.3115/1557690.1557740 ↗
60. Bethard, S., Corvey, W., Klingenstein, S., and Martin, J.H. (2008, January 28–30). Building a Corpus of Temporal-Causal Structure. Proceedings of the European Language Resources Association (ELRA), Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco.
61. Rink, B., and Bejan, C. (2010, January 19–21). Learning textual graph patterns to detect causal event relations. Proceedings of the Twenty-Third International FLAIRS Conference, Datona Beach, FL, USA.
62. Do, Q., Chan, Y.S., and Roth, D. (2011, January 27–31). Minimally Supervised Event Causality Identification. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing(EMNLP 2011), Edinburgh, Scotland, UK.
63. A pdtb-styled end-to-end discourse parser10.1017/S1351324912000307 ↗
64. Riaz, M., and Girju, R. (2014, January 26). Recognizing Causality in Verb-Noun Pairs via Noun and Verb Semantics. Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), Gothenburg, Sweden.10.3115/v1/W14-0707 ↗
65. Multi level causal relation identification using extended features10.1016/j.eswa.2014.05.044 ↗
66. Kingsbury, P., and Palmer, M. (2002, January 29–31). From TreeBank to PropBank. Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02), Las Palmas, Canary Islands, Spain.
67. Mirza, P., and Kessler, F.B. (2014, January 22–27). Extracting Temporal and Causal Relations between Events. Proceedings of the ACL 2014 Student Research Workshop, Baltimore, MA, USA.10.3115/v1/P14-3002 ↗
68. Mirza, P., Sprugnoli, R., Tonelli, S., and Speranza, M. (2015). Annotating Causality in the TempEval-3 Corpus, Association for Computational Linguistics (ACL).10.3115/v1/W14-0702 ↗
69. Event causality extraction based on connectives analysis10.1016/j.neucom.2015.09.066 ↗
70. Hidey, C., and Mckeown, K. (2016, January 7–12). Identifying Causal Relations Using Parallel Wikipedia Articles. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany.10.18653/v1/P16-1135 ↗
71. Extracting Causal Relations from Emergency Cases Based on Conditional Random Fields10.1016/j.procs.2017.08.252 ↗
72. Rehbein, I., and Ruppenhofer, J. (2017, January 3). Catching the Common Cause: Extraction and Annotation of Causal Relations and their Participants. Proceedings of the 11th Linguistic Annotation Workshop, Valencia, Spain.10.18653/v1/W17-0813 ↗
73. Koehn, P. (2005, January 12–16). Europarl: A Parallel Corpus for Statistical Machine Translation. Proceedings of the MT Summit, Phuket, Thailand.
74. Dunietz, J., Levin, L., and Carbonell, J. (2017, January 3). The BECauSE Corpus 2.0: Annotating Causality and Overlapping Relations. Proceedings of the 11th Linguistic Annotation Workshop, Valencia, Spain.10.18653/v1/W17-0812 ↗
75. Zhao, S., Jiang, M., Liu, M., Qin, B., and Liu, T. (September, January 29). CausalTriad: Toward Pseudo Causal Relation Discovery and Hypotheses Generation from Medical Text Data. Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health, Washington, DC, USA.10.1145/3233547.3233555 ↗
76. Ning, Q., Feng, Z., Wu, H., and Roth, D. (2018, January 15–20). Joint reasoning for temporal and causal relations. Proceedings of the ACL 2018—56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-1212 ↗
77. Craciunescu, T., Murari, A., and Gelfusa, M. (2019). Causality detection methods applied to the investigation of malaria epidemics. Entropy, 21.10.3390/e21080784 ↗
78. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A.K., and Webber, B.L. (2008, January 28–30). The Penn Discourse TreeBank 2.0. Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco.
79. The TIMEBANK Corpus
80. Learning to Predict from Textual Data10.1613/jair.3865 ↗
81. Riaz, M., and Girju, R. (2013, January 22–24). Toward a Better Understanding of Causality between Verbal Events: Extraction and Analysis of the Causal Power of Verb-Verb Associations. Proceedings of the 14th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial), Metz, France.
82. Incremental Construction of Causal Network from News Articles
83. Cross-Sentence N -ary Relation Extraction with Graph LSTMs10.1162/tacl_a_00049 ↗
84. Marcus, M., Kim, G., Marcinkiewicz, M.A., Macintyre, R., Bies, A., Ferguson, M., Katz, K., and Schasberger, B. (1994, January 8–11). The Penn TreeBank: Annotating Predicate Argument Structure. Proceedings of the Human Language Technology: Proceedings of a Workshop, Plainsboro, NJ, USA.10.3115/1075812.1075835 ↗
85. Sandhaus, E. (2008, January 17). The new york times annotated corpus. Proceedings of the Linguistic Data Consortium, University of Philadelphia, Philadelphia, PA, USA.
86. Smith, N.A., Cardie, C., Washington, A.L., and Wilkerson, J.D. (2014, January 26). Overview of the 2014 NLP Unshared Task in PoliInformatics. Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, Baltimore, MD, USA.10.3115/v1/W14-2505 ↗
87. Ide, N., Baker, C., Fellbaum, C., and Passonneau, R. (2010, January 11–16). The Manually Annotated Sub-Corpus: A Community Resource for and By the People. Proceedings of the ACL 2010 Conference Short Papers, Stroudsburg, PA, USA.
88. UZMAY, G., and Gokce, K. (2018, January 29–30). The Causality Effect of Interest in the Financial Crisis and Oil Market on Food Prices: A Case Study of Internet Search Engine Behavior. Proceedings of the IX. IBANESS Congress Series, Edirne, Turkey.
89. Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique
90. Recurrence plots of dynamical systems
91. Investigating causal relations by econometric models and cross-spectral methods
92. Kernel method for nonlinear Granger causality10.1103/PhysRevLett.100.144103 ↗
93. Causal decomposition in the mutual causation system
94. Craciunescu, T., Murari, A., and Gelfusa, M. (2018). Improving entropy estimates of complex network topology for the characterization of coupling in dynamical systems. Entropy, 20.10.3390/e20110891 ↗
95. Haque, U., Hashizume, M., Glass, G.E., Dewan, A.M., Overgaard, H.J., and Yamamoto, T. (2010). The role of climate variability in the spread of malaria in bangladeshi highlands. PLoS ONE, 5.10.1371/journal.pone.0014341 ↗
96. The role of El Niño southern oscillation (ENSO) on variations of monthly Plasmodium falciparum malaria cases at the cayenne general hospital, 1996–200910.1186/1475-2875-10-100 ↗
97. Causality analysis of Google Trends and dengue incidence in Bandung, Indonesia with linkage of digital data modeling: Longitudinal observational study10.2196/17633 ↗
98. A tutorial survey of architectures, algorithms, and applications for deep learning
99. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986). Learning Internal Representations by Error Propagation. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Foundations, MIT Press.
100. Deep learning in neural networks: An overview10.1016/j.neunet.2014.09.003 ↗