Automatic Fact Checking Using an Interpretable Bert-Based Architecture on COVID-19 Claims
Source abstract
Abstract
We present a neural network architecture focused on verifying facts against evidence found in a knowledge base. The architecture can perform relevance evaluation and claim verification, parts of a well-known three-stage method of fact-checking. We fine-tuned BERT to codify claims and pieces of evidence separately. An attention layer between the claim and evidence representation computes alignment scores to identify relevant terms between both. Finally, a classification layer receives the vector representation of claims and evidence and performs the relevance and verification classification. Our model allows a more straightforward interpretation of the predictions than other state-of-the-art models. We use the scores computed within the attention layer to show which evidence spans are more relevant to classify a claim as supported or refuted. Our classification models achieve results compared to the state-of-the-art models in terms of classification of relevance evaluation and claim verification accuracy on the FEVER dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Misinformation and Its Impacts · Topic Modeling · Explainable Artificial Intelligence (XAI)
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Posgrado en Ciencia e Ingeniería de la Computación, Universidad Nacional Autónoma de México, Mexico City 04510, Mexico
Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas, Universidad Nacional Autónoma de México, Mexico City 04510, Mexico
Funding
SECTEI (Mexican Government) · DGAPA-UNAM PAPIIT
References (20)
Show source-provided references
1. Fake News Detection on Social Media: A Data Mining Perspective10.1145/3137597.3137600 ↗
2. Potthast, M., Kiesel, J., Reinartz, K., Bevendorff, J., and Stein, B. (2017). A Stylometric Inquiry into Hyperpartisan and Fake News. arXiv.10.18653/v1/P18-1022 ↗
3. Kwon, S., Cha, M., Jung, K., Chen, W., and Wang, Y. (2013, January 7–10). Prominent Features of Rumor Propagation in Online Social Media. Proceedings of the IEEE International Conference on Data Mining, ICDM, Dallas, TX, USA.10.1109/ICDM.2013.61 ↗
4. Ma, J., Gao, W., Wei, Z., Lu, Y., and Wong, K.F. (2015, January 18–23). Detect Rumors Using Time Series of Social Context Information on Microblogging Websites. Proceedings of the CIKM ’15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Australia.10.1145/2806416.2806607 ↗
5. Hassan, N., Adair, B., Hamilton, J., Li, C., Tremayne, M., Yang, J., and Yu, C. (2015). The Quest to Automate Fact-Checking. Proc. Comput. J. Symp., 1–5.
6. Chernyavskiy, A., and Ilvovsky, D. (2019, January 3–7). Extract and Aggregate: A Novel Domain-Independent Approach to Factual Data Verification. Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), Hong Kong, China.10.18653/v1/D19-6612 ↗
7. Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., and Lin, J. (2019). End-to-End Open-Domain Question Answering with BERTserini. arXiv.10.18653/v1/N19-4013 ↗
8. Thorne, J., Vlachos, A., Christodoulopoulos, C., and Mittal, A. (2018, January 1–6). FEVER: A large-scale dataset for Fact Extraction and VERification. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.10.18653/v1/N18-1074 ↗
9. Hanselowski, A., Zhang, H., Li, Z., Sorokin, D., Schiller, B., Schulz, C., and Gurevych, I. (2018). UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification. arXiv.10.18653/v1/W18-5516 ↗
10. Chen, Q., Zhu, X., Ling, Z.H., Wei, S., Jiang, H., and Inkpen, D. (August, January 30). Enhanced LSTM for Natural Language Inference. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada.
11. Soleimani, A., Monz, C., and Worring, M. (2020). BERT for Evidence Retrieval and Claim Verification, ECIR.10.1007/978-3-030-45442-5_45 ↗
12. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
13. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
14. Clark, K., Khandelwal, U., Levy, O., and Manning, C. (2019, January 1). What Does BERT Look At? An Analysis of BERT’s Attention. Proceedings of the Second BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, Florence, Italy.10.18653/v1/W19-4828 ↗
15. Long Short-term Memory10.1162/neco.1997.9.8.1735 ↗
16. Reimers, N., and Gurevych, I. (2019, January 3–7). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Hong Kong, China.10.18653/v1/D19-1410 ↗
17. Language Models are Unsupervised Multitask Learners
18. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
19. Popat, K., Mukherjee, S., Yates, A., and Weikum, G. (2018). DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning. arXiv.10.18653/v1/D18-1003 ↗
20. Wang, L., Lo, K., Chandrasekhar, Y., Reas, R., Yang, J., Eide, D., Funk, K., Kinney, R., Liu, Z., and Merrill, W. (2020). CORD-19: The COVID-19 Open Research Dataset. arXiv.