exBAKE: Automatic Fake News Detection Model Based on Bidirectional Encoder Representations from Transformers (BERT)
Source abstract
Abstract
News currently spreads rapidly through the internet. Because fake news stories are designed to attract readers, they tend to spread faster. For most readers, detecting fake news can be challenging and such readers usually end up believing that the fake news story is fact. Because fake news can be socially problematic, a model that automatically detects such fake news is required. In this paper, we focus on data-driven automatic fake news detection methods. We first apply the Bidirectional Encoder Representations from Transformers model (BERT) model to detect fake news by analyzing the relationship between the headline and the body text of news. To further improve performance, additional news data are gathered and used to pre-train this model. We determine that the deep-contextualizing nature of BERT is best suited for this task and improves the 0.14 F-score over older state-of-the-art models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Misinformation and Its Impacts · Topic Modeling · Spam and Phishing Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Korea University, Seoul 02841, Korea
Human-Inspired AI & Computing Research Center, Korea University, Seoul 02841, Korea
Creative Information & Computer Institute, Korea University, Seoul 02841, Korea
Department of Convergence Contents, Global Cyber University, Seoul 02841, Korea
Funding
This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIT)(IITP-2018-0-00705, Algorithm design and software modeling for judge fake news based on artificial Intelligence)
References (43)
Show source-provided references
1. Pham, L. (2019, January 11–15). Transferring, Transforming, Ensembling: The Novel Formula of Identifying Fake News. Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Melbourne, Australia.
2. Liu, S., Liu, S., and Ren, L. (2019, January 11–15). Trust or Suspect? An Empirical Ensemble Framework for Fake News Classification. Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Melbourne, Australia.
3. Yang, K.C., Niven, T., and Kao, H.Y. (2019, January 11–15). Fake News Detection as Natural Language Inference. Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Melbourne, Australia.
4. Omidvar, A., Jiang, H., and An, A. (2018). Using Neural Network for Identifying Clickbaits in Online News Media. Annual International Symposium on Information Management and Big Data, Lima, Peru, 3–5 September 2018, Springer.
5. Zhou, Y. (2017). Clickbait detection in tweets using self-attentive network. arXiv.
6. Grigorev, A. (2017). Identifying clickbait posts on social media with an ensemble of linear models. arXiv.
7. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5–10). Distributed representations of words and phrases and their compositionality. Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
8. Enriching word vectors with subword information10.1162/tacl_a_00051 ↗
9. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv.10.18653/v1/N18-1202 ↗
10. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
11. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2019, September 20). Improving Language Understanding by Generative Pre-Training. Available online: https://s3-us-west-2.amazonaws.com/openai-assets/researchcovers/languageunsupervised/languageunderstandingpaper.pdf.
12. Bidirectional recurrent neural networks10.1109/78.650093 ↗
13. Wang, W.Y. (2017). “Liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv.10.18653/v1/P17-2067 ↗
14. Ruchansky, N., Seo, S., and Liu, Y. (2017, January 6–10). Csi: A hybrid deep model for fake news detection. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore.
15. Kochkina, E., Liakata, M., and Augenstein, I. (2017). Turing at semeval-2017 task 8: Sequential approach to rumour stance classification with branch-lstm. arXiv.10.18653/v1/S17-2083 ↗
16. Popat, K., Mukherjee, S., Yates, A., and Weikum, G. (2018). DeClarE: Debunking fake news and false claims using evidence-aware deep learning. arXiv.10.18653/v1/D18-1003 ↗
17. Yang, Y., Zheng, L., Zhang, J., Cui, Q., Li, Z., and Yu, P.S. (2018). TI-CNN: Convolutional neural networks for fake news detection. arXiv.
18. Rasool, T., Butt, W.H., Shaukat, A., and Akram, M.U. (2019, January 23–25). Multi-Label Fake News Detection using Multi-layered Supervised Learning. Proceedings of the 2019 11th International Conference on Computer and Automation Engineering, Perth, Australia.10.1145/3313991.3314008 ↗
19. Yang, S., Shu, K., Wang, S., Gu, R., Wu, F., and Liu, H. (February, January 27). Unsupervised fake news detection on social media: A generative approach. Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.10.1609/aaai.v33i01.33015644 ↗
20. Feng, S., Banerjee, R., and Choi, Y. (2012, January 8–14). Syntactic stylometry for deception detection. Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers—Volume 2, Jeju Island, Korea.
21. Sean, B., Doug, S., and Yuxi, P. (2019, September 20). Talos Targets Disinformation with Fake News Challenge Victory. Available online: https://blog.talosintelligence.com/2017/06/talos-fake-news-challenge.html.
22. A multilinear singular value decomposition10.1137/S0895479896305696 ↗
23. Ramos, J. (2003, January 3–8). Using tf-idf to determine word relevance in document queries. Proceedings of the First Instructional Conference on Machine Learning, Piscataway, NJ, USA.
24. Davis, R., and Proctor, C. (2019, September 20). Fake News, Real Consequences: Recruiting Neural Networks for the Fight against Fake News. Available online: https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1174/reports/2761239.pdf.
25. Hanselowski, A., PVS, A., Schiller, B., Caspelherr, F., Chaudhuri, D., Meyer, C.M., and Gurevych, I. (2018). A retrospective analysis of the fake news challenge stance detection task. arXiv.
26. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
27. Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K.Q. (2013). Training and Analysing Deep Recurrent Neural Networks. Advances in Neural Information Processing Systems 26, Curran Associates, Inc.
28. Riedel, B., Augenstein, I., Spithourakis, G.P., and Riedel, S. (2017). A simple but tough-to-beat baseline for the Fake News Challenge stance detection task. arXiv.
29. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł, and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing System, Long Beach, CA, USA.
30. Dynamically enhanced static handwriting representation for Parkinson’s disease detection10.1016/j.patrec.2019.08.018 ↗
31. Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C.H., and Kang, J. (2019). Biobert: Pre-trained biomedical language representation model for biomedical text mining. arXiv.10.1093/bioinformatics/btz682 ↗
32. Liu, Y., and Lapata, M. (2019). Text Summarization with Pretrained Encoders. arXiv.10.18653/v1/D19-1387 ↗
33. See, A., Liu, P.J., and Manning, C.D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv.10.18653/v1/P17-1099 ↗
34. Liu, Y. (2019). Fine-tune BERT for Extractive Summarization. arXiv.
35. Liu, L., Lu, Y., Yang, M., Qu, Q., Zhu, J., and Li, H. (2018, January 2–7). Generative adversarial network for abstractive text summarization. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12141 ↗
36. Paulus, R., Xiong, C., and Socher, R. (2017). A deep reinforced model for abstractive summarization. arXiv.
37. Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). Teaching Machines to Read and Comprehend. Advances in Neural Information Processing Systems 28, Curran Associates, Inc.
38. Yang, K., Lee, D., Whang, T., Lee, S., and Lim, H. (2019). EmotionX-KU: BERT-Max based Contextual Emotion Classifier. arXiv.
39. Aurelio, Y.S., de Almeida, G.M., de Castro, C.L., and Braga, A.P. (2019). Learning from imbalanced data sets with weighted cross-entropy function. Neural Process. Lett., 1–13.10.1007/s11063-018-09977-1 ↗
40. Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M.J. (2017). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.10.1007/978-3-319-67558-9_28 ↗
41. Neyshabur, B., Bhojanapalli, S., McAllester, D., and Srebro, N. (2017, January 4–9). Exploring generalization in deep learning. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
42. Zhang, Z., and Sabuncu, M. (2018, January 3–8). Generalized cross entropy loss for training deep neural networks with noisy labels. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
43. A novel feature selection algorithm for text categorization10.1016/j.eswa.2006.04.001 ↗