journal article·2026-07-06Open access

TERN: Type-Aware Evidence Reasoning for Multimodal Fake News Detection

Mingshu Zhang, Hongyu Jin, Yuechuan Zhang, Bin Wei, Yaxuan Wang

Applied Sciences· Volume 16· Issue 13

Abstract

Multimodal fake news detection remains challenging because deceptive posts exhibit heterogeneous manipulation patterns, while most existing methods still rely on a unified fusion strategy. This mismatch limits their ability to adapt to different evidence preferences across samples, encourages entanglement between deception cues and topical semantics, and weakens decision making when textual, visual, and cross-modal signals conflict. To address these issues, we propose TERN, a type-aware evidence reasoning network for multimodal fake news detection. TERN induces latent deception types from image-side multimodal features through prototype-based clustering, uses the induced assignments as structural priors for downstream veracity prediction, disentangles type-discriminative factors from semantic content, and performs type-conditioned hierarchical reasoning over text semantics, image authenticity, and cross-modal consistency. Experiments on MR2-Chinese, MR2-English, Weibo, and PHEME show that TERN achieves an average accuracy of 93.21% and an average F1 score of 91.09% while also improving Matthews correlation coefficient over representative multimodal baselines.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Misinformation and Its Impacts · Spam and Phishing Detection · Topic Modeling

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

College of Cryptography Engineering, Engineering University of People’s Armed Police, Xi’an 710086, China

Key Laboratory of Network and Information Security, Engineering University of People’s Armed Police, Xi’an 710086, China

References (54)

Show source-provided references
1. The spread of true and false news online10.1126/science.aap9559 ↗
2. The science of fake news10.1126/science.aao2998 ↗
3. Wu, Y., Zhan, P., Zhang, Y., Wang, L., and Xu, Z. (2021). Multimodal fusion with co-attention networks for fake news detection. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Association for Computational Linguistics.10.18653/v1/2021.findings-acl.226 ↗
4. Qian, S., Wang, J., Hu, J., Fang, Q., and Xu, C. (2021). Hierarchical multi-modal contextual attention network for fake news detection. SIGIR ’21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery.10.1145/3404835.3462871 ↗
5. Chen, Y., Li, D., Zhang, P., Sui, J., Lv, Q., Tun, L., and Shang, L. (2022). Cross-modal ambiguity learning for multimodal fake news detection. WWW ’22: Proceedings of the ACM Web Conference 2022, Association for Computing Machinery.10.1145/3485447.3511968 ↗
6. Wang, L., Zhang, C., Xu, H., Xu, Y., Xu, X., and Wang, S. (2023). Cross-modal contrastive learning for multimodal fake news detection. MM ’23: Proceedings of the 31st ACM International Conference on Multimedia, Association for Computing Machinery.10.1145/3581783.3613850 ↗
7. A multifaceted reasoning network for explainable fake news detection10.1016/j.ipm.2024.103822 ↗
8. Multimodal fusion and inconsistency reasoning for explainable fake news detection10.1016/j.inffus.2023.101944 ↗
9. Wang, J., Zhu, Z., Liu, C., Li, R., and Wu, X. (2024). LLM-Enhanced multimodal detection of fake news. PLoS ONE, 19.10.1371/journal.pone.0312240 ↗
10. Ai, W., Tan, Y., Shou, Y., Meng, T., Chen, H., He, Z., and Li, K. (2026). The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection. arXiv.10.1016/j.cosrev.2026.100893 ↗
11. Dong, L., Chen, Y., Ke, W., Wu, H., Deng, L., and Liao, G. (2026). Multimodal Fake News Detection via Evidence Retrieval and Visual Forensics with Large Vision-Language Models. Information, 17.10.3390/info17040317 ↗
12. SR-CIBN: Semantic relationship-based consistency and inconsistency balancing network for multimodal fake news detection10.1016/j.neucom.2025.129997 ↗
13. Multi-View mutual learning network for multimodal fake news detection10.1016/j.eswa.2025.127407 ↗
14. A comprehensive survey of multimodal fake news detection techniques: Advances, challenges, and opportunities10.1007/s13735-023-00296-3 ↗
15. Zhu, Y., Wang, Y., and Yu, Z. (2025, January 16–22). Multimodal Fake News Detection: MFND Dataset and Shallow-Deep Multitask Learning. Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, Montreal, QC, Canada.10.24963/ijcai.2025/891 ↗
16. Representation learning: A review and new perspectives10.1109/TPAMI.2013.50 ↗
17. Human cognition-based consistency inference networks for multi-modal fake news detection10.1109/TKDE.2023.3280555 ↗
18. Wu, L., Liu, P., and Zhang, Y. (2023, January 7–14). See how you read? Multi-reading habits fusion reasoning for multi-modal fake news detection. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i11.26609 ↗
19. Caron, M., Bojanowski, P., Joulin, A., and Douze, M. (2018, January 8–14). Deep clustering for unsupervised learning of visual features. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_9 ↗
20. Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2020, January 6–12). Unsupervised learning of visual features by contrasting cluster assignments. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual.
21. Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., Su, L., and Gao, J. (2018). EANN: Event adversarial neural networks for multi-modal fake news detection. KDD ’18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery.10.1145/3219819.3219903 ↗
22. Singhal, S., Shah, R.R., Chakraborty, T., Kumaraguru, P., and Satoh, S. (2019, January 11–13). SpotFake: A multi-modal framework for fake news detection. Proceedings of the 2019 IEEE Fifth International Conference on Multimedia Big Data, Singapore.10.1109/BigMM.2019.00-44 ↗
23. A Comprehensive Survey of Multimodal Fake News Detection: Datasets, Methods, and Challenges10.53941/tai.2026.100009 ↗
24. Harris, S., Ta, V.T., Trovati, M., Nakhla, G., Latif, F., and Korkontzelos, I. (2026). Multimodal misinformation detection across diverse languages using RAG and LLMs. J. Intell. Inf. Syst., 1–28.10.1007/s10844-026-01042-x ↗
25. Zhou, X., Wu, J., and Zafarani, R. (2020, January 11–14). SAFE: Similarity-aware multi-modal fake news detection. Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Singapore.10.1007/978-3-030-47436-2_27 ↗
26. Positive unlabeled fake news detection via multi-modal masked transformer network10.1109/TMM.2023.3263552 ↗
27. MRAN: Multimodal relationship-aware attention network for fake news detection10.1016/j.csi.2023.103822 ↗
28. Shen, X., Huang, M., Hu, Z., Cai, S., and Zhou, T. (2024). Multimodal Fake News Detection with Contrastive Learning and Optimal Transport. Front. Comput. Sci., 6.10.3389/fcomp.2024.1473457 ↗
29. Wei, Z., Pan, H., Qiao, L., Niu, X., Dong, P., and Li, D. (2022, January 27). Cross-modal knowledge distillation in multi-modal fake news detection. Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP, Singapore.10.1109/ICASSP43922.2022.9747280 ↗
30. Zhou, Y., Yang, Y., Ying, Q., Qian, Z., and Zhang, X. (2023). Multi-modal fake news detection on social media via multi-grained information fusion. ICMR ’23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval, Association for Computing Machinery.10.1145/3591106.3592271 ↗
31. Improving multimodal fake news detection by leveraging cross-modal content correlation10.1016/j.ipm.2025.104120 ↗
32. Zeng, F., Li, W., Gao, W., and Pang, Y. (2024). Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs. Findings of the Association for Computational Linguistics: EMNLP 2024, Association for Computational Linguistics.10.18653/v1/2024.findings-emnlp.613 ↗
33. Hu, S., Hu, J., and Zhang, H. (2025). Synergizing LLMs with Global Label Propagation for Multimodal Fake News Detection. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics.
34. Li, Y., Lu, W., Yu, H., and Wang, Y. (2026). Retrieval-Augmented Multimodal Model for Fake News Detection. arXiv.
35. Qi, P., Cao, J., Li, X., Liu, H., Sheng, Q., Mi, X., He, Q., Lv, Y., Guo, C., and Yu, Y. Improving fake news detection by using an entity-enhanced framework to fuse diverse multimodal clues. MM ’21: Proceedings of the 29th ACM International Conference on Multimedia.
36. Hu, L., Yang, T., Zhang, L., Zhong, W., Tang, D., Shi, C., Duan, N., and Zhou, M. (2021). Compare to the knowledge: Graph neural fake news detection with external knowledge. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics.10.18653/v1/2021.acl-long.62 ↗
37. Cao, B., Wu, Q., Cao, J., Liu, B., and Gui, J. (March, January 25). External reliable information-enhanced multimodal contrastive learning for fake news detection. Proceedings of the AAAI, Philadelphia, PA, USA.
38. Yan, Z., Qi, P., Hsu, W., and Lee, M.L. (2025, January 4–9). TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detection. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China.10.18653/v1/2025.emnlp-main.284 ↗
39. TFGIN: Tight-fitting graph inference network for table-based fact verification
40. Chen, J., Li, Y., Ng, K.C., Wang, H., and Zhang, L.-J. (2026, January 20–27). Toward Multimodal Fake News Detection by Multi-perspective Rationale Generation and Verification. Proceedings of the AAAI Conference on Artificial Intelligence, Singapore.10.1609/aaai.v40i1.36965 ↗
41. He, K., Chen, X., Xie, S., Li, Y., Dollar, P., and Girshick, R. (2022, January 27). Masked autoencoders are scalable vision learners. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition CVPR, New Orleans, LA, USA.10.1109/CVPR52688.2022.01553 ↗
42. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning transferable visual models from natural language supervision. Proceedings of the ICML, Virtual.
43. Howard, A., Sandler, M., Chen, B., Chu, G., Chen, L., Chen, B., Tan, M., Wang, W., Zhu, Y., and Pang, R. (November, January 27). Searching for MobileNetV3. Proceedings of the ICCV, Seoul, Republic of Korea.
44. Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
45. Honnibal, M., and Montani, I. (2017). spaCy 2: Natural Language Understanding with Bloom Embeddings, Convolutional Neural Networks and Incremental Parsing, Explosion AI.
46. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv.
47. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis10.1016/0377-0427(87)90125-7 ↗
48. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the NeurIPS, Long Beach, CA, USA.
49. Hu, X., Guo, Z., Chen, J., Wen, L., and Yu, P.S. (2023, January 23–27). MR2: A benchmark for multimodal retrieval-augmented rumor detection in social media. Proceedings of the ACM SIGIR, Taipei, Taiwan.10.1145/3539618.3591896 ↗
50. Jin, Z., Cao, J., Guo, H., Zhang, Y., and Luo, J. (2017, January 23–27). Multimodal fusion with recurrent neural networks for rumor detection on microblogs. Proceedings of the 25th ACM International Conference on Multimedia (MM), Mountain View, CA, USA.10.1145/3123266.3123454 ↗
51. Zubiaga, A., Liakata, M., and Procter, R. (2017). Exploiting context for rumour detection in social media. Social Informatics, Springer.10.1007/978-3-319-67217-5_8 ↗
52. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
53. Loshchilov, I., and Hutter, F. (2019). Decoupled Weight Decay Regularization. arXiv.
54. Multimodal fake news detection through data augmentation-based contrastive learning10.1016/j.asoc.2023.110125 ↗