journal article·2025-11-28Open access

Fake News Detection in Short Videos by Integrating Semantic Credibility and Multi-Granularity Contrastive Learning

Yukun Yang, Xiwei Shi, Haoxu Li, Buwei Fan, Yijia Xu

Applied Sciences· Volume 15· Issue 23

Abstract

Short videos have become a primary medium for news delivery, but their low cost, rapid diffusion, and multimodal nature make misinformation easier to generate and harder to verify. Existing methods often rely on single-modality cues or shallow cross-modal correlations, making it difficult to distinguish manipulations from benign edits and limiting interpretability. We propose a robust and interpretable framework for fake news detection in short videos. It combines LLM-based video understanding and online search for multi-dimensional credibility assessment, employs RoBERTa and capsule networks for semantic aggregation, and leverages a diffusion model with multi-granularity contrastive learning to enforce cross-modal consistency. A neuro-symbolic rule engine further calibrates predictions with logical constraints to provide traceable rationales. Experiments on the FakeSV dataset demonstrate an accuracy of 89.11% and an F1 score of 89.53%, significantly outperforming mainstream baseline models. This performance surpasses the current state-of-the-art OpEvFake model, which recorded an accuracy of 87.80% and an F1 score of 87.71%, and also substantially outperforms the representative short-video detection method SV-FEND, which achieved an accuracy of 81.69% and an F1 score of 81.78%. The framework shows robustness against emotional manipulation, title–content inconsistency, audio–video desynchronization, and local tampering, while offering explanatory evidence through rule triggers and modality contributions.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Misinformation and Its Impacts · Generative Adversarial Networks and Image Synthesis · Spam and Phishing Detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Cyber Science and Engineering, Sichuan University, Chengdu 610207, China

Funding

National Natural Science Foundation of China

References (42)

Show source-provided references
1. Business of Apps (2025, May 26). TikTok App Report. Available online: https://www.businessofapps.com/data/tiktok-app-report/.
2. Social media and fake news in the 2016 election10.1257/jep.31.2.211 ↗
3. Misinformation during the COVID-19 outbreak in China: Cultural, social and political entanglements10.1109/TBDATA.2021.3055758 ↗
4. Shahzad, S.A., Hashmi, A., Peng, Y.-T., Tsao, Y., and Wang, H.-M. (2023). AV-Lip-Sync+: Leveraging AV-HuBERT to exploit multimodal inconsistency for video deepfake detection. arXiv.
5. Not all fake news is semantically similar: Contextual semantic representation learning for multimodal fake news detection10.1016/j.ipm.2023.103564 ↗
6. Audio–visual deepfake detection using articulatory features
7. Audio–visual synchronization and lip movement analysis for deepfake detection
8. Bohacek, M., and Farid, H. (2024, January 17–18). Lost in translation: Lip-sync deepfake detection from audio-video mismatch. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW63382.2024.00435 ↗
9. Liu, W., Zhang, Y., and Wang, X. (2024, January 9–15). Spotting the temporal inconsistency between audio and lip movements for deepfake detection. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.10.52202/079017-2892 ↗
10. Dual emotion based fake news detection: A deep attention mechanism approach
11. Fake news detection based on joint-training of dual-branch networks
12. Detecting fake news with capsule neural networks10.1016/j.asoc.2020.106991 ↗
13. Microblog rumor detection method based on multi-task learning
14. Temporally evolving graph neural network for fake news detection10.1016/j.ipm.2021.102712 ↗
15. WELFake: Word Embedding over linguistic features for fake news detection10.1109/TCSS.2021.3068519 ↗
16. Silva, A., Luo, L., Karunasekera, S., and Leckie, C. (2021, January 2–9). Embracing domain differences in fake news: Cross-domain fake news detection using multi-modal data. Proceedings of the 35th AAAI Conference on Artificial Intelligence, Online.10.1609/aaai.v35i1.16134 ↗
17. Dou, Y.T., Shu, K., Xia, C.Y., and Yu, P.S. (2021, January 11–15). User preference-aware fake news detection. Proceedings of the 44th ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3404835.3462990 ↗
18. Text sentiment distribution label augmentation method based on Plutchik’s wheel of emotions and sentiment lexicon
19. Towards better representation learning using hybrid deep learning model for fake news detection10.1007/s13278-022-00986-6 ↗
20. Alonso, M.A., Vilares, D., Gómez-Rodríguez, C., and Vilares, J. (2021). Sentiment Analysis for Fake News Detection. Electronics, 10.10.3390/electronics10111348 ↗
21. Qian, S.S., Wang, J.G., Hu, J., Fang, Q., and Xu, C. (2021, January 11–15). Hierarchical multi-modal contextual attention network for fake news detection. Proceedings of the 44th ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.10.1145/3404835.3462871 ↗
22. Yi, S.R., Soleymani, S., Arabnia, H.R., and Li, S. (2021, January 8–10). Socially aware multimodal deep neural networks for fake news classification. Proceedings of the IEEE 4th International Conference on Multimedia Information Processing and Retrieval, Tokyo, Japan.
23. Choi, H., and Ko, Y. (2021, January 1–5). Using topic modeling and adversarial neural networks for fake news video detection. Proceedings of the 30th ACM International Conference on Information and Knowledge Management, Online.10.1145/3459637.3482212 ↗
24. Wu, Y., Zhan, P.W., Zhang, Y.J., Wang, L., and Xu, Z. (2021). Multimodal fusion with co-attention networks for fake news detection. Findings of the ACL-IJCNLP, Association for Computational Linguistics.10.18653/v1/2021.findings-acl.226 ↗
25. Improving fake news detection by using an entity-enhanced framework to fuse diverse multimodal clues
26. Qi, P., Bu, Y., Cao, J., Ji, W., Shui, R., Xiao, J., Wang, D., and Chua, T.-S. (2023, January 7–14). FakeSV: A multimodal benchmark with rich social context for fake news detection on short video platforms. Proceedings of the 37th AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i12.26689 ↗
27. Zong, L., Zhou, J., Lin, W., Liu, X., Zhang, X., and Xu, B. (2024). Unveiling opinion evolution via prompting and diffusion for short video fake news detection. Findings of the Association for Computational Linguistics: ACL, Association for Computational Linguistics.10.18653/v1/2024.findings-acl.642 ↗
28. Li, J., Bin, Y., Zou, J., Zou, J., Wang, G., and Yang, Y. (2023). Cross-modal consistency learning with fine-grained fusion network for multimodal fake news detection. arXiv.10.1145/3595916.3626397 ↗
29. Wang, W.Y. (August, January 30). “Liar, Liar Pants on Fire”: A new benchmark dataset for fake news detection. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), Vancouver, BC, Canada.
30. Golbeck, J., Mauriello, M., Auxier, B., Gieringer, C., Graney, J., Hoffman, K.M., Huth, L., Ma, Z., Jha, M., and Khan, M. (2018, January 20–30). Fake News vs Satire: A Dataset and Analysis. Proceedings of the 10th ACM Conference on Web Science (WebSci), Amsterdam, The Netherlands.10.1145/3201064.3201100 ↗
31. Abu Salem, F.K., Al Feel, R., Elbassuoni, S., Jaber, M., and Farah, M. (2019, January 11–14). FA-KES: A Fake News Dataset around the Syrian War. Proceedings of the 13th International AAAI Conference on Web and Social Media (ICWSM), Munich, Germany.10.1609/icwsm.v13i01.3254 ↗
32. Zubiaga, A., Kochkina, E., Liakata, M., Procter, R., and Lukasik, M. (2016, January 11–16). Stance classification in rumours as a sequential task exploiting the tree structure of social media conversations. Proceedings of the 26th International Conference on Computational Linguistics (COLING), Osaka, Japan.
33. Nakamura, K., Levy, S., and Wang, W.Y. (2020, January 11–16). Fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. Proceedings of the 12th Language Resources and Evaluation Conference (LREC), Marseille, France.
34. Jindal, S., Sood, R., Singh, R., Vatsa, M., and Chakraborty, T. (2020, January 7–12). NewsBag: A benchmark multimodal dataset for fake news detection. Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, USA.
35. Zhu, Y., Liu, Y., and Zhang, X. (2025). MFND Dataset and Shallow-Deep Multitask Learning. arXiv.
36. Li, Y., Zhang, H., Zhang, J., and Wang, C. (2020). MM-COVID: A multilingual multimodal dataset for COVID-19 fake news detection. arXiv.
37. Tian, Y., Krishnan, D., and Isola, P. (2020, January 23–28). Contrastive multiview coding. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58621-8_45 ↗
38. van den Oord, A., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv.
39. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 12–18). A Simple Framework for Contrastive Learning of Visual Representations. Proceedings of the 37th International Conference on Machine Learning (ICML), Vienna, Austria.
40. Ho, J., Jain, A., and Abbeel, P. (2020, January 6–12). Denoising diffusion probabilistic models. Proceedings of the Advances in Neural Information Processing Systems, Virtual.
41. Shi, T., and Huang, S.-L. (2023, January 9–14). Multiemo: An attention-based correlation-aware multimodal fusion framework for emotion recognition in conversations. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), Toronto, ON, Canada.10.18653/v1/2023.acl-long.824 ↗
42. Qi, P., Zhao, Y., Shen, Y., Ji, W., Cao, J., and Chua, T.-S. (2023, January 9–14). Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023, Toronto, ON, Canada.10.18653/v1/2023.findings-acl.756 ↗