journal article·2025-11-06Open access

DATTAMM: Domain-Aware Test-Time Adaptation for Multimodal Misinformation Detection

Kaicheng Xu, Shasha Wang, Zipeng Diao

Applied Sciences· Volume 15· Issue 21

Abstract

The rapid proliferation of multimodal misinformation across diverse news categories poses unprecedented challenges to digital ecosystems, where existing detection systems exhibit critical limitations in domain adaptation and fairness. Current methods suffer from two fundamental flaws: (1) severe performance variance (>35% accuracy drop in education/science categories) due to category-specific semantic shifts; (2) systemic real/fake detection bias causing up to 68.3% false positives in legitimate content—risking suppression of factual reporting especially in high-stakes domains like public health discourse. To address these dual challenges, this paper proposes the DATTAMM (Domain-Adaptive Tensorized Multimodal Model), a novel framework integrating category-aware attention mechanisms and adversarial debiasing modules. Our approach dynamically aligns textual–visual features while suppressing domain-irrelevant noise through the following: (a) semantic disentanglement layers extracting category-invariant patterns; (b) cross-modal verification units resolving inter-modal conflicts; (c) real/fake gradient alignment regularizers. Extensive experiments on nine news categories demonstrate that the DATTAMM achieves an average F1-score of 0.854, outperforming state-of-the-art baselines by 32.7%. The model maintains consistent performance with less than 5.4% variance across categories, significantly reducing accuracy drops in education and science content where baselines degrade by over 35%. Crucially, the DATTAMM narrows the real/fake F1 gap to merely 0.017, compared to 0.243–0.547 in baseline models, while cutting false positives in high-stakes domains like health news to 5.8% versus the 38.2% baseline average. These advances lower societal costs of misclassification by 79.7%, establishing a new paradigm for robust and equitable misinformation detection in evolving information ecosystems.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Misinformation and Its Impacts · Topic Modeling · Hate Speech and Cyberbullying Detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

China National Institute of Standardization, Beijing 100191, China

SAMR Defective Product Recall Technical Center, Beijing 100101, China

Funding

Special Fundamental Research Fund for the Central Public Scientific Research Institutes · National Key Research and Development Program

References (55)

Show source-provided references
1. Consistency regularization-based mutual alignment for source-free domain adaptation10.1016/j.eswa.2023.122577 ↗
2. Yuan, L., Xie, B., and Li, S. (2023, January 17–24). Robust Test-Time Adaptation in Dynamic Scenarios. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01528 ↗
3. Robust gradient aware and reliable entropy minimization for stable test-time adaptation in dynamic scenarios10.1007/s00371-024-03327-0 ↗
4. Guo, Z., Jin, T., Xu, W., Lin, W., and Wu, Y. (2024). Bridging the Gap for Test-Time Multimodal Sentiment Analysis. arXiv.
5. Wang, J., Zhang, H., Liu, C., and Yang, X. (2024, January 14–18). Fake News Detection via Multi-scale Semantic Alignment and Cross-modal Attention. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington, DC, USA.10.1145/3626772.3657905 ↗
6. Modality-dropout contrastive learning for source-free domain adaptation
7. Chen, Y., Zhu, F., and Shen, L. (2023, January 18–22). Temporal stability guided pseudo-label filtering in test-time adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.
8. Lee, Y., Lee, S., Park, C., Cha, J., and Park, E. (2025). TAGF: Time-aware Gated Fusion for Multimodal Valence-Arousal Estimation. arXiv.
9. Bias-mitigated test-time adaptation via adversarial gradient alignment
10. Mao, W., Wu, J., Liu, H., Sui, Y., and Wang, X. (2024). Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization. arXiv.10.1007/s11704-025-40798-3 ↗
11. Lightweight test-time adaptation for edge deployment
12. Tao, C., Shen, L., and Mondal, S. (2024). Meta-TTT: A Meta-learning Minimax Framework For Test-Time Training. arXiv.
13. Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation10.1109/TPAMI.2024.3476487 ↗
14. Meta-learning for source-free cross-domain visual question answering
15. Dynamic fusion networks for multimodal robustness under corruption
16. Jang, M., and Chung, H.W. (2024). Label Distribution Shift-Aware Prediction Refinement for Test-Time Adaptation. arXiv.
17. Li, Z., Li, J., and Yang, Y. (2024, January 17–18). Gradient alignment for bias reduction in multimodal fake news detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
18. Contrastive test-time adaptation with cross-modal consistency
19. Huang, M., Gong, C., and Grauman, K. (2024, January 23–27). Self-supervised domain generalization for multimodal perception. Proceedings of the European Conference on Computer Vision (ECCV), Tel-Aviv, Israel.
20. Lei, J., He, K., Sun, X., Yuan, Z., and Zhang, J. (2025, January 14–17). Adaptive Multimodal Fusion via Attention-Guided Feature Selection for Histopathology Image Classification. Proceedings of the 2025 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.10.1109/ICIP55913.2025.11084638 ↗
21. Unsupervised domain adaptation for semantic segmentation via cross-region alignment10.1016/j.cviu.2023.103743 ↗
22. Chen, L., Wang, H., and He, X. (2023, January 23–29). Energy-based uncertainty estimation in test-time adaptation. Proceedings of the IEEE International Conference on Machine Learning (ICML), Honolulu, HI, USA.
23. Hierarchical modality dropout for robust multimodal learning
24. Lee, T., Tremblay, J., Blukis, V., Wen, B., Lee, B.U., Shin, I., Birchfield, S., Kweon, I.S., and Yoon, K.J. (2023). TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation. arXiv.10.1109/CVPR52729.2023.02039 ↗
25. Continual Test-Time Adaptation with Weighted Contrastive Learning and Pseudo-Label Correction10.1109/TETC.2025.3528985 ↗
26. Multi-modal recursive prompt learning with mixup embedding for generalization recognition10.1016/j.knosys.2024.111726 ↗
27. Privacy-preserving test-time adaptation via split learning
28. Yang, T., Zhou, S., Wang, Y., Lu, Y., and Zheng, N. (2022). Test-time batch normalization. arXiv.
29. Adversarial data augmentation for robust multimodal fusion
30. Wang, R., Xu, M., and Mei, T. (November, January 29). Dynamic importance weighting for multimodal test-time adaptation. Proceedings of the ACM Multimedia (MM), Ottawa, ON, Canada.
31. Decoupled prototype learning for reliable test-time adaptation10.1109/TMM.2025.3535400 ↗
32. Self-distillation for test-time robustness in multimodal networks
33. Roy, A., Sarkar, S., and Mitra, P. (2024, January 16–22). On evaluating multimodal misinformation detection: Dataset drift and pitfalls. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.
34. SARD: Fake news detection based on CLIP contrastive learning and multimodal semantic alignment10.1016/j.jksuci.2024.102160 ↗
35. Efficient transformer adaptation for resource-constrained devices
36. Liu, Z., Zhang, Y., and Chen, K. (2025, January 20–23). Cross-lingual robustness in multimodal fake news detection. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Vancouver, BC, Canada.
37. Stile, V., Caldelli, R., Guerrero-Contreras, G., Balderas-Díaz, S., and Medina-Bulo, I. (July, January 30). Analysis of DeepFake Detection through Semi-Supervised Facial Attribute Labeling. Proceedings of the 11th Spanish-German Symposium on Applied Computer Science (SGSOACS), Vienna, Austria.
38. A many-objective ensemble optimization algorithm for the edge cloud resource scheduling problem
39. Balderas-Díaz, S., Guerrero-Contreras, G., Muñoz, A., Durães, D., and Novais, P. (2025, January 23–26). Explainable artificial intelligence for audio-based detection of emergency vehicles. Proceedings of the 21st International Conference on Intelligent Environments (IE), Darmstadt, Germany.10.1109/IE64880.2025.11130127 ↗
40. Hybrid edge-cloud collaborator resource scheduling approach based on deep reinforcement learning and multi-objective optimization10.1109/TC.2023.3326977 ↗
41. A hybrid many-objective optimization algorithm for task offloading and resource allocation in multi-server mobile edge computing networks10.1109/TSC.2023.3268990 ↗
42. Audio-visual synchronization and lip movement analysis for real-time deepfake detection10.1007/s44196-025-00911-7 ↗
43. Many-objective optimization based intrusion detection for in-vehicle network security10.1109/TITS.2023.3296002 ↗
44. Hu, J., Yang, M., Tang, B., and Hu, J. (2025). Integrating message content and propagation path for enhanced false information detection using bidirectional graph convolutional neural networks. Appl. Sci., 15.10.3390/app15073457 ↗
45. An intelligent edge dual-structure ensemble method for data stream detection and releasing10.1109/JIOT.2023.3286185 ↗
46. A two-stage federated optimization algorithm for privacy computing in Internet of Things10.1016/j.future.2023.03.042 ↗
47. Zilean: A modularized framework for large-scale temporal concept drift type classification10.1016/j.ins.2025.122134 ↗
48. Multimodal fusion and knowledge distillation for improved anomaly detection10.1007/s00371-024-03723-6 ↗
49. Zhang, M., Li, J., and Huang, X. (2024, January 21–27). Benchmarking robustness of multimodal models under realistic corruptions. Proceedings of the IEEE International Conference on Machine Learning (ICML), Vienna, Austria.
50. An, W., Zhong, W., Jiang, F., Ma, H., and Huang, J. (October, January 29). Causal Subgraphs and Information Bottlenecks: Redefining OOD Robustness in Graph Neural Networks. Proceedings of the Computer Vision—{ECCV} 2024—18th European Conference, Milan, Italy.10.1007/978-3-031-73223-2_26 ↗
51. Zhang, L., Deng, Y., and Li, J. (October, January 29). Adversarial augmentation for robust test-time adaptation. Proceedings of the European Conference on Computer Vision (ECCV), Milan, Italy.
52. Lu, W., Tong, Y., and Ye, Z. (March, January 25). DAMMFND: Domain-Aware Multimodal Multi-view Fake News Detection. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA.
53. Nan, Q., Cao, J., Zhu, Y., Wang, Y., and Li, J. (2021, January 1–5). MDFEND: Multi-domain Fake News Detection. Proceedings of the ACM International Conference on Information & Knowledge Management (CIKM), Queensland, Australia.10.1145/3459637.3482139 ↗
54. Knowledge Augmented Transformer for Adversarial Multidomain Multiclassification Multimodal Fake News Detection10.1016/j.neucom.2021.07.077 ↗
55. Tong, Y., Lu, W., Zhao, Z., Lai, S., and Shi, T. (November, January 28). MMDFND: Multi-modal Multi-Domain Fake News Detection. Proceedings of the ACM International Conference on Multimedia (MM), Melbourne, VIC, Australia.