Multimodal Neural Machine Translation Based on Knowledge Distillation and Anti-Noise Interaction
Source abstract
Abstract
Within the realm of multimodal neural machine translation (MNMT), addressing the challenge of seamlessly integrating textual data with corresponding image data to enhance translation accuracy has become a pressing issue. We s... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (46)
Show source-provided references
1. A survey on multimodal bidirectional machine learning translation of image and natural language processing10.1016/j.eswa.2023.121168 ↗
2. Virtual visual-guided domain-shadow fusion via modal exchanging for domain-specific multi-modal neural machine translation
3. Multimodality information fusion for automated machine translation10.1016/j.inffus.2022.10.018 ↗
4. Yin Y, Meng F, Su J, Zhou C, Yang Z, Zhou J, et al. A novel graph-based multi-modal fusion encoder for neural machine translation [M.S. dissertation]. China: Xiamen University; 2020.10.18653/v1/2020.acl-main.273 ↗
5. Noise-robust cross-modal interactive learning with text2image mask for multi-modal neural machine translation
6. Supervised visual attention for multimodal neural machine translation
7. Li J, Ataman D, Sennrich R. Vision matters when it should: sanity checking multimodal machine translation models [M.S. dissertation]. Switzerland: ETH Zürich; 2021.10.18653/v1/2021.emnlp-main.673 ↗
8. Visual pivoting unsupervised multimodal machine translation in low-resource distant language Pairs
9. Modeling homophone noise for robust neural machine translation
10. Multimodal transformer for multimodal machine translation
11. Wang F, Yan J, Meng F, Zhou J. Selective knowledge distillation for neural machine translation [M.S. dissertation]. China: Peking University; 2021.
12. Membership inference attacks on sequence-to-sequence models: is my data in your machine translation system?10.1162/tacl_a_00299 ↗
13. Gain B, Bandyopadhyay D, Ekbal A. Experiences of adapting multimodal machine translation techniques for hindi [M.S. dissertation]. India: Indian Institute of Technology Patna; 2021.
14. Attention-based multimodal neural machine translation
15. Calixto I, Liu Q, Campbell N. Incorporating global visual features into attention-based neural machine translation [M.S. dissertation]. Irish: ADAPT Centre Dublin City University; 2017.10.18653/v1/D17-1105 ↗
16. Elliott D, Frank S, Barrault L, Bougares F, Specia L. Findings of the second shared task on multimodal machine translation and multilingual image description [M.S. dissertation]. England: School of Informatics University of Edinburgh; 2017.10.18653/v1/W17-4718 ↗
17. Word-region alignment-guided multimodal neural machine translation10.1109/TASLP.2021.3138719 ↗
18. Enhancing neural machine translation with dual-side multimodal awareness10.1109/TMM.2021.3092187 ↗
19. Dynamic context-guided capsule network for multimodal machine translation
20. Model compression
21. Hinton G. Distilling the knowledge in a neural network [Ph.D. dissertation]. USA: Google Inc.; 2015.
22. Romero A, Ballas N, Kahou SE, Chassang A, Gatta C, Bengio Y. FitNets: hints for thin deep nets [M.S. dissertation]. Spanish: Universitat de Barcelona; 2014.
23. A gift from knowledge distillation: fast optimization, network minimization and transfer learning
24. Cross modal distillation for supervision transfer
25. Text-to-image synthesis via symmetrical distillation networks
26. Mass: masked sequence to sequence pre-training for language generation
27. Li P, Li L, Zhang M, Wu M, Liu Q. Universal conditional masked language pre-training for neural machine translation [Ph.D. dissertation]. China: Huawei Noah’s Ark Lab; 2022.10.18653/v1/2022.acl-long.442 ↗
28. Huang H, Liang Y, Duan N, Gong M, Shou L, Jiang D, et al. Unicoder: a universal language encoder by pre-training with multiple cross-lingual tasks [Ph.D. dissertation]. China: Microsoft Research Asia; 2019.10.18653/v1/D19-1252 ↗
29. Exploring the limits of transfer learning with a unified text-to-text transformer
30. UniLMv2: pseudo-masked language models for unified language model pre-training
31. Enhanced copy-move forgery detection using deep convolutional neural network (DCNN) employing the ResNet-101 transfer learning model10.1007/s11042-023-15724-z ↗
32. MST: masked self-supervised transformer for visual representation
33. Elliott D, Frank S, Sima’an K, Specia L. Multi30k: multilingual english-german image descriptions [Ph.D. dissertation]. Holland: University of Amsterdam; 2016.10.18653/v1/W16-3210 ↗
34. From image descriptions to visual denotations: new similarity metrics for semantic inference over event descriptions10.1162/tacl_a_00166 ↗
35. Mathur N, Baldwin T, Cohn T. Tangled up in BLEU: reevaluating the evaluation of automatic machine translation evaluation metrics [M.S. dissertation]. Australia: The University of Melbourne; 2020.10.18653/v1/2020.acl-main.448 ↗
36. Automatic evaluating of Russian-Arabic machine translation quality using METEOR method10.1063/5.0067018 ↗
37. Calixto I, Rios M, Aziz W. Latent variable model for multi-modal translation [M.S. dissertation]. Holland: The University of Amsterdam; 2018. LIUM-CVC Submissions for WMT17 Multimodal Translation Task [M.S. dissertation]. French: University of Le Mans; 2017.
38. Zhou M, Cheng R, Lee YJ, Yu Z. A visual attention grounding neural model for multimodal machine translation [M.S. dissertation]. Canada: University of California; 2018.10.18653/v1/D18-1400 ↗
39. Caglayan O, Madhyastha P, Specia L, Barrault L. Probing the need for visual context in multimodal machine translation [M.S. dissertation]. French: Le Mans University; 2019.10.18653/v1/N19-1422 ↗
40. Efficient object-level visual context modeling for multimodal machine translation: masking irrelevant objects helps grounding
41. Vaswani A. Attention is all you need [Ph.D. dissertation]. USA: Google Research; 2017.
42. Wu Z, Kong L, Bi W, Li X, Kao B. Good for misconceived reasons: an empirical revisiting on the need for visual context in multimodal machine translation [M.S. dissertation]. China: The University of Hong Kong; 2021.10.18653/v1/2021.acl-long.480 ↗
43. Gong H, Jia M, Jing L. Multimodal interaction modeling via self-supervised multi-task learning for review helpfulness prediction [M.S. dissertation]. China: Sun Yat-sen University; 2024.
44. The progressive alignment-aware multimodal fusion with easy2hard strategy for multimodal neural machine translation
45. Ji B, Zhang T, Zou Y, Hu B, Shen S. Increasing visual awareness in multimodal neural machine translation from an information theoretic perspective [M.S. dissertation]. China: Tencent Minority-Mandarin Translation; 2022.10.18653/v1/2022.emnlp-main.453 ↗
46. Dual-level interactive multimodal-mixup encoder for multi-modal neural machine translation10.1007/s10489-022-03331-8 ↗