Neural Machine Translation Research on Syntactic Information Fusion Based on the Field of Electrical Engineering
Source abstract
Abstract
Neural machine translation has achieved good translation results, but needs further improvement in low-resource and domain-specific translation. To this end, the paper proposed to incorporate source language syntactic information into neural machine translation models. Two novel approaches, namely Contrastive Language–Image Pre-training(CLIP) and Cross-attention Fusion (CAF), were compared to a base transformer model on EN–ZH and ZH–EN pair machine translation focusing on the electrical engineering domain. In addition, an ablation study on the effect of both proposed methods was presented. Among them, the CLIP pre-training method improved significantly compared with the baseline system, and the BLEU values in the EN–ZH and ZH–EN tasks increased by 3.37 and 3.18 percentage points, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Machine Learning in Bioinformatics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China
Henan Province New Energy Vehicle Power Electronics and Power Transmission Engineering Research Center, Luoyang 471023, China
School of Foreign Languages, Henan University of Science and Technology, Luoyang 471023, China
Funding
National Natural Science Foundation of China
References (27)
Show source-provided references
1. Tiejun, Z. (2001). Principles of Machine Translation, Harbin Institute of Technology Press.
2. Zhiwei, F. (2010, January 6–8). Machine Translation: From Rule-Based Techniques to Statistics-Based Techniques. Proceedings of the 2010 China Translation Professional Exchange Conference Proceedings, Macau, China.
3. A Survey of Neural Machine Translation
4. Chen, H., Huang, S., Chiang, D., and Chen, J. (2017). Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder. arXiv.10.18653/v1/P17-1177 ↗
5. Aharoni, R., and Goldberg, Y. (2017). Towards String-to-Tree Neural Machine Translation. arXiv.10.18653/v1/P17-2021 ↗
6. Zheng, X., Chen, H., Ma, Y., and Wang, Q. (2022). ITM Web of Conferences, EDP Sciences.
7. Duan, S., Zhao, H., Zhou, J., and Wang, R. (2019, January 15–17). Syntax-aware Transformer Encoder for Neural Machine Translation. Proceedings of the International Conference on Asian Language Processing (IALP), Shanghai, China.10.1109/IALP48816.2019.9037672 ↗
8. Ma, C., Tamura, A., Utiyama, M., Sumita, E., and Zhao, T. (2019, January 2–7). Improving Neural Machine Translation with Neural Syntactic Distance. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.10.18653/v1/N19-1205 ↗
9. Bugliarello, E., and Okazaki, N. (2019). Enhancing Machine Translation with Dependency-Aware Self-Attention. arXiv.10.18653/v1/2020.acl-main.147 ↗
10. Neural Machine Translation Method Based on Source Language Syntax Enhanced Decoding
11. Gao, S., Zhang, Y., Yu, Z., and Zhu, H. (2021). Chinese-Vietnamese Parallel Sentence Extraction Method with Syntactic Structure and Tree-LSTM Integration:CN202010978713.0[P], ResearchGate.
12. Chinese-Vietnamese Convolutional Neural Machine Translation with Syntactic Parsing Trees Integration
13. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning Transferable Visual Models From Natural Language Supervision. Proceedings of the International Conference on Machine Learning, Virtual Event.
14. Graph neural networks: A review of methods and applications10.1016/j.aiopen.2021.01.001 ↗
15. Du, Q. (2021). A Multimodal Machine Translation Data Augmentation Method Based on Image Description: CN202011212067.3[P], ResearchGate.
16. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17), Long Beach, CA, USA.
17. Transformers in Vision: A Survey10.1145/3505244 ↗
18. A graph-transformer for whole slide image classification10.1109/TMI.2022.3176598 ↗
19. A Review of the Application of Visual Transformers in Medical Image Analysis
20. Xie, F., Zhang, D., and Liu, C. (2022). Global–Local Self-Attention Based Transformer for Speaker Verification. Appl. Sci., 12.10.3390/app121910154 ↗
21. Chinese Speech Recognition Based on DFCNN-CTC and Transformer
22. Language Models are Few-Shot Learners
23. Training language models to follow instructions with human feedback
24. A Study on Chinese-English Machine Translation Based on Migration Learning and Neural Networks10.1142/S0218213022500312 ↗
25. Short Sequence Chinese-English Machine Translation Based on Generative Adversarial Networks of Emotion
26. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6–12). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th annual meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
27. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2019). HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv.10.18653/v1/2020.emnlp-demos.6 ↗