Domain Adaptation for Arabic Machine Translation: Financial Texts as a Case Study
Source abstract
Abstract
Neural machine translation (NMT) has shown impressive performance when trained on large-scale corpora. However, generic NMT systems have demonstrated poor performance on out-of-domain translation. To mitigate this issue, several domain adaptation methods have recently been proposed which often lead to better translation quality than genetic NMT systems. While there has been some continuous progress in NMT for English and other European languages, domain adaption in Arabic has received little attention in the literature. The current study, therefore, aims to explore the effectiveness of domain-specific adaptation for Arabic MT (AMT), in yet unexplored domain, financial news articles. To this end, we developed a parallel corpus for Arabic-English (AR-EN) translation in the financial domain to benchmark different domain adaptation methods. We then fine-tuned several pre-trained NMT and Large Language models including ChatGPT-3.5 Turbo on our dataset. The results showed that fine-tuning pre-trained NMT models on a few well-aligned in-domain AR-EN segments led to noticeable improvement. The quality of ChatGPT translation was superior to other models based on automatic and human evaluations. To the best of our knowledge, this is the first work on fine-tuning ChatGPT towards financial domain transfer learning. To contribute to research in domain translation, we made our datasets and fine-tuned models available.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Translation Studies and Practices
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Center of Excellence in AI and Data Sciences, King Abdulaziz University, Jeddah 21589, Saudi Arabia
ASAS AI Lab, Riyadh 13518, Saudi Arabia
Funding
General Authority for Literature, Publishing and Translation
References (68)
Show source-provided references
1. Koehn, P., and Knowles, R. (2017). Six challenges for neural machine translation. arXiv.10.18653/v1/W17-3204 ↗
2. Daumé Iii, H., and Jagarlamudi, J. (2011, January 19–24). Domain adaptation for machine translation by mining unseen words. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA.
3. Domain adaptation and multi-domain adaptation for neural machine translation: A survey10.1613/jair.1.13566 ↗
4. A survey of domain adaptation for machine translation
5. Moslem, Y., Haque, R., and Way, A. (2023). Adaptive machine translation with large language models. arXiv.
6. Transforming machine translation: A deep learning system reaches news translation quality comparable to human professionals10.1038/s41467-020-18073-9 ↗
7. Moslem, Y., Haque, R., Kelleher, J.D., and Way, A. (2022). Domain-Specific Text Generation for Machine Translation. arXiv.
8. Hatem, A., and Omar, N. (2010). Syntactic reordering for Arabic-English phrase-based machine translation. Database Theory and Application, Bio-Science and Bio-Technology, Springer.10.1007/978-3-642-17622-7_20 ↗
9. Assessment of Google and Microsoft Bing translation of journalistic texts
10. Sennrich, R., Haddow, B., and Birch, A. (2015). Neural machine translation of rare words with subword units. arXiv.10.18653/v1/P16-1162 ↗
11. Costa-Jussa, M.R., and Fonollosa, J.A. (2016). Character-based neural machine translation. arXiv.10.18653/v1/W17-4123 ↗
12. Luong, M.T., and Manning, C.D. (2016). Achieving open vocabulary neural machine translation with hybrid word-character models. arXiv.10.18653/v1/P16-1100 ↗
13. Müller, M., Rios, A., and Sennrich, R. (2019). Domain robustness in neural machine translation. arXiv.
14. Oudah, M., Almahairi, A., and Habash, N. (2019). The impact of preprocessing on Arabic-English statistical and neural machine translation. arXiv.
15. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6–12). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
16. Bapna, A., Arivazhagan, N., and Firat, O. (2019). Simple, scalable adaptation for neural machine translation. arXiv.10.18653/v1/N19-1191 ↗
17. Yang, S., Wang, Y., and Chu, X. (2020). A survey of deep learning techniques for neural machine translation. arXiv.
18. Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8–13). Sequence to sequence learning with neural networks. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
19. Google’s multilingual neural machine translation system: Enabling zero-shot translation10.1162/tacl_a_00065 ↗
20. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
21. Deep recurrent models with fast-forward connections for neural machine translation10.1162/tacl_a_00105 ↗
22. Ataman, D., Aziz, W., and Birch, A. (2019). A latent morphology model for open-vocabulary neural machine translation. arXiv.
23. Meng, F., Lu, Z., Wang, M., Li, H., Jiang, W., and Liu, Q. (2015). Encoding source language with convolutional neural network for machine translation. arXiv.10.3115/v1/P15-1003 ↗
24. Gehring, J., Auli, M., Grangier, D., and Dauphin, Y.N. (2016). A convolutional encoder model for neural machine translation. arXiv.10.18653/v1/P17-1012 ↗
25. Kalchbrenner, N., Espeholt, L., Simonyan, K., Oord, A.v.d., Graves, A., and Kavukcuoglu, K. (2016). Neural machine translation in linear time. arXiv.
26. Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017, January 6–11). Convolutional sequence to sequence learning. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia.
27. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
28. Luong, M.T., Pham, H., and Manning, C.D. (2015). Effective approaches to attention-based neural machine translation. arXiv.10.18653/v1/D15-1166 ↗
29. Neural machine translation: A review10.1613/jair.1.12007 ↗
30. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
31. Antoun, W., Baly, F., and Hajj, H. (2020). Arabert: Transformer-based model for Arabic language understanding. arXiv.
32. Lan, W., Chen, Y., Xu, W., and Ritter, A. (2020, January 16–20). Gigabert: Zero-shot transfer learning from english to arabic. Proceedings of the 2020 Conference on Empirical Methods on Natural Language Processing (EMNLP), Online.
33. Abdul-Rauf, S., Kiani, K., Zafar, A., and Nawaz, R. (2019, January 1–2). Exploring transfer learning and domain data selection for the biomedical translation. Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2), Florence, Italy.
34. Liu, B., and Huang, L. (2021). ParaMed: A parallel corpus for English–Chinese translation in the biomedical domain. BMC Med. Inform. Decis. Mak., 21.10.1186/s12911-021-01621-8 ↗
35. Martínez-Domínguez, R., Rikters, M., Vasiļevskis, A., Pinnis, M., and Reichenberg, P. (2020, January 6–9). Customized Neural Machine Translation Systems for the Swiss Legal Domain. Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 2: User Track), Online.
36. Läubli, S., Amrhein, C., Düggelin, P., Gonzalez, B., Zwahlen, A., and Volk, M. (2019). Post-editing productivity with neural machine translation: An empirical assessment of speed and quality in the banking and finance domain. arXiv.
37. Grangier, D., and Iter, D. (2021). The trade-offs of domain adaptation for neural language models. arXiv.10.18653/v1/2022.acl-long.264 ↗
38. Poncelas, A., Shterionov, D., Way, A., Wenniger, G., and Passban, P. (2018). Investigating Backtranslation in Neural Machine Translation. arXiv.
39. Chu, C., Dabre, R., and Kurohashi, S. (2017, January 9–14). An empirical comparison of domain adaptation methods for neural machine translation. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Toronto, ON, Canada.10.18653/v1/P17-2061 ↗
40. Deng, Y., Yu, H., Yu, H., Duan, X., and Luo, W. (2020, January 16–20). Factorized transformer for multi-domain neural machine translation. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event.10.18653/v1/2020.findings-emnlp.377 ↗
41. Bakr, H.A., Shaalan, K., and Ziedan, I. (2008, January 27–28). A hybrid approach for converting written Egyptian colloquial dialect into diacritized Arabic. Proceedings of the 6th International Conference on Informatics and Systems, infos2008, Cairo University, Citeseer, Cairo, Egypt.
42. Mohamed, E., Mohit, B., and Oflazer, K. (2012, January 8–14). Transforming standard Arabic to colloquial Arabic. Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Jeju Island, Republic Korea.
43. Habash, N., and Hu, J. (2009, January 30–31). Improving Arabic-Chinese statistical machine translation using English as pivot language. Proceedings of the Fourth Workshop on Statistical Machine Translation, Athens, Greece.10.3115/1626431.1626467 ↗
44. Nagoudi, E.M.B., Elmadany, A., and Abdul-Mageed, M. (2021). AraT5: Text-to-text transformers for Arabic language generation. arXiv.10.18653/v1/2022.acl-long.47 ↗
45. Arabic Machine Translation: A Survey With Challenges and Future Directions10.1109/ACCESS.2021.3132488 ↗
46. Arabic machine translation: A survey of the latest trends and challenges10.1016/j.cosrev.2020.100305 ↗
47. Introduction to Arabic natural language processing
48. English-Arabic hybrid machine translation system using EBMT and translation memory
49. Scott, D., Bel, N., and Zong, C. (2020, January 8–13). AraBench: Benchmarking Dialectal Arabic-English Machine Translation. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.
50. Khondaker, M.T.I., Waheed, A., Nagoudi, E.M.B., and Abdul-Mageed, M. (2023). GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP. arXiv.10.18653/v1/2023.emnlp-main.16 ↗
51. Antoun, W., Baly, F., and Hajj, H. (2021, January 19). AraGPT2: Pre-Trained Transformer for Arabic Language Generation. Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine.
52. Language Models are Unsupervised Multitask Learners
53. Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (2021, January 7). mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.
54. Tiedemann, J., and Thottingal, S. (2020, January 3–5). OPUS-MT—Building open translation services for the World. Proceedings of the European Association for Machine Translation Conferences/Workshops, Lisbon, Portugal.
55. Tiedemann, J. (2020, January 3–5). The Tatoeba Translation Challenge—Realistic Data Sets for Low Resource and Multilingual MT. Proceedings of the Fifth Conference on Machine Translation, Lisbon, Portugal.
56. Kudo, T., and Richardson, J. (November, January 31). SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium.10.18653/v1/D18-2012 ↗
57. Team, N., Costa-jussà, M.R., Cross, J., Çelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., and Licht, D. (2022). No Language Left Behind: Scaling Human-Centered Machine Translation. arXiv.
58. Popović, M. (2015, January 17–18). chrF: Character n-gram F-score for automatic MT evaluation. Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal.10.18653/v1/W15-3049 ↗
59. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006, January 5–8). A study of translation edit rate with targeted human annotation. Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, Cambridge, MA, USA.
60. Rei, R., Stewart, C., Farinha, A.C., and Lavie, A. (2020, January 16–20). COMET: A Neural Framework for MT Evaluation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.213 ↗
61. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2020). BERTScore: Evaluating Text Generation with BERT. arXiv.
62. Peng, K., Ding, L., Zhong, Q., Shen, L., Liu, X., Zhang, M., Ouyang, Y., and Tao, D. (2023). Towards Making the Most of ChatGPT for Machine Translation. arXiv.10.2139/ssrn.4390455 ↗
63. Khoshafah, F. (2024, July 12). ChatGPT for Arabic-English Translation: Evaluating the Accuracy. Available online: https://www.researchsquare.com/article/rs-2814154/v2.10.21203/rs.3.rs-2814154/v1 ↗
64. Alyafeai, Z., Alshaibani, M.S., AlKhamissi, B., Luqman, H., Alareqi, E., and Fadel, A. (2023). Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models. arXiv.
65. Banimelhem, O., and Amayreh, W. (2023, January 21–23). Is ChatGPT a Good English to Arabic Machine Translation Tool?. Proceedings of the 2023 14th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan.10.1109/ICICS60529.2023.10330525 ↗
66. Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y.J., Afify, M., and Awadalla, H.H. (2023). How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. arXiv.
67. Coughlin, D. (2003, January 23–27). Correlating automated and human assessments of machine translation quality. Proceedings of the Machine Translation Summit IX: Papers, New Orleans, LA, USA.
68. Barrault, L., Bojar, O., Bougares, F., Chatterjee, R., Costa-jussà, M.R., Federmann, C., Fishel, M., Fraser, A., Graham, Y., and Guzman, P. (June, January 30). Unbabel’s Participation in the WMT20 Metrics Shared Task. Proceedings of the Fifth Conference on Machine Translation, Online.