Text Data Augmentation for the Korean Language
Source abstract
Abstract
Data augmentation (DA) is a universal technique to reduce overfitting and improve the robustness of machine learning models by increasing the quantity and variety of the training dataset. Although data augmentation is essential in vision tasks, it is rarely applied to text datasets since it is less straightforward. Some studies have concerned text data augmentation, but most of them are for the majority languages, such as English or French. There have been only a few studies on data augmentation for minority languages, e.g., Korean. This study fills the gap by demonstrating several common data augmentation methods and Korean corpora with pre-trained language models. In short, we evaluate the performance of two text data augmentation approaches, known as text transformation and back translation. We compare these augmentations among Korean corpora on four downstream tasks: semantic textual similarity (STS), natural language inference (NLI), question duplication verification (QDV), and sentiment classification (STC). Compared to cases without augmentation, the performance gains when applying text data augmentation are 2.24%, 2.19%, 0.66%, and 0.08% on the STS, NLI, QDV, and STC tasks, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Computational and Text Analysis Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of ICT Convergence System Engineering, Chonnam National University, 77, Yongbong-ro, Buk-gu, Gwangju 500757, Korea
Department of Computer Engineering, Chonnam National University, 77, Yongbong-ro, Buk-gu, Gwangju 500757, Korea
References (25)
Show source-provided references
1. A survey on image data augmentation for deep learning10.1186/s40537-019-0197-0 ↗
2. Unsupervised Data Augmentation for Consistency Training
3. Google’s multilingual neural machine translation system: Enabling zero-shot translation10.1162/tacl_a_00065 ↗
4. Wei, J., and Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv.10.18653/v1/D19-1670 ↗
5. Kobayashi, S. (2018). Contextual augmentation: Data augmentation by words with paradigmatic relations. arXiv.10.18653/v1/N18-2072 ↗
6. Wu, X., Lv, S., Zang, L., Han, J., and Hu, S. (2019, January 12–14). Conditional Bert contextual augmentation. Proceedings of the International Conference on Computational Science, Faro, Portugal.10.1007/978-3-030-22747-0_7 ↗
7. Kumar, V., Choudhary, A., and Cho, E. (2020). Data augmentation using pre-trained transformer models. arXiv.
8. Yang, Y., Malaviya, C., Fernandez, J., Swayamdipta, S., le Bras, R., Wang, J.-P., Bhagavatula, C., Choi, Y., and Downey, D. (2020). Generative Data Augmentation for Commonsense Reasoning. Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics.10.18653/v1/2020.findings-emnlp.90 ↗
9. Schick, T., and Schütze, H. (2021). Generating Datasets with Pretrained Language Models. arXiv.10.18653/v1/2021.emnlp-main.555 ↗
10. Thakur, N., Reimers, N., Daxenberger, J., and Gurevych, I. (2020). Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks. arXiv.10.18653/v1/2021.naacl-main.28 ↗
11. Schick, T., and Hinrich, S. (2011, January 7–11). Few-Shot Text Generation with Natural Language Instructions. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic.10.18653/v1/2021.emnlp-main.32 ↗
12. Edunov, S., Ott, M., Auli, M., and Grangier, D. (November, January 31). Understanding Back-Translation at Scale. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1045 ↗
13. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
14. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2019). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. arXiv.10.18653/v1/2020.acl-main.703 ↗
15. Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv.10.18653/v1/D19-1410 ↗
16. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
17. Reimers, N., and Gurevych, I. (2020). Making monolingual sentence embeddings multilingual using knowledge distillation. arXiv.10.18653/v1/2020.emnlp-main.365 ↗
18. Cho, W.I., Moon, S., and Song, Y. Open Korean Corpora: A Practical Report. Proceedings of the Second Workshop for NLP Open Source Software (NLP-OSS).
19. Lee, S., Jang, H., Baik, Y., Park, S., and Shin, H. (2020). Kr-bert: A small-scale korean-specific language model. arXiv.10.5626/JOK.2020.47.7.682 ↗
20. Ham, J., Choe, Y.J., Park, K., Choi, I., and Soh, H. (2020). KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding. Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics.10.18653/v1/2020.findings-emnlp.39 ↗
21. Lee, H., Yoon, J., Hwang, B., Joe, S., Min, S., and Gwon, Y. (2021, January 10–15). Korealbert: Pretraining a lite bert model for korean language understanding. Proceedings of the 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.10.1109/ICPR48806.2021.9412023 ↗
22. Tang, Y., Tran, C., Li, X., Chen, P.-J., Goya, N., Chaudhary, V., Gu, J., and Fan, A. (2020). Multilingual translation with extensible multilingual pretraining and fine-tuning. arXiv.
23. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
24. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv.
25. Clark, K., Luong, M.-T., Le, Q.V., and Manning, C.D. (2020). Electra: Pre-training text encoders as discriminators rather than generators. arXiv.