TAWC: Text Augmentation with Word Contributions for Imbalance Aspect-Based Sentiment Classification
Source abstract
Abstract
Text augmentation plays an important role in enhancing the generalizability of language models. However, traditional methods often overlook the unique roles that individual words play in conveying meaning in text and imbalance class distribution, thereby risking suboptimal performance and compromising the model’s generalizability. This limitation motivated us to develop a novel technique called Text Augmentation with Word Contributions (TAWC). Our approach tackles this problem in two core steps: Firstly, it employs analytical correlation and semantic similarity metrics to discern the relationships between words and their associated aspect polarities. Secondly, it tailors distinct augmentation strategies to individual words based on their identified functional contributions in the text. Extensive experiments on two aspect-based sentiment analysis datasets demonstrate that the proposed TAWC model significantly improves the classification performances of popular language models, achieving gains of up to 4% compared with the case of data without augmentation, thereby setting a new standard in the field of text augmentation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sentiment Analysis and Opinion Mining · Advanced Text Analysis Techniques · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Graduate School of Science and Technology, Kumamoto University, Kumamoto 860-8555, Japan
Departement of Business Statistics, Institut Teknologi Sepuluh Nopember, Surabaya 60111, Indonesia
Faculty of Advanced Science and Technology, Kumamoto University, Kumamoto 860-8555, Japan
References (38)
Show source-provided references
1. Sentiment Classification Using Negative and Intensive Sentiment Supplement Information10.1007/s41019-019-0094-8 ↗
2. Improving aspect-based sentiment analysis with contrastive learning10.1016/j.nlp.2023.100009 ↗
3. Wang, W., Gan, Z., Wang, W., Shen, D., Huang, J., Ping, W., Satheesh, S., and Carin, L. (2017). Topic Compositional Neural Language Model. arXiv.
4. Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. arXiv.10.3115/v1/D14-1181 ↗
5. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
6. Small data machine learning in materials science10.1038/s41524-023-01000-z ↗
7. Data imbalance in classification: Experimental evaluation10.1016/j.ins.2019.11.004 ↗
8. Wei, J., and Zou, K. (2019, January 3–7). EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1670 ↗
9. Sugiyama, A., and Yoshinaga, N. (2019, January 3). Data augmentation using back-translation for context-aware neural machine translation. Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019), Stroudsburg, PA, USA.10.18653/v1/D19-6504 ↗
10. Kobayashi, S. (2018, January 1–6). Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, LA, USA.10.18653/v1/N18-2072 ↗
11. Data augmentation for aspect-based sentiment analysis10.1007/s13042-022-01535-5 ↗
12. Santoso, N., Mendonça, I., and Aritsugi, M. (2023, January 13–16). Text Augmentation Based on Integrated Gradients Attribute Score for Aspect-based Sentiment Analysis. Proceedings of the 2023 IEEE International Conference on Big Data and Smart Computing (BigComp), Kota Kinabalu, Malaysia.10.1109/BigComp57234.2023.00044 ↗
13. Semantic Structure and Interpretability of Word Embeddings10.1109/TASLP.2018.2837384 ↗
14. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
15. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
16. Rodrigues, J.M.F., Cardoso, P.J.S., Monteiro, J., Lam, R., Krzhizhanovskaya, V.V., Lees, M.H., Dongarra, J.J., and Sloot, P.M. (2019, January 12–14). Conditional BERT Contextual Augmentation. Proceedings of the Computational Science—ICCS 2019, Faro, Portugal.10.1007/978-3-030-22747-0 ↗
17. Data augmentation using Heuristic Masked Language Modeling10.1007/s13042-023-01784-y ↗
18. Improving classification accuracy using data augmentation on small data sets10.1016/j.eswa.2020.113696 ↗
19. Anaby-Tavor, A., Carmeli, B., Goldbraich, E., Kantor, A., Kour, G., Shlomov, S., Tepper, N., and Zwerdling, N. (2019). Not Enough Data? Deep Learning to the Rescue!. arXiv.10.1609/aaai.v34i05.6233 ↗
20. Kumar, V., Choudhary, A., and Cho, E. (2020). Data Augmentation using Pre-trained Transformer Models. arXiv.10.18653/v1/2020.lifelongnlp-1.3 ↗
21. Wang, W.Y., and Yang, D. (2015, January 17–21). That’s So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using petpeeve Tweets. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA.10.18653/v1/D15-1306 ↗
22. Unsupervised Data Augmentation for Consistency Training
23. Text augmentation using a graph-based approach and clonal selection algorithm
24. Chen, J., Zhang, R., Luo, Z., Hu, C., and Mao, Y. (2023, January 7–14). Adversarial word dilution as text data augmentation in low-resource regime. Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence, Washington, DC, USA. AAAI’23/IAAI’23/EAAI’23.
25. Tailored text augmentation for sentiment analysis10.1016/j.eswa.2022.117605 ↗
26. Liesting, T., Frasincar, F., and Truşcă, M.M. (2021, January 22–26). Data Augmentation in a Hybrid Approach for Aspect-Based Sentiment Analysis. Proceedings of the 36th Annual ACM Symposium on Applied Computing, New York, NY, USA. SAC ’21.10.1145/3412841.3441958 ↗
27. A review: Data pre-processing and data augmentation techniques10.1016/j.gltp.2022.04.020 ↗
28. A review: Preprocessing techniques and data augmentation for sentiment analysis10.1186/s40649-020-00080-x ↗
29. Calzolari, N., Choukri, K., Cieri, C., Declerck, T., Goggi, S., Hasida, K., Isahara, H., Maegaard, B., Mariani, J., and Mazo, H. (2018, January 7–12). Utilizing Large Twitter Corpora to Create Sentiment Lexica. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan.
30. Padmakumar, V., and He, H. (2021). Unsupervised Extractive Summarization using Pointwise Mutual Information. arXiv.10.18653/v1/2021.eacl-main.213 ↗
31. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv.
32. Reimers, N., and Gurevych, I. (2019, January 3–7). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1410 ↗
33. Pontiki, M., Galanis, D., Papageorgiou, H., Manandhar, S., and Androutsopoulos, I. (2015, January 4–5). SemEval-2015 Task 12: Aspect Based Sentiment Analysis. Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado.10.18653/v1/S15-2082 ↗
34. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., AL-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., and De Clercq, O. (2016, January 16–17). SemEval-2016 Task 5: Aspect Based Sentiment Analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, CA, USA.10.18653/v1/S16-1002 ↗
35. Guo, B., Han, S., and Huang, H. (2022). Selective Text Augmentation with Word Roles for Low-Resource Text Classification. arXiv.
36. Loshchilov, I., and Hutter, F. (2017). Fixing Weight Decay Regularization in Adam. arXiv.
37. Learning variable-length representation of words10.1016/j.patcog.2020.107306 ↗
38. Long text feature extraction network with data augmentation10.1007/s10489-022-03185-0 ↗