journal article·2025Open access

Optimizing Airline Review Sentiment Analysis: A Comparative Analysis of LLaMA and BERT Models through Fine-Tuning and Few-Shot Learning

Konstantinos I. Roumeliotis, Nikolaos D. Tselikas, Dimitrios K. Nasiopoulos

Computers, Materials & Continua· Volume 82· Issue 2

Abstract

In the rapidly evolving landscape of natural language processing (NLP) and sentiment analysis, improving the accuracy and efficiency of sentiment classification models is crucial. This paper investigates the performance of two advanced models, the Large Language Model (LLM) LLaMA model and NLP BERT model, in the context of airline review sentiment analysis. Through fine-tuning, domain adaptation, and the application of few-shot learning, the study addresses the subtleties of sentiment expressions in airline-related text data. Employing predictive modeling and comparative analysis, the research evaluates the effectiveness of Large Language Model Meta AI (LLaMA) and Bidirectional Encoder Representations from Transformers (BERT) in capturing sentiment intricacies. Fine-tuning, including domain adaptation, enhances the models' performance in sentiment classification tasks. Additionally, the study explores the potential of few-shot learning to improve model generalization using minimal annotated data for targeted sentiment analysis. By conducting experiments on a diverse airline review dataset, the research quantifies the impact of fine-tuning, domain adaptation, and few-shot learning on model performance, providing valuable insights for industries aiming to predict recommendations and enhance customer satisfaction through a deeper understanding of sentiment in user-generated content (UGC). This research contributes to refining sentiment analysis models, ultimately fostering improved customer satisfaction in the airline industry.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Impact of AI and Big Data on Business and Society

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (47)

Show source-provided references
1. Sentiment analysis of customer feedback and reviews for airline services using language representation model10.1016/j.procs.2023.01.221 ↗
2. Proposal of a method for the analysis of sentiments in social networks with the use of R10.3390/informatics9030063 ↗
3. Investigating semantic differences in user-generated content by cross-domain sentiment analysis means
4. Monitoring and protecting company and brand reputation on social networks: when sites are not enough10.1504/GBER.2013.053075 ↗
5. Managing new product development and supply chain risks: the Boeing 787 case10.1080/16258312.2009.11517219 ↗
6. Customer experience—an analysis of the concept and its performance in airline brands10.1016/j.rtbm.2014.05.004 ↗
7. Identification of opinion trends using sentiment analysis of airlines passengers’ reviews10.1016/j.jairtraman.2022.102232 ↗
8. High enough? Explaining and predicting traveler satisfaction using airline reviews
9. Text mining approach to explore dimensions of airline customer satisfaction using online customer reviews10.1016/j.jairtraman.2019.101760 ↗
10. Reading between the lines: untwining online user-generated content using sentiment analysis
11. Airline passengers’ perceptions of service quality: themes in online reviews10.1108/IJCHM-09-2017-0572 ↗
12. A text analytics-based importance performance analysis and its application to airline service10.3390/su11216153 ↗
13. Airline service quality, customer satisfaction, and repurchase intention: Laotian air passengers’ perspective10.1016/j.cstp.2022.02.002 ↗
14. The identification of marketing performance using text mining of airline review data
15. Understanding customer experience and satisfaction through airline passengers
16. Siering · Decis Support Syst · 2018 Mar · 107 · 5210.1016/j.dss.2018.01.002 ↗
17. Measuring service quality from unstructured data: a topic modeling application on airline passengers’ online reviews10.1016/j.eswa.2018.09.037 ↗
18. Unscrambling customer recommendations: a novel LSTM ensemble approach in airline recommendation prediction using online reviews10.1109/TCSS.2022.3200890 ↗
19. Understanding customers’ evaluations through mining airline reviews10.5121/ijdkp.2015.5601 ↗
20. Predicting airline customers’ recommendations using qualitative and quantitative contents of online reviews10.1007/s11042-022-11972-7 ↗
21. Topic modeling and sentiment analysis using online reviews for bangladesh airlines
22. Significant labels in sentiment analysis of online customer reviews of airlines
23. Influence of prior reviews about a firm and its alliance partners on reviewers’ feedback: evidence from the airline industry10.1108/JSTP-06-2020-0139 ↗
24. Using transfer learning approach to implement convolutional neural network model to recommend airline tickets by using online reviews
25. Voice of airline passenger: a text mining approach to understand customer satisfaction10.1016/j.jairtraman.2019.04.001 ↗
26. BERT models for Arabic text classification: a systematic review
27. A BERT framework to sentiment analysis of tweets
28. Self-attention with relative position representations
29. BERT: pre-training of deep bidirectional transformers for language understanding
30. A literature review on bidirectional encoder representations from transformers10.1007/978-981-16-6723-7_23 ↗
31. BERT fine-tuning for sentiment analysis on indonesian mobile apps reviews
32. Transfer learning for sentiment analysis using BERT based supervised fine-tuning10.3390/s22114157 ↗
33. Improving BERT-based text classification with auxiliary sentence and domain knowledge10.1109/ACCESS.2019.2953990 ↗
34. Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, et al. Llama 2: open foundation and fine-tuned chat models; 2023 Jul [cited 2023 Oct 11]. Available from: https://arxiv.org/abs/2307.09288v2.
35. Wu C, Lin W, Zhang X, Zhang Y, Wang Y, Xie W. PMC-LLaMA: towards building open-source language models for medicine; 2023 Apr [cited 2023 Oct 13]. Available from: https://arxiv.org/abs/2304.14454v3.
36. Roziere B, Gehring J, Gloeckle F, Sootla S, Gat I, Tan XE, et al. Code Llama: open foundation models for code; 2023 Aug [cited 2023 Oct 13]. Available from: https://arxiv.org/abs/2308.12950v2.
37. LLMs in e-commerce: a comparative analysis of GPT and LLaMA models in product review evaluation10.1016/j.nlp.2024.100056 ↗
38. ChatDoctor: a medical chat model fine-tuned on a large language model meta-AI (LLaMA) using medical domain knowledge10.7759/cureus.40895 ↗
39. GitHub-kroumeliotis/Airline-Review-Sentiment-Analysis-LLaMA-2-BERT-Fine-Tuning: Optimizing Airline Review Sentiment Analysis: A Comparative Analysis of LLaMA-2 and BERT Models through Fine-Tuning and Few-Shot Learning [cited 2024 Mar 13]. Available from: https://github.com/kroumeliotis/Airline-Review-Sentiment-Analysis-LLaMA-2-BERT-Fine-Tuning.
40. Airline Reviews [cited 2024 Mar 14]. Available from: https://www.kaggle.com/datasets/juhibhojani/airline-reviews.
41. kroumeliotis/airline-reviews-6k–Run with an API on Replicate [cited 2024 Mar 21]. Available from: https://replicate.com/kroumeliotis/airline-reviews-6k.
42. kroumeliotis/airline-reviews-2k–Run with an API on Replicate [cited 2024 Mar 21]. Available from: https://replicate.com/kroumeliotis/airline-reviews-2k.
43. BERT—transformers 3.0.2 documentation [cited 2023 Dec 17]. Available from: https://huggingface.co/transformers/v3.0.2/model_doc/bert.html#berttokenizer.
44. Pretrained models—transformers 3.3.0 documentation [cited 2023 Dec 17]. Available from: https://huggingface.co/transformers/v3.3.1/pretrained_models.html.
45. Twigraph: discovering and visualizing influential words between twitter profiles10.1007/978-3-319-67256-4_26 ↗
46. Explaining anomalies detected by autoencoders using Shapley Additive Explanations10.1016/j.eswa.2021.115736 ↗
47. Evaluating local interpretable model-agnostic explanations on clinical machine learning classification models