A review of sentiment analysis research based on BERT and its improved models

Abstract

In recent years, the rapid evolution of social media and online reviews has exposed limitations in traditional sentiment analysis methods, which rely on sentiment lexicons and machine learning classifiers, particularly in terms of contextual modeling and generalization capabilities. In past few years, Deep learning, particularly convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their variants, has demonstrated superior feature learning capabilities in sentiment recognition. However, these methods are still limited by the long-distance dependency modeling and the complexity of Chinese semantics. The introduction of BERT has opened a new chapter for pre-trained language models in sentiment analysis. The model’s semantic understanding capabilities have been significantly enhanced through a bidirectional Transformer architecture and large-scale corpus pre-training. Subsequently, not only ALBERT, but also improved models such as RoBERTa played a significant role in English sentiment analysis tasks after fine-tuning according to task requirements. A series of improved models such as RoBERTa-wwm-ext, ERNIE, ALBERT-zh, and MacBERT continued to refresh performance records in Chinese sentiment analysis tasks. This paper systematically reviews the research progress of sentiment analysis based on BERT and its improved models in recent years. It focuses on comparing the performance of different models in text sentiment classification, implicit sentiment recognition, and fine-grained sentiment analysis, and reveals their advantages and challenges in semantic modeling, cross-domain transfer, and multi-granularity information fusion. Finally, this paper discusses the current bottlenecks faced by the research and looks forward to the future development direction of sentiment analysis.