Comparison of Sarcasm Detection Models Based on Lightweight BERT Model - Differences between ALBERT-Chinese-tiny and TinyBERT in Small Sample Scenarios
DOI:
https://doi.org/10.61173/6jpm4n27Keywords:
Chinese Sarcasm Detection, Lightweight BERT model, ALBERT-Chinese-tiny, TinyBERTAbstract
Large Language models have excellent performance in processing NLP tasks, but there are not many references for lightweight models to process NLP tasks due to performance differences. In this article, I selected two lightweight BERT models, ALBERT-Chinese-tiny and TinyBERT, to process the Chinese sarcasm detection task, using annotated public Chinese sarcasm detection datasets, collecting F1 indicators, training time and other data for comparative analysis, and verifying the performance of TinyBERT in processing Chinese sentiment analysis tasks, and supplementing the training data of lightweight BERT models in Chinese sarcasm detection tasks.
References
[1] Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun LiuTinyBERT: Distilling BERT for Natural Language Understanding, https://arxiv.org/ abs/1909.10351
[2] Yangyang Li, Yuelin Li, Shihuai Zhang, Guangyuan Liu, Yanqiao Chen, Ronghua Shang, Licheng Jiao, An attentionbased, context-aware multimodalfusionmethodforsarcas m detection using inter-modality inconsistency [3]dr. ir. F. Lelli , P.K. Medappa , Sarcasm Detection in Structured Text using DistilBERT: Evaluating the Impact of Text Normalization on Model Performance, https:/doi.org/10.1016/ j.knosys.2024.111457
[4] ADITYA JOSHI,PUSHPAK BHATTACHARYYA,MARKJ. CARMAN,Automatic Sarcasm Detection: A Survey, ACMComputing Surveys, Vol. 50, No. 5, Article 73. Publication date: September 2017.
[5] WangqunChen,FuqiangLin,GuoweiLi,BoLiu,Asurveyofa utomaticsarcasmdetection:Fundamentaltheories, formulation, datasets, detection methods, and opportunities, https://doi. org/10.1016/j.neucom.2024.127428
[6] YafengRen,ZilinWang,QiongPeng,DonghongJi,Aknowledgeaugmentedneuralnetworkmodelforsarcasm detection,https://doi. org/10.1016/j.ipm.2023.103521
[7] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut, ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, https://arxiv.org/abs/1909.11942
[8] Naman Bhargava, Mohammed I. Radaideh, O Hwang Kwon, Aditi Verma, Majdi I. Radaideh, On the Impact of Language Nuances on Sentiment Analysis with Large Language Models: Paraphrasing, Sarcasm, and Emojis, https://arxiv.org/ abs/2504.05603
[9] Harleen Kaur Bagga, Jasmine Bernard, Sahil Shaheen, Sarthak Arora, Was that Sarcasm?: A Literature Survey on Sarcasm Detection, https://arxiv.org/abs/2412.00425
Downloads
Published
Issue
Section
License
Copyright (c) 2025 by the authors.
This work is licensed under a Creative Commons Attribution 4.0 International License.
