Research and Application Analysis of Image-Based Sentiment Analysis Methods

Authors

  • Yuxuan Shi
  • Beichen Zhang

DOI:

https://doi.org/10.61173/tgqfb923

Keywords:

Image-Based Emotion Analysis, Multimodal Fusion, Vision Transformer, Lightweight Models

Abstract

Emotion, as the core of human subjective experience, exerts a profound impact on mental health and social interactions. With the development of computer vision technology, image-based emotion analysis has become a critical research direction, finding wide application in driving safety, advertising marketing, healthcare, and other fields. However, the diversity and complexity of emotional expressions, along with environmental interference and challenges in multimodal fusion, still hinder the improvement of its accuracy and robustness. This paper provides a comprehensive review and analysis of imagebased emotion analysis methods and their practical applications. First, it briefly introduces the research background and application value of this field, highlighting its key roles in scenarios such as driver/passenger emotion monitoring in autonomous driving and consumer emotion recognition in advertising. Subsequently, four categories of core technical methods are elaborated in detail. This paper also conducts a systematic comparison of the four method categories in terms of their core technical frameworks, key innovations, performance (efficiency and robustness), applicable scenarios, and limitations. Finally, it summarizes three major current challenges: the diversity of emotional expressions, data annotation issues, and multimodal fusion difficulties. Corresponding future directions are proposed, including enhancing model robustness and generalization, optimizing data and efficient models, and advancing adaptive multimodal fusion technologies, aiming to provide comprehensive technical references for the development and application of this field.

References

[1] Schuller BW, Schuller DM. Audiovisual affect assessment and autonomous automobiles: Applications. arXiv preprint arXiv:2203.07482. 2022.

[2] Pilli S, Patwardhan M, Pedanekar N, Karande S. Predicting sentiments in image advertisements using semantic relations among sentiment labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020. p. 408–9.

[3] Budu T, Sato W, Shimokawa K, Hsu CT, Kochiyama T. Development of pixel-based facial thermal image analysis for emotion sensing. Computers in Human Behavior Reports. 2025;100761.

[4] Sato W, Kochiyama T, Yoshikawa S. Physiological correlates of subjective emotional valence and arousal dynamics while viewing films. Biological Psychology. 2020;157:107974.

[5] Cheng Z, Cheng ZQ, He JY, Wang K, Lin Y, Lian Z, et al. Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning. Advances in Neural Information Processing Systems. 2024;37:110805–53.

[6] Abbas SMA, Abbas Q, Naqi SM. EmotionNet-X: An optimized CNN architecture for robust facial emotion analysis. IEEE Access. 2025.

[7] Cai Y, Li X, Zhang Y, Li J, Zhu F, Rao L. Multimodal sentiment analysis based on multi-layer feature fusion and multitask learning. Scientific Reports. 2025;15(1):2126.

[8] Sagaidachnyi AA, Fomin AV, Usanov DA, Skripal AV. Thermography-based blood flow imaging in human skin of the hands and feet: A spectral filtering approach. Physiological Measurement. 2017;38(2):272.

[9] Xia C, Wang X, Lv F, Hao X, Shi Y. Vit-comer: Vision transformer with convolutional multi-scale feature interaction for dense predictions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. p. 5493– 502.

[10] Haruna Y, Qin S, Chukkol AHA, Yusuf AA, Bello I, Lawan A. Exploring the synergies of hybrid convolutional neural network and Vision Transformer architectures for computer vision: A survey. Engineering Applications of Artificial Intelligence. 2025;144:110057.

[11] Nadeem M, Sohail SS, Javed L, Anwer F, Saudagar AKJ, Muhammad K. Vision-enabled large language and deep learning models for image-based emotion recognition. Cognitive Computation. 2024;16(5):2566–79.

Downloads

Published

2025-12-19