Multimodal sentiment analysis with BERT-ResNet50

Senchang Zhang; Yue He; Lei Li; Yaowen Dou

doi:10.1117/12.2679113

8 May 2023 Multimodal sentiment analysis with BERT-ResNet50

Senchang Zhang, Yue He, Lei Li, Yaowen Dou

Proceedings Volume 12635, Second International Conference on Algorithms, Microchips, and Network Applications (AMNA 2023); 1263510 (2023) https://doi.org/10.1117/12.2679113
Event: International Conference on Algorithms, Microchips, and Network Applications 2023, 2023, Zhengzhou, China

Abstract

Aiming at the problem that the information difference between modalities in the current multimodal sentiment analysis model and the insufficient fusion between modalities lead to the low accuracy of network prediction, this paper designs a multimodal sentiment analysis model based on BERT-ResNet50. The model uses BERT and ResNet50 to extract text and image features respectively, fuses multi-modal information through the encoder layer of Transformer, and finally uses the Softmax layer to classify multi-modal information. The dataset used in this paper is the Twitter sarcasm public dataset. Through experiments, the BERT-ResNet50 model proposed in this paper is higher than the comparison models in accuracy, recall rate and F1 value, and the accuracy reaches 74.05%. Ablation experiments show that the accuracy of the model in multi-modal sentiment analysis is higher than that in single-modal sentiment analysis.

Citation Download Citation

Senchang Zhang, Yue He, Lei Li, and Yaowen Dou "Multimodal sentiment analysis with BERT-ResNet50", Proc. SPIE 12635, Second International Conference on Algorithms, Microchips, and Network Applications (AMNA 2023), 1263510 (8 May 2023); https://doi.org/10.1117/12.2679113

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
5 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Feature extraction

Image processing

Transformers

Image fusion

Engineering

Reflection

Show All Keywords

Keywords/Phrases

Search In:

Publication Years