Speech emotion analysis based on vision transformer

Xiaogang Huang; Qifeng Zheng; Yuanyuan Zhang; Dong Cheng; Yuting Liu; Chen Dong

doi:10.1117/12.2673332

13 April 2023 Speech emotion analysis based on vision transformer

Xiaogang Huang, Qifeng Zheng, Yuanyuan Zhang, Dong Cheng, Yuting Liu, Chen Dong

Proceedings Volume 12605, 2022 2nd Conference on High Performance Computing and Communication Engineering (HPCCE 2022); 126051K (2023) https://doi.org/10.1117/12.2673332
Event: Second Conference on High Performance Computing and Communication Engineering, 2022, Harbin, China

Abstract

Emotion is an essential aspect of human life, and effectively identifying corresponding emotions from different scenarios will help promote the development of human-computer interaction systems. Therefore, emotion classification has gradually become a challenging and popular research field. Compared with text emotion analysis, emotion analysis of audio data is still relatively immature. Traditional audio sentiment analysis research is based on feature information such as MFCC, MFSC, etc. while using time-memory models such as LSTM and RNN for emotion analysis. Due to the rapid development of transformers and attention mechanisms, many scholars have shifted their research from the RNN family to the transformer family or deep learning models with attention mechanisms. Therefore, this paper proposes a method to convert audio data into a spectrogram and use a vision transformer model based on transfer learning for emotion classification. This paper conducts experiments on the IEMOCAP dataset and the MELD dataset. The experimental results show that the emotion classification accuracy of the Vision transformer in the IEMOCAP and the MELD datasets reach 56.18% and 37.1%.

Citation Download Citation

Xiaogang Huang, Qifeng Zheng, Yuanyuan Zhang, Dong Cheng, Yuting Liu, and Chen Dong "Speech emotion analysis based on vision transformer", Proc. SPIE 12605, 2022 2nd Conference on High Performance Computing and Communication Engineering (HPCCE 2022), 126051K (13 April 2023); https://doi.org/10.1117/12.2673332

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available