Audio-Textual Emotion Recognition Based on Improved Neural Networks
Joint Authors
Zhou, Sitong
Hu, Yaxin
Dong, Jiangong
Cai, Linqin
Source
Mathematical Problems in Engineering
Issue
Vol. 2019, Issue 2019 (31 Dec. 2019), pp.1-9, 9 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2019-12-31
Country of Publication
Egypt
No. of Pages
9
Main Subjects
Abstract EN
With the rapid development in social media, single-modal emotion recognition is hard to satisfy the demands of the current emotional recognition system.
Aiming to optimize the performance of the emotional recognition system, a multimodal emotion recognition model from speech and text was proposed in this paper.
Considering the complementarity between different modes, CNN (convolutional neural network) and LSTM (long short-term memory) were combined in a form of binary channels to learn acoustic emotion features; meanwhile, an effective Bi-LSTM (bidirectional long short-term memory) network was resorted to capture the textual features.
Furthermore, we applied a deep neural network to learn and classify the fusion features.
The final emotional state was determined by the output of both speech and text emotion analysis.
Finally, the multimodal fusion experiments were carried out to validate the proposed model on the IEMOCAP database.
In comparison with the single modal, the overall recognition accuracy of text increased 6.70%, and that of speech emotion recognition soared 13.85%.
Experimental results show that the recognition accuracy of our multimodal is higher than that of the single modal and outperforms other published multimodal models on the test datasets.
American Psychological Association (APA)
Cai, Linqin& Hu, Yaxin& Dong, Jiangong& Zhou, Sitong. 2019. Audio-Textual Emotion Recognition Based on Improved Neural Networks. Mathematical Problems in Engineering،Vol. 2019, no. 2019, pp.1-9.
https://search.emarefa.net/detail/BIM-1194838
Modern Language Association (MLA)
Cai, Linqin…[et al.]. Audio-Textual Emotion Recognition Based on Improved Neural Networks. Mathematical Problems in Engineering No. 2019 (2019), pp.1-9.
https://search.emarefa.net/detail/BIM-1194838
American Medical Association (AMA)
Cai, Linqin& Hu, Yaxin& Dong, Jiangong& Zhou, Sitong. Audio-Textual Emotion Recognition Based on Improved Neural Networks. Mathematical Problems in Engineering. 2019. Vol. 2019, no. 2019, pp.1-9.
https://search.emarefa.net/detail/BIM-1194838
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1194838