![](/images/graphics-bg.png)
Multimodal Fusion Method Based on Self-Attention Mechanism
Joint Authors
Zhu, Hu
Wang, Ze
Hua, Yingying
Xu, Guoxia
Deng, Lizhen
Shi, Yu
Source
Wireless Communications and Mobile Computing
Issue
Vol. 2020, Issue 2020 (31 Dec. 2020), pp.1-8, 8 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2020-09-23
Country of Publication
Egypt
No. of Pages
8
Main Subjects
Information Technology and Computer Science
Abstract EN
Multimodal fusion is one of the popular research directions of multimodal research, and it is also an emerging research field of artificial intelligence.
Multimodal fusion is aimed at taking advantage of the complementarity of heterogeneous data and providing reliable classification for the model.
Multimodal data fusion is to transform data from multiple single-mode representations to a compact multimodal representation.
In previous multimodal data fusion studies, most of the research in this field used multimodal representations of tensors.
As the input is converted into a tensor, the dimensions and computational complexity increase exponentially.
In this paper, we propose a low-rank tensor multimodal fusion method with an attention mechanism, which improves efficiency and reduces computational complexity.
We evaluate our model through three multimodal fusion tasks, which are based on a public data set: CMU-MOSI, IEMOCAP, and POM.
Our model achieves a good performance while flexibly capturing the global and local connections.
Compared with other multimodal fusions represented by tensors, experiments show that our model can achieve better results steadily under a series of attention mechanisms.
American Psychological Association (APA)
Zhu, Hu& Wang, Ze& Shi, Yu& Hua, Yingying& Xu, Guoxia& Deng, Lizhen. 2020. Multimodal Fusion Method Based on Self-Attention Mechanism. Wireless Communications and Mobile Computing،Vol. 2020, no. 2020, pp.1-8.
https://search.emarefa.net/detail/BIM-1214697
Modern Language Association (MLA)
Zhu, Hu…[et al.]. Multimodal Fusion Method Based on Self-Attention Mechanism. Wireless Communications and Mobile Computing No. 2020 (2020), pp.1-8.
https://search.emarefa.net/detail/BIM-1214697
American Medical Association (AMA)
Zhu, Hu& Wang, Ze& Shi, Yu& Hua, Yingying& Xu, Guoxia& Deng, Lizhen. Multimodal Fusion Method Based on Self-Attention Mechanism. Wireless Communications and Mobile Computing. 2020. Vol. 2020, no. 2020, pp.1-8.
https://search.emarefa.net/detail/BIM-1214697
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1214697