A Hybrid Neural Network BERT-Cap Based on Pre-Trained Language Model and Capsule Network for User Intent Classification
Joint Authors
Liu, Yuanxia
Wong, Leung-Pun
Lee, Lap-Kei
Hao, Tianyong
Liu, Hai
Source
Issue
Vol. 2020, Issue 2020 (31 Dec. 2020), pp.1-11, 11 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2020-11-21
Country of Publication
Egypt
No. of Pages
11
Main Subjects
Abstract EN
User intent classification is a vital component of a question-answering system or a task-based dialogue system.
In order to understand the goals of users’ questions or discourses, the system categorizes user text into a set of pre-defined user intent categories.
User questions or discourses are usually short in length and lack sufficient context; thus, it is difficult to extract deep semantic information from these types of text and the accuracy of user intent classification may be affected.
To better identify user intents, this paper proposes a BERT-Cap hybrid neural network model with focal loss for user intent classification to capture user intents in dialogue.
The model uses multiple transformer encoder blocks to encode user utterances and initializes encoder parameters with a pre-trained BERT.
Then, it extracts essential features using a capsule network with dynamic routing after utterances encoding.
Experiment results on four publicly available datasets show that our model BERT-Cap achieves a F1 score of 0.967 and an accuracy of 0.967, outperforming a number of baseline methods, indicating its effectiveness in user intent classification.
American Psychological Association (APA)
Liu, Hai& Liu, Yuanxia& Wong, Leung-Pun& Lee, Lap-Kei& Hao, Tianyong. 2020. A Hybrid Neural Network BERT-Cap Based on Pre-Trained Language Model and Capsule Network for User Intent Classification. Complexity،Vol. 2020, no. 2020, pp.1-11.
https://search.emarefa.net/detail/BIM-1144954
Modern Language Association (MLA)
Liu, Hai…[et al.]. A Hybrid Neural Network BERT-Cap Based on Pre-Trained Language Model and Capsule Network for User Intent Classification. Complexity No. 2020 (2020), pp.1-11.
https://search.emarefa.net/detail/BIM-1144954
American Medical Association (AMA)
Liu, Hai& Liu, Yuanxia& Wong, Leung-Pun& Lee, Lap-Kei& Hao, Tianyong. A Hybrid Neural Network BERT-Cap Based on Pre-Trained Language Model and Capsule Network for User Intent Classification. Complexity. 2020. Vol. 2020, no. 2020, pp.1-11.
https://search.emarefa.net/detail/BIM-1144954
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1144954