Clustering Vietnamese conversations from Facebook page to build training dataset for chatbot
Joint Authors
Nguyen, Thanh Q.
Nguyen, Trieu Hai
Pham, Thi Kim Ngoan
Bui, Thi-Hong-Minh
Source
Jordanian Journal of Computetrs and Information Technology
Issue
Vol. 8, Issue 1 (31 Mar. 2022), pp.1-17, 17 p.
Publisher
Princess Sumaya University for Technology
Publication Date
2022-03-31
Country of Publication
Jordan
No. of Pages
17
Main Subjects
Information Technology and Computer Science
Abstract EN
The biggest challenge of building chatbots is training data.
The required data must be realistic and large enough to train chatbots.
We create a tool to get actual training data from Facebook messenger of a Facebook page.
After text preprocessing steps, the newly obtained dataset generates FVnC and Sample dataset.
We use the Retraining of BERT for Vietnamese (PhoBERT) to extract features of our text data.
K-Means and DBSCAN clustering algorithms are used for clustering tasks based on output embeddings from PhoBERTbase.
We apply V-measure score and Silhouette score to evaluate the performance of clustering algorithms.
We also demonstrate the efficiency of PhoBERT compared to other models in feature extraction on the Sample dataset and wiki dataset.
A GridSearch algorithm that combines both clustering evaluations is also proposed to find optimal parameters.
Thanks to clustering such a number of conversations, we save a lot of time and effort to build data and storylines for training chatbot.
American Psychological Association (APA)
Nguyen, Trieu Hai& Pham, Thi Kim Ngoan& Bui, Thi-Hong-Minh& Nguyen, Thanh Q.. 2022. Clustering Vietnamese conversations from Facebook page to build training dataset for chatbot. Jordanian Journal of Computetrs and Information Technology،Vol. 8, no. 1, pp.1-17.
https://search.emarefa.net/detail/BIM-1416101
Modern Language Association (MLA)
Nguyen, Trieu Hai…[et al.]. Clustering Vietnamese conversations from Facebook page to build training dataset for chatbot. Jordanian Journal of Computetrs and Information Technology Vol. 8, no. 1 (Mar. 2022), pp.1-17.
https://search.emarefa.net/detail/BIM-1416101
American Medical Association (AMA)
Nguyen, Trieu Hai& Pham, Thi Kim Ngoan& Bui, Thi-Hong-Minh& Nguyen, Thanh Q.. Clustering Vietnamese conversations from Facebook page to build training dataset for chatbot. Jordanian Journal of Computetrs and Information Technology. 2022. Vol. 8, no. 1, pp.1-17.
https://search.emarefa.net/detail/BIM-1416101
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references : p. 15-17
Record ID
BIM-1416101