An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences
Joint Authors
Zhao, Zheng
Shi, Xiaodong
Zheng, Jiangbin
Wu, Chong
Chen, Yidong
Tong, Yiqi
Chen, Min
Chen, Jing
Source
Computational Intelligence and Neuroscience
Issue
Vol. 2020, Issue 2020 (31 Dec. 2020), pp.1-11, 11 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2020-10-24
Country of Publication
Egypt
No. of Pages
11
Main Subjects
Abstract EN
Sign language translation (SLT) is an important application to bridge the communication gap between deaf and hearing people.
In recent years, the research on the SLT based on neural translation frameworks has attracted wide attention.
Despite the progress, current SLT research is still in the initial stage.
In fact, current systems perform poorly in processing long sign sentences, which often involve long-distance dependencies and require large resource consumption.
To tackle this problem, we propose two explainable adaptations to the traditional neural SLT models using optimized tokenization-related modules.
First, we introduce a frame stream density compression (FSDC) algorithm for detecting and reducing the redundant similar frames, which effectively shortens the long sign sentences without losing information.
Then, we replace the traditional encoder in a neural machine translation (NMT) module with an improved architecture, which incorporates a temporal convolution (T-Conv) unit and a dynamic hierarchical bidirectional GRU (DH-BiGRU) unit sequentially.
The improved component takes the temporal tokenization information into consideration to extract deeper information with reasonable resource consumption.
Our experiments on the RWTH-PHOENIX-Weather 2014T dataset show that the proposed model outperforms the state-of-the-art baseline up to about 1.5+ BLEU-4 score gains.
American Psychological Association (APA)
Zheng, Jiangbin& Zhao, Zheng& Chen, Min& Chen, Jing& Wu, Chong& Chen, Yidong…[et al.]. 2020. An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. Computational Intelligence and Neuroscience،Vol. 2020, no. 2020, pp.1-11.
https://search.emarefa.net/detail/BIM-1138851
Modern Language Association (MLA)
Zheng, Jiangbin…[et al.]. An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. Computational Intelligence and Neuroscience No. 2020 (2020), pp.1-11.
https://search.emarefa.net/detail/BIM-1138851
American Medical Association (AMA)
Zheng, Jiangbin& Zhao, Zheng& Chen, Min& Chen, Jing& Wu, Chong& Chen, Yidong…[et al.]. An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. Computational Intelligence and Neuroscience. 2020. Vol. 2020, no. 2020, pp.1-11.
https://search.emarefa.net/detail/BIM-1138851
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1138851