Automatic translation of Arabic text to Arabic sign language using deep learning

Abstract EN

Deaf and dumb people are an integral part of society, must be merged with it, and must be able to communicate natively in order to get involved with the various aspects of life.

the language of communication between the deaf and dumb is sign language; a language that is not known by almost all those who do not suffer from the deficiency.

therefore, this research focuses on automating the translation of Arabic text into Arabic sign language (ARSL) in order to enable normal people to communicate with the deaf and dumb without being overburdened.

this article discusses how deep learning and neural machine translation (NMT), particularly encoder-decoder transformer architecture model, can aid this translation process.

the proposed model has been trained on a manually generated dataset of 6500 pairs of Arabic sentences and their corresponding intermediate representation of Arabic sign sentences.

the produced learning model was able to translate an input Arabic sentence into an intermediate format of sign language with an accuracy of 72%.

after generating an intermediate sentence, a video is then generated for its corresponding sign language.

the model achieved an average bleu score of 69% on the test data.