Adaptive Language Processing Based on Deep Learning in Cloud Computing Platform

Abstract EN

With the continuous advancement of technology, the amount of information and knowledge disseminated on the Internet every day has been developing several times.

At the same time, a large amount of bilingual data has also been produced in the real world.

These data are undoubtedly a great asset for statistical machine translation research.

Based on the dual-sentence quality corpus screening, two corpus screening strategies are proposed first, based on the double-sentence pair length ratio method and the word-based alignment information method.

The innovation of these two methods is that no additional linguistic resources such as bilingual dictionary and syntactic analyzer are needed as auxiliary.

No manual intervention is required, and the poor quality sentence pairs can be automatically selected and can be applied to any language pair.

Secondly, a domain adaptive method based on massive corpus is proposed.

The method based on massive corpus utilizes massive corpus mechanism to carry out multidomain automatic model migration.

In this domain, each domain learns the intradomain model independently, and different domains share the same general model.

Through the method of massive corpus, these models can be combined and adjusted to make the model learning more accurate.

Finally, the adaptive method of massive corpus filtering and statistical machine translation based on cloud platform is verified.

Experiments show that both methods have good effects and can effectively improve the translation quality of statistical machines.