Adopting Arabic taggers to annotate a Libyan dialect text with a pre-tagging processing and term substitutions

Abstract EN

This paper reports on research work that attempts to compensate for the lack of resources when it comes to Natural Language Understanding for Libyan Arabic Dialect (LAD), particularly POS tagging.

Of prime importance is dealing with the lack of LAD POS tagging facilities.

This deficiency necessitates the use of Modern Standard Arabic (MAS) taggers in order to improve tagging results for LAD.

It is our aim to investigate the use of two commonly available taggers, namely Camel Tools and Sanford taggers for LAD.

Initial application of such POS tools to LAD collections showed poor performance.

Through the application of processing and improvements on the sentences before tagging them using MAS taggers had a positive influence on the results.

Using of an external lexicon and substitution rules base on a selected set of LAD words gave very encouraging and even better than expected.

The pilot set of experiments, even with such primitive processing based on very shallow lexicon are very encouraging with overall improvement above 30% (37.43/26.37, for camel tagger and Stanford tagger respectively).

Future work taking into account improvements on many LAD challenges relating to lexical, morphological variations and sense disambiguation are guaranteed to bring about much better results