On the entropy of Arabic

Abstract EN

The entropy of a language is a parameter with important applications ١١؛ cryptography, coding, and natural language processing.

Entropies for languages such as English have been estimated to reasonable levels of accuracy due to the availability of extensive statistics for the letter and word frequencies of these languages.

Similar statistics for Arabic became available only recently and have since been used to estimate the entropy of Arabic.

This paper reviews two approaches for estimating the entropy of Arabic, and presents a new one.

The first approach uses the letter frequencies for estimating the entropy, the second gives estimate of the entropy based on word frequencies and roots count.

The new approach gives estimates of the entropy based on a proposed method to upperbound the number of meaningful Arabic sentences of length n letters.

The new method results in estimates that are close to those given by higher order entropies of Arabic as computed by some researchers, but with much less computational effort.