The Unifying Frameworks of Information Measures

Abstract EN

Information measures are capable of providing us with fundamental methodologies to analyze uncertainty and unveiling the substantive characteristics of random variables.

In this paper, we address the issues of different types of entropies through q-generalized Kolmogorov-Nagumo averages, which lead to the propositions of the survival Rényi entropy and survival Tsallis entropy.

Therefore, we make an inventory of eight types of entropies and then classify them into two categories: the density entropy that is defined on density functions and survival entropy that is defined on survival functions.

This study demonstrates that, for each type of the density entropy, there exists a kind of the survival entropy corresponding to it.

Furthermore, the similarity measures and normalized similarity measures are, respectively, proposed for each type of entropies.

Generally, functionals of different types of information-theoretic metrics are equally diverse, while, simultaneously, they also exhibit some unifying features in all their manifestations.

We present the unifying frameworks for entropies, similarity measures, and normalized similarity measures, which helps us deal with the available information measures as a whole and move from one functional to another in harmony with various applications.