![](/images/graphics-bg.png)
A Unified Definition of Mutual Information with Applications in Machine Learning
Author
Source
Mathematical Problems in Engineering
Issue
Vol. 2015, Issue 2015 (31 Dec. 2015), pp.1-12, 12 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2015-03-31
Country of Publication
Egypt
No. of Pages
12
Main Subjects
Abstract EN
There are various definitions of mutual information.
Essentially, these definitions can be divided into two classes: (1) definitions with random variables and (2) definitions with ensembles.
However, there are some mathematical flaws in these definitions.
For instance, Class 1 definitions either neglect the probability spaces or assume the two random variables have the same probability space.
Class 2 definitions redefine marginal probabilities from the joint probabilities.
In fact, the marginal probabilities are given from the ensembles and should not be redefined from the joint probabilities.
Both Class 1 and Class 2 definitions assume a joint distribution exists.
Yet, they all ignore an important fact that the joint or the joint probability measure is not unique.
In this paper, we first present a new unified definition of mutual information to cover all the various definitions and to fix their mathematical flaws.
Our idea is to define the joint distribution of two random variables by taking the marginal probabilities into consideration.
Next, we establish some properties of the newly defined mutual information.
We then propose a method to calculate mutual information in machine learning.
Finally, we apply our newly defined mutual information to credit scoring.
American Psychological Association (APA)
Zeng, Guoping. 2015. A Unified Definition of Mutual Information with Applications in Machine Learning. Mathematical Problems in Engineering،Vol. 2015, no. 2015, pp.1-12.
https://search.emarefa.net/detail/BIM-1073207
Modern Language Association (MLA)
Zeng, Guoping. A Unified Definition of Mutual Information with Applications in Machine Learning. Mathematical Problems in Engineering No. 2015 (2015), pp.1-12.
https://search.emarefa.net/detail/BIM-1073207
American Medical Association (AMA)
Zeng, Guoping. A Unified Definition of Mutual Information with Applications in Machine Learning. Mathematical Problems in Engineering. 2015. Vol. 2015, no. 2015, pp.1-12.
https://search.emarefa.net/detail/BIM-1073207
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1073207