Convergence Analysis of Contrastive Divergence Algorithm Based on Gradient Method with Errors

الملخص EN

Contrastive Divergence has become a common way to train Restricted Boltzmann Machines; however, its convergence hasnot been made clear yet.

This paper studies the convergence of Contrastive Divergence algorithm.

We relate ContrastiveDivergence algorithm to gradient method with errors and derive convergence conditions of Contrastive Divergence algorithmusing the convergence theorem of gradient method with errors.

We give specific convergence conditions of ContrastiveDivergence learning algorithm for Restricted Boltzmann Machines in which both visible units and hidden units can only takea finite number of values.

Two new convergence conditions are obtained by specifying the learning rate.

Finally, we givespecific conditions that the step number of Gibbs sampling must be satisfied in order to guarantee the Contrastive Divergencealgorithm convergence.