Computation of Invariant Measures and Stationary Expectations for Markov Chains with Block-Band Transition Matrix

Abstract EN

This paper deals with the computation of invariant measures and stationary expectations for discrete-time Markov chains governed by a block-structured one-step transition probability matrix.

The method generalizes in some respect Neuts’ matrix-geometric approach to vector-state Markov chains.

The method reveals a strong relationship between Markov chains and matrix continued fractions which can provide valuable information for mastering the growing complexity of real-world applications of large-scale grid systems and multidimensional level-dependent Markov models.

The results obtained are extended to continuous-time Markov chains.