giddy.ergodic.steady_state¶

giddy.ergodic.steady_state(P, fill_empty_classes=False)[source]¶

Generalized function for calculating the steady state distribution for a regular or reducible Markov transition matrix P.

Parameters:

Parray: (k, k), an ergodic or non-ergodic Markov transition probability matrix.
fill_empty_classes: bool, optional: If True, assign 1 to diagonal elements which fall in rows full of 0s to ensure the transition probability matrix is a stochastic one. Default is False.

Returns:

: array: If the Markov chain is irreducible, meaning that there is only one communicating class, there is one unique steady state distribution towards which the system is converging in the long run. Then steady_state is the same as _steady_state_ergodic (k, ). If the Markov chain is reducible, but only has 1 recurrent class, there will be one steady state distribution as well. If the Markov chain is reducible and there are multiple recurrent classes (num_rclasses), the system could be trapped in any one of these recurrent classes. Then, there will be num_rclasses steady state distributions. The returned array will of (num_rclasses, k) dimension.

Examples

>>> import numpy as np
>>> from giddy.ergodic import steady_state

Irreducible Markov chain

>>> p = np.array([[.5, .25, .25],[.5,0,.5],[.25,.25,.5]])
>>> steady_state(p)
array([0.4, 0.2, 0.4])

Reducible Markov chain: two communicating classes

>>> p = np.array([[.5, .5, 0],[.2,0.8,0],[0,0,1]])
>>> steady_state(p)
array([[0.28571429, 0.71428571, 0.        ],
       [0.        , 0.        , 1.        ]])

Reducible Markov chain: two communicating classes

>>> p = np.array([[.5, .5, 0],[.2,0.8,0],[0,0,0]])
>>> steady_state(p, fill_empty_classes = True)
array([[0.28571429, 0.71428571, 0.        ],
       [0.        , 0.        , 1.        ]])

>>> steady_state(p, fill_empty_classes = False)
Traceback (most recent call last):
    ...
ValueError: Input transition probability matrix has 1 rows full of 0s. Please set fill_empty_classes=True to set diagonal elements for these rows to be 1 to make sure the matrix is stochastic.