Markov Chain

1
Markov Chain

• A Markov chain is a mathematical system that
  undergoes transitions from one state to another
  on a state space. Based on Markov property, the
  next state depends only on the current state and
  not on the sequence of events that preceded
  it.e.g. Two-state Markov chain
• Stationary process is a stochastic processwhose
  joint probability distribution does not change
  when shifted in time. A stationary Markov chain
  is the Markov chain with constant transition
  matrix P.
• A stationary distribution (also called an
  equilibrium distribution) of a Markov chain is a
  probability distribution which satisfy(It
  actually is related to the Eigen vector of P
  corresponding to Eigen value ? 1)

Transition matrix P
2
Entropy Rate of Markov Chain

• Entropy is defined by
• The Entropy rate of a stochastic process Xi is
  defined by
• When the limit exists. (For stationary
  processes, it exists.)
• For stationary Markov chain Xi with stationary
  distribution and transition matrix P. The
  Entropy rate is given by
• Conditional entropy is calculated using the
  stationary distribution.