Markov Chains: Part I

1
Markov Chains Part I
2
A Typical Problem
The city of Math Island is experiencing a
movement of its population to the suburbs. At
present, 85 of the total population lives in the
city and 15 lives in the suburbs. But each
year, 7 of the city people move to the suburbs,
while only 1 of the suburb people move back to
the city. Assume that the total population
remains constant.
Let C0, C1, C2, represent the percentages of
the population in the city, respectively, now, 1
year from now, 2 years from now, and so on.
Also, let S0, S1, S2, be the percentages of
the population in the suburbs, respectively, now,
1 year from now, 2 years from now, and so on.
3
Question 1
Use the tree diagram shown below to verify that
after 1 year 79.2 of the residents of Math
Island live in the city and 20.8 live in the
suburbs.
Now
1 year from now
C1
.93
C0
S1
.85
.07
C1
.01
.15
S0
S1
.99
4
Question 2
Use the extended tree diagram to verify that
after 2 years 73.9 of the residents of Math
Island live in the city and 26.1 live in the
suburbs.
Now
1 year from now
2 years from now
C2
.93
C1
S2
.93
.07
C2
.01
C0
S1
.85
S2
.07
.99
C2
.93
.01
C1
.15
S2
S0
.07
S1
.01
.99
C2
S2
.99
5
Question 3
If we used a tree diagram to determine the
probability distribution of the residents in 8
years, how many branches would there be on the
tree?
Hint The number of branches is a power of 2.
6
Would you make a tree diagram to find the
distribution of the population 8 years from now?
Just imagine how long it would take!
We hope that there is a more efficient way to
predict the future distribution of the
population.
7
A Different Approach
In life, we often need to change our ways of
dealing with an issue.
We now change our view of the Typical Problem.
Instead of just using probability tree diagrams
to find the future distribution of the
population, let us turn to matrix multiplication.
Try to see what makes one think of using matrix
multiplication!
8
Question 4
Consider a 12 matrix that represents the initial
probability distribution of the residents, say
City Suburbs
City Suburbs
and a 22 matrix that represents the movement of
population, say
City Suburbs
Find the elements of the matrices D0 and M.
9
A word on terminology
In Markov Chain theory, the matrix M is called a
Transition Matrix. Why is the name
appropriate? It is usually denoted by the letter
T.
The matrix D0 is referred to as the Initial
State Matrix. It usually denoted by S0.
10
Question 5
Verify that the probability distribution of the
residents of Math Island after 1 year and after
2 years are given, respectively, by D1 D0 M
and D2 D1 M.
11
Question 6
a) Find an expression for the probability
distribution after 2 years, D2, in terms of D0.
b) Deduce from that an expression for the
probability distributions after 3, 4, and n
years, i.e. D3, D4, and Dn in terms of D0.
Hint Substitute and find a pattern.
12
Question 7
Find the population distribution 8 years from
now.
We dont need to draw a tree diagram with 512
branches.
13
The Big Picture
The Typical Problem that weve been working with
is an example of a Markov chain, or Markov
process.
The idea is that a system is evolving from one
state to another in such a way that chances are
involved in progressing from one state to the
next.
We consider only the case when the transition
matrix (the square matrix that indicates the
probability of moving from one state to another)
is constant.
14
Let S0 be the initial state matrix and T the
transition matrix for the Markov chain. Then, as
was the case in the Typical Problem, the
probability distribution at the nth state (when
the experiment has been repeated n times) is
15
Example
The buying pattern of Lost City home buyers who
buy single-family homes and condominiums has been
observed and it was discovered that 85 of
single-family homeowners buy again single-family
homes and 65 of condominium owners buy again
condominiums. Currently, 80 of the homeowners
live in single-family homes and 20 live in
condominiums. If this trend continues, what will
be the percentage of homeowners in the city that
will own single-family homes and condominiums 2
years from now? 5 years from now?
16
More Questions
In Part II, well deal with some
thought-provoking questions

• In the Typical Problem, it is evident that the
  percent of the total population that remains in
  the city is decreasing. Does the situation ever
  stabilize?
• In the long run, does the probability
  distribution of the residents depend on the
  initial distribution D0?