Solutions to group exercises

1
Solutions to group exercises

• 1. (a) Truncating the chain is equivalent to
  setting transition probabilities to any state in
  M1,... to zero. Renormalizing the transition
  matrix to make it stochastic we get
• (b) We have so

2

• and yielding detailed
• balance for the truncated chain.
• 2. Let Zn (X2n,X2n1). Then Zn is a Markov
  chain with transition matrix
• Let Tk P(hit (0,1) before (1,0)start at k).
• First step analysis yields the equations

3

• (b) If q0p1p we get p01,011/2, fair coin.
• 3. (a)
• so
• whence the differential equation follows by
  letting .
• (b) Since P1(t)1-P0(t) we get

4

• which can either be solved directly, or one can
  check that the given solution satisfies the
  differential equation.
• (c) Letting we get
• This value as a starting distribution also yields
  a marginal distribution that is free of t, so it
  behaves like a stationary distribution (which we
  will define later).

5
Announcement

• MathAcrossCampus Colloquium
• (http//www.math.washington.edu/mac/)
• Evolutionary trees, coalescents, and gene trees
  can mathematicians find the woods?
• JOE FELSENSTEIN
• Genome Sciences, UW
• Thursday, November 13, 2008, 330
  Kane Hall 210
• Reception to follow

6
The Markov property

• X(t) is a Markov process if for any n
• for all j, i0,...,in in S and any t0ltt1lt...lttnltt.
• The transition probabilities
• are homogeneous if pij(s,t)pij(0,t-s).
• We will usually assume this, and write pij(t).

7
Semigroup property

• Let Pt be pij(t). Then Pt is a substochastic
  semigroup, meaning that
• P0 I
• Pst PsPt
• Pt is a substochastic matrix, i.e. has
  nonnegative entries with row sums at most 1.

8
Proof

1. ?

9
Standard semigroup

• Pt,t0 is a standard semigroup if as .
• Theorem
• For a standard semigroup the transition
  probabilities are continuous.
• Proof
• By Chapman-Kolmogorov
• Unless otherwise specified we will consider
  standard semigroups.

10
Infinitesimal generator

• By continuity of the transition probabilities,
  Taylor expansion suggests
• We must have gij0, gii0. Let Ggij.
• Then (under regularity conditions)
• G is called the infinitesimal generator of Pt.

11
Birth process

• G
• Under regularity conditions we have
• so we must have

12
Forward equations

• so
• and or

13
Backward equations

• Instead of looking at (t,th look at (0,h
• so

14
Formal solution

• In many cases we can solve both these equations
  by
• But this can be difficult to actually calculate.

15
The 0-1 case
16
0-1 case, continued

• Thus

17
Marginal distribution

• Let . Then for a starting distribution ?(0)
  we have ?(t)?(0)Pt.
• For the 0-1 process we get

18
Exponential holding times

• Suppose X(t)j. Consider
• Let ? be the time spent in j until the next
  transition after time t. By the Markov property,
  P(stay in j in (u,uv, given stays at least u)
  is precisely P(stay in j v time units).
  Mathematically
• Let g(v)P(?gtv). Then we have g(uv)g(u)g(v),
  and it follows that g(u)exp(-?u). By the
  backward eqn
• and P(?gtv)pjj(v).

19
Jump chain

• Given that the chain jumps from i at a particular
  time, the probability that it jumps to j is
  -gij/gii.
• Here is why (roughly)
• Suppose tlt?ltth, and there is only one jump in
  (t,th (likely for small h). Then

20
Construction

• The way the continuous time Markov chains work
  is
• Draw an initial value i0 from ?(0)
• If , stay in i0 for a random time which is
• Draw a new state from the distribution where

21
Death process

• Let gi,i-1 ?i - gi,i. The forward equation is
• Write . Then
• This is a Lagrange equation with sln
• or