Stochastic Hybrid Systems and Biological Applications

1
Stochastic Hybrid Systems andBiological
Applications

• Jianghai Hu
• Ian Mitchell
• Wei-Chung Wu
• Shankar Sastry

2
Outline

• Deterministic safety analysis
• Collision detection and avoidance
• Adding Uncertainty
• Stochastic hybrid automata
• Computation
• Hamilton-Jacobi PDEs
• Markov chain discretizations
• Biological models
• Stochastic Ion Gating
• E. coli population

3
Example Hybrid System

• Seven mode collision avoidance protocol
• Aircraft execute synchronized sequence of curved
  and straight flight segments to avoid potential
  collision
• Where should we initiate the protocol?

4
Reachability Analysis Results
safe with switch
unsafe with or without switch
safe without switch unsafe to switch
5
Reachable Sets What and Why?

• One application safety analysis
• What states are doomed to become unsafe?
• What states are safe given an appropriate control
  strategy?

unsafe initialization
unsafe (uncontrollable)
initial conditions
unsafe (a priori)
safe (under appropriate control)
6
Continuous Collision Avoidance

• Classical game of two identical vehicles
• Collision occurs if vehicles get within five
  units
• Evader chooses turn rate u 1 to avoid
  collision
• Pursuer chooses turn rate d 1 to cause
  collision
• Fixed equal velocity v 5

dynamics (pursuer)
7
Avoiding Collision

• Softwall demonstration
• Pursuer turn to head toward evader
• Evader turn to head east
• Evaders input is filtered to guarantee that
  pursuer does not enter the reachable set

8
Computing Reachable Sets

• Modified Hamilton-Jacobi partial differential
  equation

Converged reachable set G
Growth of reachable set G(t)
9
Collision Alert for ATC

• Use reachable set to detect potential collisions
  and warn ATC
• Find aircraft pairs in ETMS database whose flight
  plans intersect
• Check whether either aircraft is in the others
  collision region
• If so, examine ETMS data to see if aircraft path
  is deviated
• One hour sample in Oakland centers airspace
• 1590 pairs, 1555 no conflict, 25 detected
  conflicts, 2 false alerts

Work with Claire Tomlin, Alex Bayen Shriram
Santhanam at Stanford
10
Outline

• Deterministic safety analysis
• Collision detection and avoidance
• Adding Uncertainty
• Stochastic hybrid automata
• Computation
• Hamilton-Jacobi PDEs
• Markov chain discretizations
• Biological models
• Stochastic Ion Gating
• E. coli population

11
(non)Deterministic Hybrid Automata

• Discrete modes and transitions
• Continuous evolution within each mode

s1 initiate maneuver
t p/4
t T
t p/2
t p/4
t T
12
Stochastic Hybrid Automata

• Uncertainty can be introduced in many locations
• Timing of switches
• New mode after a switch
• Continuous state jump during a switch
• Continuous state evolution within a mode
• Response to inputs

13
Stochastic Differential Equations

• Uncertainty in continuous evolution
• Itô integral
• Uncertainty arises from white noise process W(t)
• Differential form

14
Outline

• Deterministic safety analysis
• Collision detection and avoidance
• Adding Uncertainty
• Stochastic hybrid automata
• Computation
• Hamilton-Jacobi PDEs
• Markov chain discretizations
• Biological models
• Stochastic Ion Gating
• E. coli population

15
Hamilton-Jacobi formulation

• ODE leads to first order nonlinear HJ PDE
• SDE leads to second order linear HJ PDE
• Choice of terminal conditions determines
  interpretation of solution J(x,t)
• Implicit surface function representation of
  expected collision set
• Maximum probability of collision

Work with Andrew Zimdars
16
Markov Chain Discretization

• Idea approximate the solution to the SDE with a
  Markov chain defined on some grid points

D
17
Iso-Probability Surfaces
Algorithm return a set of iso-probability
surfaces Soft reachability sets
US Class C AirSpace
18
Outline

• Deterministic safety analysis
• Collision detection and avoidance
• Adding Uncertainty
• Stochastic hybrid automata
• Computation
• Hamilton-Jacobi PDEs
• Markov chain discretizations
• Biological models
• Stochastic Ion Gating
• E. coli population

19
Stochastic Hybrid Systems

• State variables
• continuous variable X
• discrete variable S.
• System dynamics
• continuous dynamics
• discrete dynamics

X(t) Ordinary or stochastic differential
equation
20
A Typical Sample Path
21
Inter-dependent Dynamics

• Discrete dynamics ? continuous dynamics
• Different continuous dynamics for different S
• Reset of X after jumps
• Continuous dynamics ? discrete dynamics
• Transition probabilities depend on X
• Time between jumps depends on X

dX(t)/dtf(X,S,t)?(X,S,t)dW(t)
Transition matrix P pij(X)ij1,,K
22
A Simple Example

• Continuous variable X?R1
• Discrete variable S0,1,,K
• Continuous dynamics linear, deterministic
• Discrete dynamics Markov chain

dX(t)/dtf(S)
23
Stochastic Gating of Ion Channels

• Two-state ion channel
• Transport selected ions across membrane
• In either open or closed state
• Voltage across membrane V
• Difference in ion concentrations

24
Stochastic Gating of Ion Channels

• V voltage across membrane
• S number of open ion channels
• Continuous dynamics
• Discrete dynamics

C dV(t)/dt Ileak(K-S)IcSIo
Single gate
S birth and death chain
25
E. Coli

• A population of K E. coli
• Certain amount of foods available
• Each E. coli is either piliated or unpiliated.
• Piliated E. coli generate foods
• Unpiliated E. coli consume foods
• More foods imply higher probability of
• Piliated ? Piliated
• Unpilated ? Unpiliated

26
E. Coli

• X amount of foods available
• S number of piliated E. coli
• Continuous dynamics
• Discrete dynamics S a birth and death chain

dX(t)/dt u0(K-S)uuSup, uult0ltup
27
Embedded Markov Chains

• (Xk,Sk) samples of X and S at moments ?k when
  the k-th jump has just completed.
• (Xk,Sk) is an (embedded) Markov chain
• Characterizes (X,S) completely.
• Computational tractable
• ?(k)(x,s) the joint distribution of (Xk,Sk)
• ?(?)(x,s) the equilibrium distribution

?(k)(x,s)? ?(?)(x,s), k??
28
Some Analytical Results

• Continuous dynamics
• Regardless of the transition probabilities,
• In particular, if K1,

dX(t)/dtui , if S(t)i, i0,,K
u1P?(S0)uKP?(SK)0
P?(S0)u1/(u1-u0) P?(S1)-u0/(u1-u0) E?XkSk0
- E?XkSk1(u1-u0)?/2
29
Numerical Simulations

• Conditional expectation of Xk depends on
  transition probabilities
• Examples of equilibrium distributions (K1)

u0-30,u19
u0-9,u130
30
Future Works

• Nonlinear continuous dynamics
• Limiting behavior when K??
• Stochastic Morris-Lecar Equation
• Second order statistics of equilibrium
  distributions
• Model validation (collaboration with Arkins
  group)