On the convergence of SDDP and related algorithms

1
On the convergence of SDDP and related algorithms

• Speaker Ziming Guan
• Supervisor A. B. Philpott
• Sponsor Fonterra New Zealand

2
Motivation

• Pereira and Pinto, Multi-Stage Stochastic
  Optimization Applied to Energy Planning,
  Mathematical Programming, 52, pp. 359-375, 1991.

3
Summary

• Description of problem class
• SDDP and its related algorithm
• Theoretical convergence
• Implementation issues

4
Properties for random quantities

• Random quantities appear only on the right-hand
  side of the linear constraints in each stage.
• The set of random outcomes is discrete and
  finite.
• Random quantities in different stages are
  independent.
• Can accommodate PARMA process for RHS uncertainty.

5
Scenario tree, scenario outcome, scenario
w1(2)
p21
w2(2)
w1(1)
w3(2)
p11
p21
p12
w2(1)
p13
w1(2)
p21
w2(2)
w3(1)
w3(2)
6
Hydro-thermal scheduling
7
Stage problem
8
Cuts
T(t1)
Reservoir storage, x(t1)
9
Stochastic Dual Dynamic Programming

• Pereira and Pinto, 1991
• Initialization Sample some scenarios and fix
  them through the course of the algorithm.
• Forward pass For stage t1,,T, solve the stage
  t problem for each scenario.
• Calculate the lower bound and upper bound.
• If not converge,
• Backward pass For stage tT-1,,1, for the stage
  t problem in each scenario, solve all stage t1
  problems to calculate a cut for stage t problems.
• Back to Forward pass.

10
w2(2)
w2(1)
w1(3), ?3(3)
w1(2)
w2(2)
w1(1)
w3(2)
p11
p12
w2(1)
p13
w3(1)
11
w1(1)
p11
p12
w2(1)
p13
w3(1)
12
w2(2)
w2(1)
w1(3), ?3(3)
w1(2)
w2(2)
w1(1)
w3(2)
p11
p12
w2(1)
p13
w3(1)
13
Dynamic Outer Approximation Sampling Algorithm

• No upper bound calculation until algorithm is
  terminated.

14
w2(2)
w2(1)
w3(3)
w1(2)
w2(2)
w1(1)
w3(2)
p11
p12
w2(1)
p13
w3(1)
15
w1(1)
p11
p12
w2(1)
p13
w3(1)
16
w2(2)
w2(1)
w1(3)
w1(2)
w2(2)
w1(1)
w3(2)
p11
p12
w2(1)
p13
w3(1)
17

• We have a convergence proof for DOASA.
• This can be used to understand the convergence
  behaviour of SDDP.

18
Sampling properties of DOASA

• Forward Pass Sampling Property (FPSP) Each
  scenario is traversed infinitely many times with
  probability 1 in the forward pass.

19
How do we guarantee this?

• Either
• Independently sample a single outcome in each
  stage with a positive probability for each
  scenario outcome in the forward pass.
• Repeat an exhaustive enumeration of each scenario
  in the forward pass.

20
Convergence Theorem

• Under FPSP, DOASA converges with probability 1 to
  an optimal solution to the stage 1 problem in a
  finite number of iterations.

21
Sampling in cut calculation

• Sample some stage problems.
• Keep a list of dual solutions, search the best
  one for the stage problem that are not sampled.
• Backward Pass Sampling Property (BPSP) In any
  stage, each scenario outcome is visited
  infinitely many times with probability 1 in the
  backward pass.

22
Convergence Theorem

• Under FPSP and BPSP, the algorithm converges with
  probability 1 to an optimal solution to the stage
  1 problem in a finite number of iterations.

23
Corollaries

• If every outcome is used in cut calculation we
  only need FPSP.
• We can bias sampling as long as FPSP is
  satisfied. (Note estimation of upper bound needs
  unbiased scenarios.)

24
Resampling

• SDDP does not resample the forward pass. It
  creates N scenarios of inflows at the start.
• FPSP is NOT satisfied.
• SDDP will terminate with probability 1.
• Cuts give a lower bound, but policy need not be
  optimal.

25
Always Dry, when at convergence...
?
Dry
Dry
Wet
?
Dry
Wet
Wet
26
Negative inflows

• SDDP uses PARMA model for inflows.
• Negative inflows might result not physically
  possible.
• Some implementations adjust random outcomes to
  make inflow non-negative this destroys
  stage-wise independence.
• Cut sharing is no longer valid.
• Log-normal inflows not valid for convexity
  reasons.

27
Convexity matters in backward pass

• Transmission losses can make stage problem not
  convex if free disposal is not allowed.
• Unit commitment integer effects are not convex.

28
Convergence expectation

• We run DOASA on a problem at Fonterra NZ.
• Maximum size for convergence 12 stages x 24
  states.
• In revenue management application, 8 states,
  5000 stages converge, 20 states, 5000 stages
  does not.
• Convergence is problem dependent.

29
Case study NZ model
demand
N
S
demand
30
Computational results NZ model

• 9 reservoirs
• 52 weekly stages
• 30 inflow outcomes per stage
• Model written in AMPL/CPLEX
• Takes 100 iterations and 2 hours on a standard
  Windows PC to converge

31
2005-2006 policy simulated with historical inflow
sequences
32
END