OSiL: An XML-based schema for stochastic programs

1
OSiL An XML-based schema for stochastic programs

• H.I. Gassmann, R. Fourer, J. Ma, R.K. Martin

SP XI Vienna, August 2007
2
Outline

• Motivation and review
• A four-stage investment problem
• OSiL format
• Conclusions and future work

3
Why a standard?

• Benchmarking
• Archiving
• Algorithm development
• Distributed computing
• Sharing of problem instances

4
Why XML?

• Existing parsers to check syntax
• Easy to generate automatically
• Tree structure naturally mirrors expression trees
  for nonlinear functions
• Arbitrary precision and name space
• Automatic attribute checking (e.g.,
  nonnegativity)
• Querying capabilities via XQuery
• Encryption standards being developed
• Easy integration into broader IT infrastructure

5
Stochastic programs

Any data item with nonzero subscript may be
random (including dimensions where mathematically
sensible) stands for arbitrary relations (?, ,
?)
D means with probability 1 or with
probability at least b or with expected
violation at most v or
6
Problem classes and time domain

• Single-stage problems
• Mean-variance problems (Markowitz)
• Robust optimization
• Chance-constrained problems
• Reformulated and solved as deterministic
  nonlinear problems

7
Problem classes and time domain (contd)

• Two-stage problems with recourse
• Solved by
• Deterministic equivalent methods
• Benders decomposition
• Stochastic quasigradient methods
• Stochastic decomposition (Higle and Sen)
• Monte Carlo sampling (Shapiro and Homem-de-Mello)
• Regression approximation (Deák)
• Distributions
• Known
• Approximated (by scenario generation)
• Partially known (moments, distribution type,
  support, etc.)

8
Problem classes and time domain (contd)

• Multi-stage recourse models
• Deterministic equivalent
• Nested Benders decomposition
• Progressive hedging
• Multistage stochastic decomposition
• Probabilistic constraints and risk measures can
  be added as linking constraints

9
Problem classes and time domain (contd)

• Horizon problems (Grinold, Sethi)
• Markov reward processes
• Continuous time problems

10
Example (Birge)
I 2, T 3, B 55, R 80, at1 1.25,
1.06, at2 1.14, 1.12
11
Markup languages

• Intersperse text (data) and information about it
  (formatting, etc.)
• Examples
• TeX (extensible through user \def)
• HTML
• VRML
• XML

12
OSiL Schema

• Written in XML
• Very flexible
• Intended to handle as many types of mathematical
  programs as possible
• Linear and integer
• Nonlinear
• Stochastic

13
OSiL Schema Header information
14
Header information Example

• lt?xmlversion"1.0"encoding"UTF8"?gt
• ltosil xmlns"os.optimizationservices.org
• xmlnsxsihttp//www.w3.org/2001/XMLSchema
  instance
• xsischemaLocation"OSiL.xsd"gt
• ltinstanceHeadergt
• ltnamegtFinancialPlan_JohnBirgelt/namegt
• ltsourcegt
• Birge and Louveaux, Stochastic
  Programming
• lt/sourcegt
• ltdescriptiongt
• Three-stage stochastic investment
  problem
• lt/descriptiongt
• lt/instanceHeader gt
• ltinstanceDatagt
• ...
• lt/instanceDatagt
• lt/osilgt

15
OSiL Schema Deterministic data
16
Instance data
Variables, objectives, constraints

• ltvariables numberOfVariables"8"gt
• ltvar name"invest01" type"C" lb"0.0"/gt
• ltvar name"invest02"/gt
• ltvar name"invest11"/gt
• ltvar name"invest12"/gt
• ltvar name"invest21"/gt
• ltvar name"invest22"/gt
• ltvar name"w"/gt
• ltvar name"u"/gt
• lt/variablesgt
• ltobjectives numberOfObjectives"1"gt
• ltobj maxOrMin"max" numberOfObjCoef "2"
  lb"0.0"gt
• ltcoef idx"6"/gt1.lt/coefgt
• ltcoef idx"7"/gt-4.lt/coefgt
• lt/objgt
• lt/objectivesgt
• ltconstraints numberOfConstraints"4"gt
• ltcon name"budget0" lb"55" ub"55"/gt
• ltcon name"budget1" lb"0" ub"0"/gt

17
Instance data Core matrix (sparse
matrix form)

• ltlinearConstraintCoefficients
• numberOfValues14"gt
• ltstartgt
• ltelgt0lt/elgt
• ltelgt2lt/elgt
• ltelgt4lt/elgt
• ltelgt6lt/elgt
• ltelgt8lt/elgt
• ltelgt10lt/elgt
• ltelgt12lt/elgt
• ltelgt13lt/elgt
• ltelgt14lt/elgt
• lt/startgt

ltvaluegt ltelgt1lt/elgt ltelgt1.25lt/elgt
ltelgt1lt/elgt ltelgt1.14lt/elgt
ltelgt1lt/elgt ltelgt1.25lt/elgt ltelgt1lt/elgt
ltelgt1.14lt/elgt ltelgt1lt/elgt
ltelgt1.25lt/elgt ltelgt1lt/elgt
ltelgt1.14lt/elgt ltelgt1lt/elgt ltelgt-1lt/elgt
lt/valuegt
ltrowIdxgt ltelgt0lt/elgt ltelgt1lt/elgt
ltelgt0lt/elgt ltelgt1lt/elgt ltelgt1lt/elgt
ltelgt2lt/elgt ltelgt1lt/elgt ltelgt2lt/elgt
ltelgt2lt/elgt ltelgt3lt/elgt ltelgt2lt/elgt
ltelgt3lt/elgt ltelgt3lt/elgt ltelgt3lt/elgt lt/rowIdxgt
18
OSiL Schema Dynamic structure
19
Dynamic information Example

• ltstages numberOfStages"4"gt
• ltstagegt
• ltvariables numberOfVariables"2"
  startIdx"0" endIdx"1"/gt
• ltconstraints numberOfConstraints"1"
  startIdx"0"/gt
• lt/stagegt
• ltstagegt
• ltvariables numberOfVariables"2"
  startIdx"2" endIdx"3"/gt
• ltconstraints numberOfConstraints"1"
  startIdx"1"/gt
• lt/stagegt
• ltstagegt
• ltvariables numberOfVariables"2"
  startIdx"4" endIdx"5"/gt
• ltconstraints numberOfConstraints"1"
  startIdx"2"/gt
• lt/stagegt
• ltstagegt
• ltvariables numberOfVariables"2"gt
• ltvar idx"6"gt ltvar idx"7"gt
• lt/variablesgt
• ltconstraints numberOfConstraints"1"
  startIdx"3"/gt
• lt/stagegt

20
Explicit and implicit event trees
21
OSiL Schema Scenario trees
22
Scenario tree Example

• ltstochasticInformation decisionEvenSequence"Deci
  sionAfterEvent"gt
• lteventTreegt
• ltscenarioTree numberOfScenarios"8"gt
• ltrootScenario prob"0.125"
  stage"0"/gt
• ltscenario prob"0.125" stage"3"
  parent"0"gt
• ltlinearConstraintCoefficientsgt
• ltel rowIdx"3"
  colIdx"4"gt1.06lt/elgt
• ltel rowIdx"3"
  colIdx"5"gt1.12lt/elgt
• lt/linearConstraintCoefficientsgt
• lt/scenariogt
• ltscenario prob"0.125" stage"2"
  parent"0"gt

23
Node-by-node representation for stochastic
problem dimensions
24
Node-by-node Example

• ltstochasticInformation decisionEvenSequenc
  e"DecisionAfterEvent"gt
• lteventTree gt
• ltnodalTreegt
• ltsNode prob"1" base"coreProgram"gt
• ltsNode prob"0.5"
  base"coreProgram"gt
• ltsNode prob"0.5"
  base"coreProgram"gt
• ltsNode prob"0.5"
  base"coreProgram"/gt
• ltsNode prob"0.5"
  base"firstSibling"gt
• ltchangesgt
• ltel rowIdx3"
  colIdx"4"gt1.06lt/elgt
• ltel rowIdx"3"
  colIdx"5"gt1.12lt/elgt
• lt/changesgt
• lt/sNodegt
• lt/sNodegt

25
Distributions (implicit tree)
26
Discrete random vector

• ltdistributionsgt
• ltmultivariategt
• ltdistr stage"1"gt
• ltmultiDimensionalDistributionGroupgt
• ltmultivariateDiscretegt
• ltscenariogt
• ltprobgt0.5lt/probgt
• ltelgt1.25lt/elgt
• ltelgt1.14lt/elgt
• lt/scenariogt
• ltscenariogt
• ltprobgt0.5lt/probgt
• ltelgt1.06lt/elgt
• ltelgt1.12lt/elgt
• lt/scenariogt
• lt/multivariateDiscretegt
• lt/multiDimensionalDistributionGroupgt
• lt/distrgt

27
General distribution (incomplete information)
28
Transformations

• Random variables separated from model entities
• Linked to stochastic problem elements by
  transformations (linear or nonlinear)
• Useful for factor models and other stochastic
  processes

29
OSiL schema Linear transformations
30
Linear transformation Example

• ltstochasticTransformationgt
• ltlinearTransformationgt
• ltstochasticElements
• numberOfElements"6"gt
• ltel rowIdx"1" colIdx"0"/gt
• ltel rowIdx"1" colIdx"1"/gt
• ltel rowIdx"2" colIdx"2"/gt
• ltel rowIdx"2" colIdx"3"/gt
• ltel rowIdx"3" colIdx"4"/gt
• ltel rowIdx"3" colIdx"5"/gt
• lt/stochasticElementsgt
• ltmatrixCoefficients
• numberOfElements"6"gt
• ltstartgt
• ltelgt0lt/elgt
• ltelgt1lt/elgt
• ltelgt2lt/elgt
• ltelgt3lt/elgt
• ltelgt4lt/elgt

ltrowIdxgt ltelgt0lt/elgt
ltelgt1lt/elgt
ltelgt2lt/elgt ltelgt3lt/elgt
ltelgt4lt/elgt ltelgt1lt/elgt
lt/rowIdxgt ltvaluegt
ltel mult"6"gt1.0lt/elgt lt/valuegt
lt/matrixCoefficientsgt lt/linearTransformationgt
lt/stochasticTransformationgt
31
Penalties and probabilistic constraints
32
Nonlinear expression (2x0
x12)2 (1 x0)2

• ltplusgt
• ltpowergt
• ltminusgt
• ltvar coef"2.0" idx"0"/gt
• ltpowergt
• ltvar coef"1.0" idx"1"/gt
• ltnumber value"2"/gt
• lt/powergt
• lt/minusgt
• ltnumber value"2"/gt
• lt/powergt
• ltpowergt
• ltminusgt
• ltnumber value"1"/gt
• ltvar coef"1.0" idx"0"/gt
• lt/minusgt
• ltnumber value"2"/gt
• lt/powergt
• lt/plusgt

33
Capabilities

• Arbitrary nonlinear expressions
• Arbitrary distributions
• Scenario trees
• Stochastic problem dimensions
• Simple recourse
• Soft constraints with arbitrary penalties
• Probabilistic constraints
• Arbitrary moment constraints

34
Further work

• Internal data structures (OSInstance)
• OSiLWriter SMPS ? OSiL
• Library of problems (netlib, POSTS, Ariyawansa
  and Felt, Watson, SIPLib,)
• Readers
• Solver interfaces
• Buy-in

35
QUESTIONS?
http//www.optimizationservices.org http//myweb.d
al.ca/gassmann