VI' Design of Experiments

1
VI. Design of Experiments

• M. Peter Jurkat
• CS452/Mgt532 Simulation for Managerial Decisions
• The Robert O. Anderson Schools of Management
• University of New Mexico

2
Simulation Study Representation(after Banks et
al, Figure 1.3, Page 15)
Set Objectives and Project Plan
Problem Formulation
(Re)Conceptualize Model Collect Data
Yes
No
Translate Model
Can Model be Verified?
No
Can Model be Validated?
Yes
DOE - Design Experiments
Runs and Replications
Analysis
No
Results Clear and Able to be Described?
Document, Report and Recommend
Yes
3
Full Experimental Process

• Four steps screen, model, optimize, verify
• Screen find significant few decision variables
  (those that can be set by experimenter) among all
  IVs 2 level
• Model add 3rd/4th/ level to determine if
  non-linear effects significant
• Optimize solve equations and/or search results
  in best values of decision variables
• Verify local optima first then global optimum
  among the local ones
• Culmination of simulation study?

4
Input Variables

• Identifiable, named quantities whose values can
  change - values also called levels
• Factors/decision variables levels can be set by
  experimenter - often considered the causing
  variables (e.g., reorder point levels 20, 30,
  40)
• Random variables cannot be set exactly, only by
  specifying their distribution (e.g., order
  delivery time levels U.25, 1.25)
• Uncontrollable variables important but cannot be
  set by experimenter (e.g., ambient temperature)
  these often randomized across runs (e.g.,
  shortage cost)
• For documentation see Banks et al, Figure 10.5
  Table 10.1, p367

5
Graphical RepresentationBlack Box

• BCNN 4th Ed., Figure 10.5, page 367

6
Optimization

• Optimization find values of some of the IVs
  (important factors/decision variables) yielding
  optimal/desired value of one or more DV(s)
• Often IVs are random variables (RVs) gt DVs are
  also random strict optimization not practical
• Optimize statistical measures of the DV
  distribution max/min, mean, variance,
  probability of a given value,
• Techniques exhaustive evaluation, ridge
  climbing, robust heuristics (LP/IP, GA, ),

7
Optimization Concerns

• Except for analytical technique (solve zero
  derivative equations) optimization is done by
  searching in the IV space
• Solver procedure in Excel and Optimizing in GPSS
  World automate search
• Often many IVs gt search in high dimension space
  long, difficult, high probability of missing
  optimum often fails
• Best approach reduce IV space to that defined by
  significant few IVs and search in a space of
  lower dimension - can be done efficiently by
  screening techniques

8
DOE Terminologysee also Kelton Barton (2003)
Experimental Design for Simulation, Proceedings
of the 2003 Winter Simulation Conference

• Trial one simulation execution at one
  combination of input levels - multiple trials at
  given combination are replications
• Run in DOE all trials/replications at a given
  combination of decision variables levels (in
  simulation run is often synonymous with a trial)
  randomize other variables for best control
• Experimental Design set of planned runs
• Example yields of a bulk chemical mill at all
  combinations (known as full factorial design) of
  three levels of temperature and four levels of
  pressure - 12 runs - if replicated 3 times
  results in 36 trials in all

9
Start with Two Level Designs

• Experiments which include many (all?) decision
  variables at only two levels coded as - and
• Useful for screening designs to determine which
  variables have significant effects - how
  selected?
• often obvious
• else possibly at 20 and 80 of full range
• Example yield, y, of pilot synthetic yarn plant
  from all combinations of three decision variables
  (Box, Hunter Hunter, 309-19)
• T temperature at 160o, 180o
• C concentration at 20, 40
• K catalyst A or B

10
Example Experimental Results for Two Level Full
Factorial Design
11
Main (Average) Effect of Temperature(Average
change in strength by going from low temperature
to high over all combinations of other factors)
12
Response Representation

• Visualize 3 factors on coordinate axes in 3-space
• Assign response value, y, to corners
• Example result of T, C-, and K is 83 placed at
  far lower right corner
• Visualization Figures from Box, Hunter, Hunter,
  pp 310 312

13
Main Effects Visualization

• Average four corners for high () values and then
  low (-) values and then subtract

14
Interaction Effect of Temperature and Catalyst
15
Interaction Measure over all IVs
16
2-Way Interaction Representation

• Average corners of diagonal planes in direction
  of third factor and subtract

17
Deconstruction of Interaction Effect
18
Sign Convention for Effect Calculation

• Sign patter for main effect is sign pattern of
  high/low values
• Sign pattern of TxK column is product of the
  signs in the T and K columns

19
3-Way Interaction Representation

• Average corners of inscribed regular tetrahedrons
  of diagonals and subtract
• One tetrahedron should include the (-,-,-) corner
  and the other should include the (,,) corner

20
Three Factor Full Factorial Design
21
ScreeningTwo Level Full Factorial

• All two level combinations of all factors
  balanced orthogonal get all interactions
  (need?) - run numbers restricted to powers of 2
  inefficient, expensive for many factors (need
  replications also)

22
Screening Analysis

• See FULFACT8.XLS spreadsheet for factor full
  factorial model set up and analysis
• Use the Screening procedure in GPSS World to
  perform a full factorial analysis of the (M, L,
  N) inventory simulation for a three factor design
  use the GPSS World Screening capability Edit
  -gt Insert Experiment -gt Screening get ANOVA
  table w/o p-values check critical region

23
Fractional Factorial DesignsSchmidt Launsby
(1994) Box, Hunter, and Hunter (1978) Taguchi
and Konishi (1987)

• For full factorial designs
• each additional factor raises the number of runs
  by the number of levels times all previous runs
• Design matrix is large due to interactions
  however, 4th, 5th and higher order interactions
  seldom significant and very hard to understand
  even 3rd order interaction often small
• Use fractional factorial designs to reduce runs -
  see references
• However, with fast computers and small models can
  usually use full factorials 2nd and 3rd order
  interactions useful for interaction terms in
  models