Decision Analysis and Risk Analysis Applications

1
Decision Analysis and Risk Analysis Applications

• D. Warner North
• MSE 290
• Tuesday, January 25, 2005

2
New York Times - Sunday, August 11, 2002

• If the Big One Hits the Big Easy, the Good Times
  May Be Over Forever By ADAM COHEN (NYT) 1018
  words Late Edition - Final , Section 4 , Page 12
  , Column 1
• ABSTRACT - Adam Cohen Editorial Observer on grave
  threat that big hurricane could wipe out
  low-lying city of New Orleans describes
  complacency of its people and inadequate
  preparations by government Dot Wilson knows how
  bad the Big One could be. When Hurricane Betsy
  hit New Orleans in 1965 with 125 m.p.h. winds,
  leaving 75 dead in Louisiana and South Florida,
  she walked more than a mile in chest-high water,
  holding her infant daughter overhead. But when
  Ms. Wilson held a hurricane-preparedness teach-in
  recently at the community center she heads,
  attendance was sparse. ''A lot of people don't
  see storms as serious,'' she said with a sigh.
  But people who have been around for a while know
  better, she said, adding, ''We saw the bodies.''
• New Orleans -- home to the French Quarter's
  iron-latticed buildings and the Garden District's
  stately Greek Revival mansions, to Preservation
  Hall jazz and Mardi Gras parades -- may be
  America's most architecturally distinctive and
  culturally rich city. But it is also a disaster
  waiting to happen. New Orleans is the only major
  American city below sea level, and it is wedged
  between Lake Pontchartrain and the Mississippi.
  If a bad hurricane hit, experts say, the city
  could fill up like a cereal bowl, killing tens of
  thousands and laying waste to the city's
  architectural heritage. If the Big One hit, New
  Orleans could disappear.

3
New York Times, Thursday, October 3, 2002

• Thousands Seek Safety as Hurricane Nears Gulf
  Coast By JEFFREY GETTLEMAN (NYT) 1308 words
  Late Edition - Final , Section A , Page 24 ,
  Column 3
• ABSTRACT - About half million people flee
  southern Louisiana and Texas with approach of
  Hurricane Lili, which is heading toward Gulf
  Coast with winds of more than 140 miles per hour,
  making it daunting Category 4 storm map chart
  of five most intense hurricanes to hit US since
  1928 photos (M) Highways across southern
  Louisiana and Texas were solid columns of steel
  today as more than half a million people grabbed
  their valuables and fled their homes, looking for
  higher, safer ground before Hurricane Lili hit.
• The exodus of cars and trucks, some with
  furniture lashed down on their roofs, began in
  low-lying areas but quickly spread inland as the
  storm intensified and threatened to become the
  worst natural disaster here in decades.

4
Hurricane Lili
October 4, 2002 Using state-of-the-art
equipment, including a radar collaboratively
built by the NOAA National Severe Storms
Laboratory, research scientists captured
Hurricane Lili Thursday morning as she came
onshore along the southern Louisiana coast. Three
mobile Doppler radars, as well several
instrumented towers, were strategically placed
near Lafayette, La., in order to study the
structure of the rainfall and wind flow around
the storm. The data collected may help scientists
develop better estimates of rainfall amounts,
which could lead to more accurate and timely
forecasts of inland flooding in the future.
5
New York Times, Friday, October 4, 2002

• Hurricane Hits Gulf Coast, Weakened but Still
  Punishing By JEFFREY GETTLEMAN (NYT) 1052 words
  Late Edition - Final , Section A , Page 18 ,
  Column 1
• ABSTRACT - Hurricane Lili slams into Louisiana,
  but with winds sharply diminished as storm
  devolves from Category 4 to Category 2 by time it
  makes landfall still, there is widespread
  damage, with community of Abbeville being hardest
  hit map chart of Lili's wind speeds map photo
  (M) Hurricane Lili slammed into Louisiana today
  with tornado-force winds that ripped trees from
  the ground, smashed mobile homes and caused
  widespread blackouts that may last for weeks.
• But it could have been worse. The storm lost
  significant strength overnight, dwindling from a
  Category 4 hurricane to Category 2, and no deaths
  or serious injuries were reported.

6
The Decision to Seed Hurricanes
R.A. Howard, J.E. Matheson, D.W. North

• Science, 176, 1191-1202, 1972
• (paper on class website, pdf file)

7
Decision Analysis of Hurricane Modification

• Background
• The Cost of Hurricanes Average Annual Cost of
  Hurricane Damage 400 Million Hurricane Betsy,
  1965 1.4 Billion Hurricane Camille, 1969
  1.4 Billion
• The Hurricane Debbie Experiment,
  1969 Reductions of 31, 15 in Maximum Wind
  Speed Were Observed in the Two Seeding
  Experiments

8
Decision Analysis of Hurricane Modification

• Decisions
• Strategic Decision Should Operational Seeding
  by the U.S. Government be Permitted?
• Research Policy Decision Should the Present
  Level of Research and Experimentation be
  Changed?
• Tactical Operational Decision Should a
  Particular Hurricane be Seeded? (not analyzed in
  SRI study)

9
The Operational Seeding Decision Conceptual
Overview
Property Damage
Economical Model
Meteorological Knowledge
Hurricane Model
Decision Criteria
Storm Parameters
Critical Storm Characteristics Maximum
Sustained Wind
Legal/Social Model
Government Responsibility Cost
Seeding Decision Strategic
10
Maximum Sustained Winds Over Time
Maximum Sustained Winds, w
w(t) Without Seeding
w(t1)
w(t1)
w(t) With Seeding
w(t)
w(t0)
12 Hours
t0
t1
Seeding Initiated
Landfall
11
The Seeding Decision Decision Tree
Resolution of Uncertainty Change in Maximum
Sustained Surface Wind
Decision Alternatives
Consequences
Property Damage Government Responsibility
Seed
Do Not Seed
Property Damage Government Responsibility
12
Probabilistic Model for 12-Hour Change in Maximum
Sustained Wind, Natural Hurricane
Prob. Density Function

• No Predictability from Meteorological Environment
  Assumed.
• Probability Distribution Based on
• Observations of 12-Hour Changes in Central
  Pressure
• Empirical Formula Relating Central Pressure to
  Maximum Sustained Wind

time t 0 (seeding initiated)
40
70
100
130
160
Maximum Sustained Wind, mph (or )
Prob. Density Function
time t 12 hours (landfall)
? 15.6
40
70
100
130
160
Maximum Sustained Wind, mph (or )
13
Probabilistic Model for 12-Hour Change in Maximum
Sustained Wind, Natural Hurricane
Probabilities Assigned to Hypotheses
m 85 ? 18.6
H1 Stormfury Hypothesis Seeding Causes an
Average Reduction in maximum Sustained Wind

• Probability Distribution Based on
• Probability Distribution for 12-Hour Change in
  Natural Hurricane
• Expert Judgment on Average Effect of Seeding
• Expert Judgment on Fluctuations From Average
  Effect in Seeding a Particular Storm

0.49
m 100 ? 15.6
0.49
H2 Null Hypothesis Seeding Has No Effect on
Maxium Sustained Winds
0.02
m 110 ? 18.6
H3 Pessimistic Hypothesis Seeding Causes an
Average Increase in Maximum Sustained wind
40
70
100
130
160
Maximum Sustained Wind, mph (or )
14
Probabilities for the Three Models

• Judgments from Hurricane Experts
• Before Debbie, P(H1) gt P(H3)
• After Debbie, P(H1) P(H2)
• Resulting Probabilities
• Before the Hurricane Debbie Seeding
• P(H1) 0.15 P(H2) 0.75 P(H3) 0.10
• After Hurricane Debbie Seeding
• P(H1) 0.49 P(H2) 0.49 P(H3) 0.02
• Before and After Probabilities Related by Bayes
  Rule

H1 Stormfury Hypothesis Seeding Causes an
Average Reduction in maximum Sustained Wind
H2 Null Hypothesis Seeding Has No Effect on
Maximum Sustained Winds
H3 Pessimistic Hypothesis Seeding Causes an
Average Increase in Maximum Sustained wind
15
Summary of Current Meteorological Information
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
Unseeded Hurricane
Probability the Wind Speed is Greater Than
Amount Shown
Seeded Hurricane
60
70
80
90
100
110
120
130
Maximum Sustained Wind 12 Hours After Seeding
Decision ()
16
Maximum Sustained Wind Versus Property Damage
100 x 106
Total Equivalent Residential Property Damage
(1969 )
10 x 106
d c1wc2
c2 4.363
1 x 106
50
60
70
80
90
100
110
120
130
140
150
160
180
200
w, Maximum Sustained Surface wind Speed (mph)
17
Probability Distributions on Property Damage for
the Seeded and Unseeded Hurricane
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
Unseeded
Probability That Property Damage is Greater than
Amount Shown
Seeded
0
50
100
150
200
250
300
350
400
Property Damage (M)
18
The Seeding Decision for the Nominal Hurricane
Resolution of Uncertainty
Change in Maximum Sustained Wind
Property Damage Loss (M)
Probabilities Assigned to Outcomes
32 16 0 -16 -34
335.8 191.1 100.0 46.7 16.3
0.038
0.143
Seed Expected Loss
0.392
94.08 0.25 94.33
0.255
0.172
Cost of Seeding 0.25
32 16 0 -16 -34
335.8 191.1 100.0 46.7 16.3
0.054
Do Not Seed Expected Loss
0.206
116.00
0.480
0.206
0.054
Expected Value (M)
Economic Value of Seeding 21.67 million (18.7
reduction in loss) Decision not sensitive to
specific assumptions.
19
Government Responsibility Costs
When a Hurricane Labeled U.S. Government
Seeded it is No longer a Natural Disaster

• If a Seeded
• Hurricane
• Intensifies

• Public Outrage? Lawsuits?

Concept What Increment of Property Damage
Reduction Justifies the Assumption of
Responsibility Entailed by Seeding a Hurricane?
20
Government Responsibility Cost Assumed for
Strategic Analysis(100 Million Storm)
21
The Seeding Decision for the Nominal Hurricane
(Government Responsibility Cost Included)
Government Responsibility Cost ( of property
damage)
Change in Maximum Sustained Wind
Property Damage Loss (M)
Probabilities Assigned to Outcomes
Total Cost (M)
32 16 0 -16 -34
335.8 191.1 100.0 46.7 16.3
503.7 248.4 105.0 46.7 16.3
50 30 5 0 0
0.038
0.142
Seed Expected Loss
0.392
110.67 0.25 110.92
0.255
0.172
Cost of Seeding 0.25
32 16 0 -16 -34
335.8 191.1 100.0 46.7 16.3
335.8 191.1 100.0 46.7 16.3
- - - - -
0.054
Do Not Seed Expected Loss
0.206
116.00
0.480
0.206
0.054
Expected Value (M)
Value of Seeding 5.08 million (4.4 reduction)
22
Legal findings No firm Legal Basis for
Operational Seeding Currently Appears to Exist

• Sovereign ImmunityIn conclusion, existing
  immunity law provides only partial and
  unpredictable protection at best. There are also
  grounds for recognizing that immunity defenses
  may be avoided in most cases if the plaintiff
  carefully chooses his remedy, his legal theory,
  and his forum. Only specific Congressional
  action offers a prospect of substantial,
  predictable immunity protection.
• - Appendix D, p. 9
• Basis for LawsuitsThe common law and inverse
  condemnation theories appear to offer plaintiff's
  attorneys substantial grounds for recovering
  damages where they can prove that modification
  activities caused injury, death, or property
  damage. This is particularly significant in the
  light of recent procedural developments
  (especially the decline of immunity defenses as
  discussed in an earlier memorandum). These
  tentative results suggest that the project agency
  may wish specific Congressional authorization for
  its project.
• - Appendix E, p. 15

23
The Value of an Expanded Research and
Experimentation Program

• Limiting Case
• What is the Value of Perfect Information
• Concept
• How Much Should the Government be Willing to Pay
  a Clairvoyant to Learn Which of the Three
  Hypotheses,
• H1 H2 or H3
• Is Actually True Before making the Operational
  Seeding Decision For a Single Hurricane?

24
Expected Value of the Clairvoyants Information
Which Hypothesis Describes the Effect of Seeding?
Property Damage Loss (M)
Seeding Decision
Choice of Whether to Gather Information
Outcomes
Results of Information
32 16 0 -16 -34 32 16 0 -16 -34
32 16 0 -16 -34 32 16 0 -16 -34 3
2 16 0 -16 -34 32 16 0 -16 -34 32
16 0 -16 -34 32 16 0 -16 -34
335.8 191.1 100.0 46.7 16.3 335.8 191.1 1
00.0 46.7 16.3 335.8 191.1 100.0 46.7 16.3
335.8 191.1 100.0 46.7 16.3 335.8 191.1 10
0.0 46.7 16.3 335.8 191.1 100.0 46.7 16.3
335.8 191.1 100.0 46.7 16.3 335.8 191.1 100
.0 46.7 16.3
69.42
H1 True
Seed
Do Not Seed
69.42
116.00
0.49
116.25
H2 True
Seed
93.17
0.49
Do Not Seed
0.02
116.00
Obtain Information
116.00
167.61
93.17
H3 True
Seed
Do Not Seed
116.00
94.33
116.00
Do not Obtain Information
94.33
Seed
Expected Value (M)
Do Not Seed
Value 1.16 property damage only Value 13.63
property damage plus government responsibility
cost. Value is Higher if Seeding is not Permitted
with Present Information.
94.33
116.00
25
Value of A Seeding Experiment(Government
Responsibility Cost Included)
Total Cost (millions of dollars)
Outcomes
Operational Seeding Decision
Results of Experiment
Choice of Whether to Perform Experiment
116.00
0.038
0.143
32 16 0 -16 -34 32 16 0 -16 -34
32 16 0 -16 -34 32 16 0 -16 -
34
503.7 248.4 105.0 46.7 16.3 503.7 248.4 105.0
46.7 16.3 503.7 248.4 105.0 46.7 16.3 503.7 248.
4 105.0 46.7 16.3
116.00
0.392
103.44
Perform Experiment
116.00
0.255
Seed
107.96
Do Not Seed
103.44
0.172
116.00
87.83
110.92
110.92
Do not Perform Experiment
Seed
Do Not Seed
110.92
Expected Value (M)
116.00
Value 2.96 million Actual cost 0.25 million
26
Summary of the Value of Additional Information on
the Effect of Seeding (Values in Millions of
Dollars)
Considering only the 50 of hurricanes that are
assumed to be possible candidates for seeding
because of tactical consideration. If all
hurricanes are assumed to be candidates for
operational seeding, the figures of the last two
columns should be doubled.
27
Findings and Recommendations

• Findings
• 1. Current meteorological and economic
  information indicates that the seeding
  alternative stochastically dominated the
  non-seeding alternative
• 2. No firm legal basis for operational seeding
  appears to exist
• 3. The decision to seed a particular hurricane
  should take into account its specific
  characteristics
• 4. Resolving meteorological uncertainty on the
  effect of hurricane modification is worth over
  20 millions/year

• Recommendations
• 1. The present policy prohibiting seeding any
  hurricane threatening the U.S. should be
  rescinded
• 2. A hurricane modification agency with authority
  to seed operationally should be established
• 3. Decision procedures supported by further
  analysis should be developed
• 4. Modification experiments should be conducted
  on an expanded scale to provide a more refined
  basis for making each operational seeding decision

28
http//www.csmonitor.com/2003/0102/p10s02-sten.htm
l

• Tinkering with clouds
• Researchers say evolving technologies could allow
  manipulation of major weather patterns. But
  should humans tamper?
• By Peter N. Spotts Staff writer of The
  Christian Science Monitor, Jan 2, 2003
• On Sept. 11, 1992, hurricane Iniki slammed into
  the Hawaiian island of Kauai, packing winds
  gusting up to 175 m.p.h.
• The storm inflicted an estimated 2 billion in
  damage and 105 casualties, damaged or destroyed
  10,000 homes and businesses, and left once-lush
  tropical mountainsides looking as though they'd
  been mowed by a giant weed-whacker

29
http//www.csmonitor.com/2003/0102/p10s02-sten.htm
l

• Tinkering with clouds (2)
• Over the past two decades, the idea of modifying
  large-scale storms such as hurricanes has lain
  dormant, following 20 years of inconclusive
  research. Now, however, a small group of
  atmospheric scientists is giving the concept a
  fresh look.
• Researchers seeded hurricanes in a 20-year
  federal research project dubbed Project Storm
  Fury. Scientists were testing the idea that
  seeding could be used to take some of the punch
  out of hurricanes before they made landfall. But
  the program foundered on inconclusive results.

30
http//www.csmonitor.com/2003/0102/p10s02-sten.htm
l

• Tinkering with clouds (3)
• during the Vietnam War, the US military seeded
  monsoon clouds in Operation Popeye in an attempt
  to use weather to hamper troop and supply
  movements along the Ho Chi Minh Trail. When
  information about the program was declassified in
  the mid-1970s, the international community
  established the UN Convention on the Prohibition
  of Military or Any Other Hostile Use of
  Environmental Modification Techniques.
• Federal funds for weather-modification research
  have dried up as well. According to Colorado
  State University atmospheric scientist William
  Cotton, federal dollars for weather modification
  research peaked at roughly 19 million a year in
  the 1970s. They dropped to less than 5 million a
  year during the '90s, and now hover at about
  500,000.
• The field has entered what Dr. Cotton calls the
  "dark ages," where weather-modification programs
  are forging ahead with little or no scientific
  research programs to back them.

31
New Methodology for Assessing the Probability of
Contaminating Mars

• D. W. North, B.R. Judd, and J.P. Pezier, Life
  Sciences and Space Research XIII, P. A. Sneath,
  ed., Akademie-Verlag, Berlin, pp. 103-109, 1975.
• Limitations, Definitions, Principles, and
  Methods of Risk Analysis, Risk Assessment for
  Veterinary Biologicals, special issue, Office
  International des Epizooties, Scientific and
  Technical Review, Vol. 14, pp. 913-923, 1995.
  (on class website)

32
Assessment of the Probability of Contaminating
Mars

• Carried out during 1972-3 for NASA Headquarters,
  Office of Planetary Programs, NASA Contracts
  2451, 2535
• Background Committee on Space Research (COSPAR)
  Resolution
• Total probability of contaminating Mars
  during the quarantine period shall be less than
  10-3.
• Mariner 9 photos evidence of past liquid
  water
• Difficulties of dry-heat sterilization for 1975
  Viking Lander

33
Mission Contamination Model
34
Bio-burden Submodel

• Input Elements
• Pre-sterilization burden by location
• Sensitivity to sterilization
• Sterilization regime
• Recontamination
• Inflight mortality or proliferation
• Contamination and subsequent
  amplification in biology experiment
• (probability 10-6)
• Outputs
• Bio-burden estimates by Viking Project
• Expected number of Viable Terrestrial
• Organisms (VTOs), by location on
• spacecraft

35
Release Submodel

• Input Elements
• Bioburden by location
• (from bio-burden submodel)
• Probability of hard landing
• Fracture ratio for hard landing
• Lethality for release, given location and
  landing
• (hard versus nominal)
• Outputs
• Implantation microbes (VTOs) directly
• deposited in Martian soil, without UV
  exposure
• Erosion VTOs relaeased through aeolian
• erosion of spacecraft into Martian
  atmosphere
• Vibration VTOs fall from spacecraft onto
• surface of Mars due to mechanical
  vibration,
• thermal effects, etc. VTOs require
  shielding to
• survive UV exposure.

36
Transport Submodel

• Input Elements
• Expected number of VTOs released, by mechanism
• Lethality of UV radiation, in normal atmosphere
  and dust storm (probability that a VTO survives
  transit)
• Extent of usable water
• Probability it exists anywhere
• Portion of surface covered
• Output
• Expected number of VTOs that will reach usable
  water

37
Reproduction Submodel

• Input Elements
• Fraction of VTOs that are facultatively anaerobic
  and psychrophilic (0.05)
• Probability that nutrients needed for
  reproduction will be present in the water
  microenvironment (0.10)
• Output
• Expected number of VTOs that reproduce at least
  once, defined as contamination

38
Mission Contamination Model Results
39
Mission Contamination Model Marginal Sensitivity
Analysis
Probability of Contamination Probability of Contamination Probability of Contamination
Contamination Model Variables Values Units 10-6 Units 10-6
Extreme Intermed. Intermed. Extreme Nominal 5.9 Nominal 5.9
Low Low NOMINAL High High Low High
Bio-Burden Variables
1. bio External 2.2 5.5 11 22 55 5 10.7
2. bio Covered 3.2 8 16 32 80 3.1 20.2
3. bio Encapsulated 4,000 10,000 20,000 40,000 100,000 5 10.4

Release Variables
1. rel Hard Landing Probability 0.0004 0.001 0.002 0.004 0.01 5.2 9.6
3. rel Newly Exposed/Hard, Encaps 0.0001 0.0002 0.001 0.005 0.01 5.4 10.9
4. rel Implanted, Soft 0.0001 0.0002 0.001 0.005 0.01 5.7 8.7
6. rel VTO/Vibration 0.001 0.002 0.01 0.05 0.01 5.4 11.1
9. rel VTO/Erosion, Encaps 0.00001 0.00002 0.0001 0.0005 0.001 5.4 10.9
40
Mission Contamination Model Marginal Sensitivity
Analysis -2
Probability of Contamination Probability of Contamination Probability of Contamination
Contamination Model Variables Values Units 10-6 Units 10-6
Extreme Intermed. Intermed. Extreme Nominal 5.9 Nominal 5.9
Low Low NOMINAL High High Low High
Transport Variables
1 tra Survive Transit 0.001 0.002 0.01 0.05 0.1 2.2 45.2
2 tra Find Water 0.0005 0.001 0.005 0.025 0.05 1.5 49.9
4 tra Water Deposition 0.00005 0.0001 0.0005 0.0025 0.005 5 15.2
5 tra Stay Lodged 0.1 0.2 0.5 0.8 0.9 5.5 10

Reproduction Variables
1 rep Pyschrophic, Anaerobic 0.005 0.01 0.05 0.1 0.25 0.6 29.6
2 rep Availability of Nutrients 0.01 0.02 0.1 0.2 0.5 0.6 29.6
41
National Academy of Sciences, Viewpoint -1992

• it is the unanimous opinion of the task
  group that terrestrial organisms have almost no
  chance of multiplying on he surface of Mars and
  in fact have little chance of surviving for long
  periods of time, especially if they are exposed
  to wind and to UV radiation.
• ---- Space Studies Board, National Research
  Council, Biological Contamination of Mars,
  1992, page 49.

42
Retrospective

• Impact of our analysis
• Acceptance of mission risk by scientific leaders
• NASAs decision to eliminate the mid-course
  correction on the Mars Orbiter
• Planetary Quarantine since Viking
• Not a major concern, perhaps excepting Mars
  sample return
• Example of Quantitative Risk Analysis built on
  highly judgmental information