Decision Analysis-1

1
IndE 311Stochastic Models and Decision Analysis

• UW Industrial Engineering
• Instructor Prof. Zelda Zabinsky

2
Operations Research The Science of Better
3
Operations Research Modeling Toolset
311
Queueing Theory
310
Markov Chains
PERT/ CPM
Network Programming
Simulation
Decision Analysis
Stochastic Programming
Linear Programming
Inventory Theory
Markov Decision Processes
Nonlinear Programming
Dynamic Programming
Forecasting
Integer Programming
Game Theory
312
4
IndE 311

• Decision analysis
• Decision making without experimentation
• Decision making with experimentation
• Decision trees
• Utility theory
• Markov chains
• Modeling
• Chapman-Kolmogorov equations
• Classification of states
• Long-run properties
• First passage times
• Absorbing states

• Queueing theory
• Basic structure and modeling
• Exponential distribution
• Birth-and-death processes
• Models based on birth-and-death
• Models with non-exponential distributions
• Applications of queueing theory
• Waiting cost functions
• Decision models

5
Decision Analysis

• Chapter 15

6
Decision Analysis

• Decision making without experimentation
• Decision making criteria
• Decision making with experimentation
• Expected value of experimentation
• Decision trees
• Utility theory

7
Decision Making without Experimentation
8
Goferbroke Example

• Goferbroke Company owns a tract of land that may
  contain oil
• Consulting geologist 1 chance in 4 of oil
• Offer for purchase from another company 90k
• Can also hold the land and drill for oil with
  cost 100k
• If oil, expected revenue 800k, if not, nothing

Payoff Payoff
Alternative Oil Dry
Drill for oil
Sell the land
Chance 1 in 4 3 in 4
9
Notation and Terminology

• Actions a1, a2,
• The set of actions the decision maker must choose
  from
• Example
• States of nature ?1, ?2, ...
• Possible outcomes of the uncertain event.
• Example

10
Notation and Terminology

• Payoff/Loss Function L(ai, ?k)
• The payoff/loss incurred by taking action ai when
  state ?k occurs.
• Example
• Prior distribution
• Distribution representing the relative likelihood
  of the possible states of nature.
• Prior probabilities P(? ?k)
• Probabilities (provided by prior distribution)
  for various states of nature.
• Example

11
Decision Making Criteria

• Can optimize the decision with respect to
  several criteria
• Maximin payoff
• Minimax regret
• Maximum likelihood
• Bayes decision rule (expected value)

12
Maximin Payoff Criterion

• For each action, find minimum payoff over all
  states of nature
• Then choose the action with the maximum of these
  minimum payoffs

State of Nature State of Nature Min Payoff
Action Oil Dry Min Payoff
Drill for oil 700 -100
Sell the land 90 90
13
Minimax Regret Criterion

• For each action, find maximum regret over all
  states of nature
• Then choose the action with the minimum of these
  maximum regrets

(Payoffs) State of Nature State of Nature
Action Oil Dry
Drill for oil 700 -100
Sell the land 90 90
(Regrets) State of Nature State of Nature Max Regret
Action Oil Dry Max Regret
Drill for oil
Sell the land
14
Maximum Likelihood Criterion

• Identify the most likely state of nature
• Then choose the action with the maximum payoff
  under that state of nature

State of Nature State of Nature
Action Oil Dry
Drill for oil 700 -100
Sell the land 90 90
Prior probability 0.25 0.75
15
Bayes Decision Rule(Expected Value Criterion)

• For each action, find expectation of payoff over
  all states of nature
• Then choose the action with the maximum of these
  expected payoffs

State of Nature State of Nature Expected Payoff
Action Oil Dry Expected Payoff
Drill for oil 700 -100
Sell the land 90 90
Prior probability 0.25 0.75
16
Sensitivity Analysis with Bayes Decision Rule

• What is the minimum probability of oil such that
  we choose to drill the land under Bayes decision
  rule?

State of Nature State of Nature Expected Payoff
Action Oil Dry Expected Payoff
Drill for oil 700 -100
Sell the land 90 90
Prior probability p 1-p
17
Decision Making with Experimentation
18
Goferbroke Example (contd)
State of Nature State of Nature
Action Oil Dry
Drill for oil 700 -100
Sell the land 90 90
Prior probability 0.25 0.75

• Option available to conduct a detailed seismic
  survey to obtain a better estimate of oil
  probability
• Costs 30k
• Possible findings
• Unfavorable seismic soundings (USS), oil is
  fairly unlikely
• Favorable seismic soundings (FSS), oil is fairly
  likely

19
Posterior Probabilities

• Do experiments to get better information and
  improve estimates for the probabilities of states
  of nature. These improved estimates are called
  posterior probabilities.
• Experimental Outcomes x1, x2,
• Example
• Cost of experiment ?
• Example
• Posterior Distribution P(? ?k X xj)

20
Goferbroke Example (contd)

• Based on past experience
• If there is oil, then
• the probability that seismic survey findings is
  USS 0.4 P(USS oil)
• the probability that seismic survey findings is
  FSS 0.6 P(FSS oil)
• If there is no oil, then
• the probability that seismic survey findings is
  USS 0.8 P(USS dry)
• the probability that seismic survey findings is
  FSS 0.2 P(FSS dry)

21
Bayes Theorem

• Calculate posterior probabilities using Bayes
  theorem
• Given P(X xj ? ?k), find P(? ?k X xj)

22
Goferbroke Example (contd)

• We have
• P(USS oil) 0.4 P(FSS oil) 0.6 P(oil)
  0.25
• P(USS dry) 0.8 P(FSS dry) 0.2 P(dry)
  0.75
• P(oil USS)
• P(oil FSS)
• P(dry USS)
• P(dry FSS)

23
Goferbroke Example (contd)

• Optimal policies
• If finding is USS

State of Nature State of Nature Expected Payoff
Action Oil Dry Expected Payoff
Drill for oil 700 -100
Sell the land 90 90
Posterior probability

• If finding is FSS

State of Nature State of Nature Expected Payoff
Action Oil Dry Expected Payoff
Drill for oil 700 -100
Sell the land 90 90
Posterior probability
24
The Value of Experimentation

• Do we need to perform the experiment?
• As evidenced by the experimental data, the
  experimental outcome is not always correct. We
  sometimes have imperfect information.
• 2 ways to access value of information
• Expected value of perfect information (EVPI)
• What is the value of having a crystal ball that
  can identify true state of nature?
• Expected value of experimentation (EVE)
• Is the experiment worth the cost?

25
Expected Value of Perfect Information

• Suppose we know the true state of nature. Then
  we will pick the optimal action given this true
  state of nature.

State of Nature State of Nature
Action Oil Dry
Drill for oil 700 -100
Sell the land 90 90
Prior probability 0.25 0.75

• EPI expected payoff with perfect information

26
Expected Value of Perfect Information

• Expected Value of Perfect Information
• EVPI EPI EOI
• where EOI is expected value with original
  information (i.e. without experimentation)
• EVPI for the Goferbroke problem

27
Expected Value of Experimentation

• We are interested in the value of the experiment.
  If the value is greater than the cost, then it
  is worthwhile to do the experiment.
• Expected Value of Experimentation
• EVE EEI EOI
• where EEI is expected value with experimental
  information.

28
Goferbroke Example (contd)

• Expected Value of Experimentation
• EVE EEI EOI
• EVE

29
Decision Trees
30
Decision Tree

• Tool to display decision problem and relevant
  computations
• Nodes on a decision tree called __________.
• Arcs on a decision tree called ___________.
• Decision forks represented by a __________.
• Chance forks represented by a ___________.
• Outcome is determined by both ___________ and
  ____________. Outcomes noted at the end of a
  path.
• Can also include payoff information on a decision
  tree branch

31
Goferbroke Example (contd)Decision Tree
32
Analysis Using Decision Trees

• Start at the right side of tree and move left a
  column at a time. For each column, if chance
  fork, go to (2). If decision fork, go to (3).
• At each chance fork, calculate its expected
  value. Record this value in bold next to the
  fork. This value is also the expected value for
  branch leading into that fork.
• At each decision fork, compare expected value and
  choose alternative of branch with best value.
  Record choice by putting slash marks through each
  rejected branch.
• Comments
• This is a backward induction procedure.
• For any decision tree, such a procedure always
  leads to an optimal solution.

33
Goferbroke Example (contd)Decision Tree Analysis
34
Painting problem

• Painting at an art gallery, you think is worth
  12,000
• Dealer asks 10,000 if you buy today (Wednesday)
• You can buy or wait until tomorrow, if not sold
  by then, can be yours for 8,000
• Tomorrow you can buy or wait until the next day
  if not sold by then, can be yours for 7,000
• In any day, the probability that the painting
  will be sold to someone else is 50
• What is the optimal policy?

35
Drawer problem

• Two drawers
• One drawer contains three gold coins,
• The other contains one gold and two silver.
• Choose one drawer
• You will be paid 500 for each gold coin and 100
  for each silver coin in that drawer
• Before choosing, you may pay me 200 and I will
  draw a randomly selected coin, and tell you
  whether its gold or silver and which drawer it
  comes from (e.g. gold coin from drawer 1)
• What is the optimal decision policy? EVPI? EVE?
  Should you pay me 200?

36
Utility Theory
37
Validity of Monetary Value Assumption

• Thus far, when applying Bayes decision rule, we
  assumed that expected monetary value is the
  appropriate measure
• In many situations and many applications, this
  assumption may be inappropriate

38
Choosing between Lotteries

• Assume you were given the option to choose from
  two lotteries
• Lottery 15050 chance of winning 1,000 or 0
• Lottery 2Receive 50 for certain
• Which one would you pick?

.5
1,000
.5
0
1
50
39
Choosing between lotteries

• How about between these two?
• Lottery 15050 chance of winning 1,000 or 0
• Lottery 2Receive 400 for certain
• Or these two?
• Lottery 15050 chance of winning 1,000 or 0
• Lottery 2Receive 700 for certain

.5
1,000
.5
0
1
400
.5
1,000
.5
0
1
700
40
Utility

• Think of a capital investment firm deciding
  whether or not to invest in a firm developing a
  technology that is unproven but has high
  potential impact
• How many people buy insurance?Is this monetarily
  sound according to Bayes rule?
• So... is Bayes rule invalidated?No- because we
  can use it with the utility for money when
  choosing between decisions
• Well focus on utility for money, but in general
  it could be utility for anything (e.g.
  consequences of a doctors actions)

41
A Typical Utility Function for Money
u(M)
4
3
What does this mean?
2
1
M
0
500
1,000
100
250
42
Decision Makers Preferences

• Risk-averse
• Avoid risk
• Decreasing utility for money
• Risk-neutral
• Monetary value Utility
• Linear utility for money
• Risk-seeking (or risk-prone)
• Seek risk
• Increasing utility for money
• Combination of these

u(M)
M
u(M)
M
u(M)
M
u(M)

M
43
Constructing Utility Functions

• When utility theory is incorporated into a real
  decision analysis problem, a utility function
  must be constructed to fit the preferences and
  the values of the decision maker(s) involved
• Fundamental propertyThe decision maker is
  indifferent between two alternative courses of
  action that have the same utility

44
Indifference in Utility

• Consider two lotteries
• The example decision maker we discussed earlier
  would be indifferent between the two lotteries
  if
• p is 0.25 and X is
• p is 0.50 and X is
• p is 0.75 and X is

p
1,000
1
X
1-p
0
45
Goferbroke Example (with Utility)

• We need the utility values for the following
  possible monetary payoffs

45
Monetary Payoff Utility
-130
-100
60
90
670
700
u(M)
M
46
Constructing Utility FunctionsGoferbroke Example

• u(0) is usually set to 0. So u(0)0
• We ask the decision maker what value of p makes
  him/her indifferent between the following
  lotteries
• The decision makers response is p0.2
• So

p
700
1
0
1-p
-130
47
Constructing Utility FunctionsGoferbroke Example

• We now ask the decision maker what value of p
  makes him/her indifferent between the following
  lotteries
• The decision makers response is p0.15
• So

p
700
1
90
1-p
0
48
Constructing Utility FunctionsGoferbroke Example

• We now ask the decision maker what value of p
  makes him/her indifferent between the following
  lotteries
• The decision makers response is p0.1
• So

p
700
1
60
1-p
0
49
Goferbroke Example (with Utility)Decision Tree
50
Exponential Utility Functions

• One of the many mathematically prescribed forms
  of a closed-form utility function
• It is used for risk-averse decision makers only
• Can be used in cases where it is not feasible or
  desirable for the decision maker to answer
  lottery questions for all possible outcomes
• The single parameter R is the one such that the
  decision maker is indifferent between

0.5
R
1
and
(approximately)
0
0.5
-R/2