Statistical decision making

1
Statistical decision making
2
Aristotle vs Avicenna

• Aristotelian view observe and deduce
• Avicennean view take advantage of deductions
  predict

3
Burning Questions

• Should I go to Vegas?
• Should I buy life insurance?
• Should I invest in the stock market?
• If I drop a tennis ball from the roof, will it
  hurt someone?
• Should I bring an umbrella?
• Should I get help?

4
Scientific Method

• Define a question (e.g., are taller people
  smarter?)
• Gather information (observe a bunch of people of
  different heights)
• Make hypothesis also what is close (tall
  people are often smarter?)
• Design experiment (IQ test groups of people who
  are similar in all ways save height)
• Perform experiment, collect data, analyze data
• Interpret data and draw conclusions that serve as
  a starting point for new hypothesis
• Publish results
• Retest (frequently done by other scientists)

5
Steps in Making Decisions

• Decisions to be made
• Is a coin fair or all l
• Will a company with lots of policies nearly
  identical make money
• The typical decay of an isotope is 1 every year
  (not same as 50 in 50 years)
• Is somebody guilty of a crime
• Should everyone take lipitor?
• Should I hire my Math 210 professor as a
  consultant?

6
Steps in Making a Decision

• Step 1- What do you think is true (H0) null
  hypothesis and what do you think could be the
  alternative (H1)
• H0 -
• chance of heads ½
• average profit gt average claim
• person is not guilty
• Lipitor safe AND effective
• Professor not very clever

7
What can go right/wrong with prediction based on
limited data acquisition?
reality vs. conclusion H0 is really true H1 is really true
You think H0 is true Good! Error (Type II)
You think H1 is true Error (Type I) Good!
8
Table in context- Trial by Jury
truth vs. conclusion H0 is really true H1 is really true
You think H0 is true Good! Innocent goes free Guilty person set free
You think H1 is true Innocent person sent to jail Good! Guilty goes to jail
9
Breast cancer screening
truth vs. conclusion H0 is really true H1 is really true
Screening result is negative H0 Greatso far Cancer may grow and spread
Screening result is positive H1 Get further tests Better to find out sooner.
10
Statins safe and effective?
truth vs. conclusion H0 is really true H1 is really true
You think H0 is true Live long and prosper Headaches, diarrhea, liver damage, all for naught
You think H1 is true Better watch your diet and get exercise! Okay, but still take care of yourself
11
Measuring the errors- Step 2

• In making a decision consider tradeoff between
  Type I and Type II error.
• Step 2- Decide on a significance level which is
  chance (Type I error) written a
• Reasonable Doubt e.g. decide how willing you
  are to send an innocent to jail,

12
How to decide a

• From utility theory there is a cost to type I
  error and type II error. We pick so that the
  overall cost is minimized
• Cost of sending innocent to jail say 1,000,000
  Cost of guilty going free 100,000 roughly
  choose type I error 10 of chance of type II error

13
Other examples

• Breast cancer screening cost of false positive
  (type I) small compared to false negative (type
  II)
• Statins safe and effective cost of false
  positive depends on consequences, same with false
  negative. If the consequences of a type II error
  are mild discomfort and unnecessary treatment,
  not so big a deal compared to risk of heart
  attack, but risk of liver damage is much more
  serious. Merits further study?

14
Step 3

• Decide what data to collect and how you will
  analyze it to help make a decision regarding
• H0 design experiment
• If fingerprints of defendant are found on weapon
  you think guilty- what kind of evidence
• Your friend has a positive screening for breast
  cancer
• Your friend who takes Lipitor develops headaches
• Your professor cant find his glasses

15
Step 4-

• Ideally decide if experiment comes out

16

• Rejecting a true hypothesis is known as type
  ___________, its chance is called the __________
  level.
• The ___________ is a scale for multiples of how
  far an observation is from expected. If data is
  bell shaped aka normal aka Gaussian aka ________
  by Taleb then _______ of it will lie within 1
  standard deviation of the average and _______
  will lie with 2 standard deviations. The quick
  decay actually says that _______ will lie
  within 5 standard deviations. 1 - 2/3,500,000

17
Fair coin, revisited

• OK how do we compute a for a coin?
• Flip the coin 4xNxN times where N is very large.
• The expected number of heads is for fair coin
  is 2xNxN.

18
Normal approximation to the binomial
If n is large enough, then an excellent
approximation to B(n, p) is given by the normal
distribution N(µ,s) Where µnp svnp(1-p)
19
Normal distribution
20

• Approximately 95 percent of values lie within 2s
  of the mean. When n2NxN and p0.5 we should get
  a number of heads within 2N of the mean 2xNxN
  about 95 times out of 100.
• Suppose that N1000. If we flip the coin 4
  million times, we should get within plus or minus
  2000 of 2 million 95 times out of 100. So if we
  get heads 2002000 times we suspect the coin is
  unfairly skewed toward heads. Here, a0.05.
• We have rejected the null hypothesis. If, in
  fact, the coin is fair, we have made a type I
  error.