Experiments, Good and Bad

1
Experiments, Good and Bad

• MA 155
• Chapter 5

2
Why Experiment?

• A family that raises rabbits in the Bay Area
  noticed that an unusual number of rabbits were
  dying of liver disease.
• They decided it was the pine shavings in their
  litterboxes

3
What Do You Call It?

• Observational Study
• Describes some group or situation
• Experiment
• Studies whether a treatment causes a change in
  the response

4
Example

• How many trials does it take to condition a rat?

5
Vocabulary

• Individuals
• Variable
• Response Variable
• Explanatory Variable
• Treatment

6
Example

• What are the individuals?
• What are the treatments?
• What is the response variable?

7
Placebo Effect

• When subjects are given a treatment, they may
  think that they feel better because they have
  been given a treatment.
• A placebo is a treatment with no effect

8
Example

• A procedure commonly used for angina was to make
  small incisions in the chest and tie knots in two
  arteries to try to increase blood flow to the
  heart. 90 of patients reported that it helped.

9
What to Watch For

• Confounding
• Occurs when the effects on a response variable
  cannot be distinguished from each other.
• Lurking Variable
• Has an effect on the variables being studied but
  is not a variable itself
• Treatment ? Response

10
Confounding

• X is confounded with Z
• Does X even influence Y?
• Examples

X
Y
Z
11
Example Does Yoga Alleviate Chronic Pain?

• 18 volunteers suffering from chronic pain
  participated in 90 minute yoga sessions three
  times a week for four weeks.
• Most volunteers reported that their pain
  decreased enough to ask their physicians to
  decrease their medication

12
Homework

• Review Chapter 5
• Read Chapter 6
• Do Exercises 5.13, 5.23

13
Design of an Experiment

• We must be able to show the effect of the
  explanatory variable
• Randomized Comparative Experiment
• Random assignment
• One group for each treatment
• Number of subjects for each group
• What treatment each group gets
• Response variable to compare

14
Instruction Example

• The Department of Statistics wants to compare
  methods of instruction. They want to see if MA155
  students perform better with actual or virtual
  instruction.
• The subjects are 300 students
• They randomly assign 150 students to virtual and
  randomly assign 150 students to actual
  instruction.

15
DUI Example

• We are studying 301 people who have been
  convicted of a DUI three times in one year. The
  treatments are a fine and suspended jail sentence
  plus
• No treatment (control group)
• Attend an alcoholism clinic
• Participate in AA.
• We want to compare the treatments and see whether
  a subject is arrested again.

16
The Logic

• Randomization produces groups that should be
  similar in all respects before the treatments are
  applied.
• Comparative design ensures that other influence
  affect groups equally.
• Differences in the response variable must be due
  to the effects of the treatments.

17
Principles

• Control the effects of lurking variables
• Randomize subjects
• Replicate to reduce variation

18
Statistical Significance

• Because of chance variation, we need to be
  careful of what we say.
• Statistical Significance is an observed effect so
  large that it would rarely occur by chance.

19
Instruction Example Continued

• The results of the study were that the average
  score on an exam for virtual is 85 and the
  average score for actual is 90.
• What if the results of the study were that the
  average score on an exam for virtual is 55 and
  the average score for actual is 90?

20
DUI Example Continued

• Results
• 44 of the no-treatment had no arrests
• 32 of the clinic and AA had no arrests
• Are these results significant?

21
Observational Studies

• Compare Matched Groups
• Measure Many Lurking Variables
• Adjust
• However, observational studies are still weaker
  in comparison to experiments when answering
  questions about causation.
• Rabbit Litterbox Example

22
Homework

• Read Chapter 5
• Do Exercises 5.5, 5.19