4.2 - Experiments

1
4.2 - Experiments
2
Observational Studies

• measures variables of interest without attempting
  to influence the responses.
• sample surveys
• watching animals in nature
• you provide no influence on responses
• just notices relationship, doesnt imply causation

3
Experiment

• deliberately imposing some treatment(s) on
  individuals to measure their responses.
• does the treatment cause a change in the
  responses?
• helps determine a cause and effect relationship

4
Lurking Variable

• a variable that is not the explanatory or the
  response variable in a study but may influence
  the response variable.
• example you provide data for the relationship
  between the number of dinners you eat with your
  family and your GPA. The number of dinners you
  eat with your family might not be the only reason
  for a higher GPA.
• A lurking variable could be the interest your
  parents have in your education - therefore your
  parents involvement is creating a higher GPA, not
  necessarily the number of dinners.

5
Confounding

• occurs when 2 variables are associated in such a
  way that their effects on a response variable
  cannot be distinguished from each other.
• By designing effectively, you can prevent lurking
  variable from becoming confounding variables.
• Observational studies often fail due to
  confounding of explanatory variables and lurking
  variables.

6
Observational or Experiment?

1. Does reducing screen brightness increase battery
   life in laptop computers? To find out,
   researchers obtained 30 new laptops of the same
   brand. They chose 15 at random and adjusted their
   screens to the brightest setting. The other 15
   laptops were left at the default setting -
   moderate brightness. Researchers then measured
   how long each machines battery lasted.
2. A study of child care enrolled 1364 infants and
   followed them through their sixth year in school.
   Later, the researchers published an article in
   which they stated that the more time children
   spent in child care from birth to age
   four-and-a-half, the more adults tended to rate
   them, both at age four-and-a-half and at
   kindergarten, as less likely to get along with
   others, as more assertive ,as disobedient, and as
   aggressive.

7
Lets look at the child care study some more

• What are the explanatory and response variables?
• Does this study show that child care causes
  children to be more aggressive? Explain
• Are their any lurking variables? Are they
  confounded?

8
Effects of binge drinking

• A common definition of binge drinking is 5 or
  more drinks at one sitting for men and 4 or more
  for women. An observational study finds that
  students who binge drink have lower average GPA
  than those who dont. Identify a lurking variable
  that may be confounded with the effects of binge
  drinking. Explain how confounding might occur.
• Could a lurking variable be controlled so that
  it is not confounded?

9
Parts of an Experiment

1. Explanatory Variable (aka Factors) - we can have
   more than one
2. Response Variable - what we are measuring as a
   result of the experiment
3. Treatment - the specific condition applied to
   individuals in an experiment. Could be 1 or many
   in one experiment.
4. Experimental Units - the smallest collection of
   individuals to which treatments are applied.
   (when they are humans they are called subjects)
5. Levels - when there is more than one treatment
   option due to multiple factors in an experiment

10
Identify the experimental units, the explanatory
and response variables, and treatments in the
following experiments.

• A study published in the New England Journal of
  Medicine in March 2010 compared 2 medicines to
  treat head lice an oral medication (ivermectin)
  and a topical lotion (malathion). Researchers
  studied 812 people in 376 households in 7 areas
  around the world. Of the 185 households randomly
  assigned to ivermectin, 171 were free from head
  lice after 2 weeks compared with only 151 of the
  191 households randomly assigned to malathion.

11
Identify the experimental units, the explanatory
and response variables, and treatments in the
following experiments.

• Does adding fertilizer affect the productivity of
  tomato plants? How about the amount of water? To
  answer these questions, a gardener plants 24
  similar tomato plants in identical pots in his
  greenhouse. He will add fertilizer to the soil in
  half the pots. Also, he will water 8 of the
  plants with 0.5 gallons of water per day, 8 of
  the plants with 1 gallon of water per day, and
  the remaining 8 plants with 1.5 gallons of water
  per day. At the end of three months, he will
  record the total weight of tomatoes produced by
  each plant.

12
A Template to Design an Experiment

Explanatory Variables/Factors
Response Variable
This will be written out and later a design can
be drawn
Subjects/Experimental units
Treatment(s)
more will be added later
13
Ch 4.2 - Experiments

• Day 5

14
Randomized Design

• To help control lurking variables, an experiment
  needs comparisons
• This helps confounding from occurring

15
Examples of a Completely Randomized Design
A health organization wants to know if a low-carb
or a low-fat diet is more effective for long-term
weight loss. The organization decides to conduct
an experiment to compare these two diet plans
with a control group that is only provided with a
brochure about healthy eating. Ninety volunteers
agree to participate in the study. Assign a
number 00 to 90 to all subjects in alphabetical
order by last name. Go to a line in Table D and
read two-digit groups from left to right
(throwing out any repeated digits or digits
larger than 90). The first 30 digits will have
the brochure, the next 30 digits will do the
low-carb diet, and the remaining 30 will have the
low-fat diet. At the end of the year the total
weight loss of each group will be compared.

• 30 students volunteer to be subjects in a
  caffeine experiment. On 30 identical slips of
  paper there are 15 As and 15 Bs. They are mixed
  in a hat and each student selects one slip of
  paper. Students who receive A drink the cola with
  caffeine and students who receive B drink the
  cola without caffeine. At the end of an hour we
  will ask the students if they still feel
  energized and compare the results.

16
Control Groups

• These are important to provide a baseline for
  comparing the effects of the treatments
• aka placebo groups (more will be discussed about
  placebos tomorrow)

17
Three Principles of Experimental Design

1. Control
2. Random Assignment
3. Replication

Control Groups Control lurking variables by
creating groups with the only difference being
the treatments
impersonal chance to assign experimental units to
treatments create roughly equal groups by
balancing the effects of lurking variables that
cant be controlled
use enough experimental units in each group so
differences from the treatments can be
distinguished from just chance differences
between the groups
Dont get lost in the vocabulary - remember the
main goal is to create large treatment groups
with no systematic differences between them other
than the treatment!
18
A Template to Design an Experiment
Sample
Explanatory Variables/Factors
Response Variable
Random Assignment of experimental units/subjects
Keep in mind 3 Principles 1. Control 2. Random
Assignment 3. Replication
Assign Treatment(s)
State what results will be measured and compared
What conclusions can be drawn?
more will be added later
19
What went wrong?

• Will cash bonuses speed the return to work of
  unemployed people? A state department of labor
  notes that last year 68 of people who filed
  claims for unemployment insurance found a new job
  within 15 weeks. As an experiment, this year the
  state offers 500 to people filing unemployment
  claims if they find a job within 15 weeks. The
  percent who do so increases to 77. What flaw
  does this design have? Is it impossible to say
  whether the bonus really caused the increase?

20
Design a completely randomized experiment.

• 150 students are willing to serve as subjects to
  study the effects of repeated exposure to an
  advertising message. The answer depends on both
  the length of the ad and on how often it is
  repeated. The different lengths that will be used
  are 30 second ads and 90 second ads. The
  commercial will either be shown 1, 3, or 5 times
  during the program. After viewing, all the
  subjects answered questions about their recall of
  the ad, their attitude toward the camera ad, and
  their intention to purchase it.

21
Ch 4.2 - Experiments

• Day 6

22
Placebo Effect

• a response to a dummy treatment
• subjects do not know they are receiving a
  placebo because the effects are so strong

23
How to Control the Placebo Effect

• Double Blind Experiments - neither the subject
  nor those who interact with them know who has
  what treatment
• Single-Blind Experiments - when the subject knows
  what treatment they receive, but those measuring
  the responses do not.
• Double Blind is the best... WHY?

24
A Template to Design an Experiment
Sample
Explanatory Variables/Factors
Response Variable
Random Assignment of experimental units/subjects
Keep in mind 3 Principles 1. Control 2. Random
Assignment 3. Replication
Assign Treatment(s)
State if there is any blinding, if so why
State what results will be measured and compared
What conclusions can be drawn?
more will be added later
25
and now, for two more infamous stat words

• Statistically Significant
• an observed effect so large that it would rarely
  occur by chance
• How can we measure if an experiments results
  were statistically significant?

26
Can random just be unlucky?

• NO!
• If you think there is going to be a difference
  between the way the experimental units will react
  to the treatment, then you need to create a
  design to control those lurking variables.
• i.e. With the distracted driving activity, lets
  say that you feel women will forget more anyways
  because they are worse drivers. Then you will
  need to design your experiment to have women and
  men in both groups, but still randomly assigned!

27
Blocking

• Used mainly when there are strata already built
  into the population where a stratified random
  sample will be appropriate. Blocking is a form of
  control in the experimental design.
• Block - the group of experimental units that are
  known before the experiment to be similar in some
  way that is expected to affect the response to
  the treatments.
• Randomized Block Design - random assignment of
  experimental units to treatments that is carried
  out separately within each block

28
Stratified Random Sample or Randomized Block
Design?
Stratified Random Sample Randomized Block Design
Controls lurking variables
Forms similar groups
Done when taking the sample from the population
Done when assigning units to the treatments
29
Examples of a Randomized Block Design

• Anne is an avid baker who would like to compare
  two different chocolate chip cookie recipes (A
  and B). So she recruits 10 volunteer taste
  testers to rate each type of cookie on a scale of
  1 (very bad) to 10 (very good). She will make 10
  of each type of cookie, for a total of 20. Each
  cookie tray will hold only 10 cookies, so she
  will use two trays to bake them at the same time
  in the same oven, one sheet on the lower rack and
  one of the upper rack. She thinks that the
  cookies will bake differently depending on which
  rack they are on, we will use the 10 locations on
  the lower-rack cookie sheet as 1 block and the 10
  locations on the upper-rack cookie sheet as the
  2nd block. On each sheet Anne will randomly place
  5 of each type of cookie. This way each type of
  cookie will have 5 on the lower rack and 5 on the
  upper rack, balancing out the effect of rack
  location.
• What are the experimental units?
• What are the treatments?
• Draw a diagram to represent this situation.

30
Matched Pairs

• a type of blocking
• block size is always 2 because only two
  treatments
• every experimental unit will receive both
  treatments, in a random order or the experimental
  units are paired as closely as possible then each
  treatment is assigned randomly to each unit
• results are compared from each individual and
  then as a whole group

31
A Template to Design an Experiment
Sample
Explanatory Variables/Factors
Response Variable
Random Assignment of experimental units/subjects
Keep in mind 3 Principles 1. Control 2. Random
Assignment 3. Replication
Assign Treatment(s)
State if there is any blinding, if so why
State what results will be measured and compared
State if there is any blocking, if so why
What conclusions can be drawn?