CHAPTER 4 Designing Studies

1
CHAPTER 4Designing Studies

• 4.2
• Experiments

2
Experiments

• DISTINGUISH between an observational study and an
  experiment.
• EXPLAIN the concept of confounding.
• IDENTIFY the experimental units, explanatory and
  response variables, and treatments in an
  experiment.
• EXPLAIN the purpose of comparison, random
  assignment, control, and replication in an
  experiment.
• DESCRIBE a completely randomized design for an
  experiment.
• DESCRIBE the placebo effect and the purpose of
  blinding in an experiment.
• INTERPRET the meaning of statistically
  significant in the context of an experiment.
• EXPLAIN the purpose of blocking in an experiment.
  DESCRIBE a randomized block design or a matched
  pairs design for an experiment.

3
Observational Study vs. Experiment
An observational study observes individuals and
measures variables of interest but does not
attempt to influence the responses. An
experiment deliberately imposes some treatment on
individuals to measure their responses.
When our goal is to understand cause and effect,
experiments are the only source of fully
convincing data. The distinction between
observational study and experiment is one of the
most important in statistics.
4

• See example on p.235

5
Observational Study vs. Experiment

• Observational studies of the effect of an
  explanatory variable on a response variable often
  fail because of confounding between the
  explanatory variable and one or more other
  variables.
• Well-designed experiments take steps to prevent
  confounding.

Confounding occurs when two variables are
associated in such a way that their effects on a
response variable cannot be distinguished from
each other.
To explain confounding you should explain how the
variable you chose is associated with the
explanatory variable and also how it affects the
response variable.
6

• AP Exam Tip on p.236
• Diet is a confounding variable because people
  with bad diets are more likely to have a heart
  attack
• Need to explain why diet is also associated with
  the choice to take hormones.
• Would need to say something about why a better
  diet might be more common in the group of women
  who choose to take hormones.
• AP common error on p.236
• For the hormone example you would need to explain
  not only that wealthier people tend to go to the
  doctor more often and to be healthier overall,
  but also that wealthier people are more likely to
  get hormone replacement. Thus, it could actually
  be the overall healthiness of wealthy women that
  caused the reduction in heart attacks, not any
  hormones they may have taken.
• CYU on p.237

7
The Language of Experiments

• An experiment is a statistical study in which we
  actually do something (a treatment) to people,
  animals, or objects (the experimental units) to
  observe the response. Here is the basic
  vocabulary of experiments.

A specific condition applied to the individuals
in an experiment is called a treatment. If an
experiment has several explanatory variables, a
treatment is a combination of specific values of
these variables. The experimental units are the
smallest collection of individuals to which
treatments are applied. When the units are human
beings, they often are called subjects.
Often times the explanatory variables in an
experiment are called factors. If an
experiment has several factors, the combinations
of each level of each factor form the treatments.
8

• See example on p.239
• http//www.ted.com/talks/michael_norton_how_to_buy
  _happinesst-107021
• Describes a multifactor experiment to determine
  if money can buy hapiness.
• There are 2 factors, amount of money how it is
  spent.

9
How to Experiment Badly

• Many laboratory experiments use a design like the
  one in the online SAT course example on p.240

Experimental Units
Treatment
Measure Response
In the lab environment, simple designs often work
well. Field experiments and experiments with
animals or people deal with more variable
conditions. Outside the lab, badly designed
experiments often yield worthless results because
of confounding.
10
How to Experiment Well

• The remedy for confounding is to perform a
  comparative experiment in which some units
  receive one treatment and similar units receive
  another. Most well designed experiments compare
  two or more treatments.
• Comparison alone isnt enough, if the treatments
  are given to groups that differ greatly, bias
  will result. The solution to the problem of bias
  is random assignment.

In an experiment, random assignment means that
experimental units are assigned to treatments
using a chance process.
See example on p.241
11
Principles of Experimental Design
Principles of Experimental Design
The basic principles for designing experiments
are as follows 1. Comparison. Use a design that
compares two or more treatments. 2. Random
assignment. Use chance to assign experimental
units to treatments. Doing so helps create
roughly equivalent groups of experimental units
by balancing the effects of other variables among
the treatment groups. 3. Control. Keep other
variables that might affect the response the same
for all groups. 4. Replication. Use enough
experimental units in each group so that any
differences in the effects of the treatments can
be distinguished from chance differences between
the groups.
12

• See example on p.243

13
Completely Randomized Design
In a completely randomized design, the treatments
are assigned to all the experimental units
completely by chance. Some experiments may
include a control group that receives an inactive
treatment or an existing baseline treatment.
Experimental Units
14

• See example on p.245
• Read exam tip on p.246
• CYU on p.247

15
Experiments What Can Go Wrong?
The logic of a randomized comparative experiment
depends on our ability to treat all the subjects
the same in every way except for the actual
treatments being compared. Good experiments,
therefore, require careful attention to details
to ensure that all subjects really are treated
identically.
The response to a dummy treatment is called the
placebo effect. In a double-blind experiment,
neither the subjects nor those who interact with
them and measure the response variable know which
treatment a subject received.
16

• See example on p.247
• http//www.cbsnews.com/videos/treating-depression-
  is-there-a-placebo-effect/

17
Inference for Experiments
In an experiment, researchers usually hope to see
a difference in the responses so large that it is
unlikely to happen just because of chance
variation. We can use the laws of probability,
which describe chance behavior, to learn whether
the treatment effects are larger than we would
expect to see if only chance were operating. If
they are, we call them statistically significant.
An observed effect so large that it would rarely
occur by chance is called statistically
significant. A statistically significant
association in data from a well-designed
experiment does imply causation.
18
Blocking
When a population consists of groups of
individuals that are similar within but
different between, a stratified random sample
gives a better estimate than a simple random
sample. This same logic applies in experiments.
A block is a group of experimental units that are
known before the experiment to be similar in some
way that is expected to affect the response to
the treatments. In a randomized block design, the
random assignment of experimental units to
treatments is carried out separately within each
block.
19
Matched Pairs Design
A common type of randomized block design for
comparing two treatments is a matched pairs
design. The idea is to create blocks by matching
pairs of similar experimental units.
A matched pairs design is a randomized blocked
experiment in which each block consists of a
matching pair of similar experimental units.
Chance is used to determine which unit in each
pair gets each treatment. Sometimes, a pair in
a matched-pairs design consists of a single unit
that receives both treatments. Since the order of
the treatments can influence the response, chance
is used to determine with treatment is applied
first for each unit.
20
Experiments

• DISTINGUISH between an observational study and an
  experiment.
• EXPLAIN the concept of confounding.
• IDENTIFY the experimental units, explanatory and
  response variables, and treatments in an
  experiment.
• EXPLAIN the purpose of comparison, random
  assignment, control, and replication in an
  experiment.
• DESCRIBE a completely randomized design for an
  experiment.
• DESCRIBE the placebo effect and the purpose of
  blinding in an experiment.
• INTERPRET the meaning of statistically
  significant in the context of an experiment.
• EXPLAIN the purpose of blocking in an experiment.
  DESCRIBE a randomized block design or a matched
  pairs design for an experiment.