Between groups designs (2)

1
Between groups designs (2) outline

• Block randomization
• Natural groups designs
• Subject loss
• Some unsatisfactory alternatives to true
  experiments
• One group posttest only design
• Posttest only with non-equivalent control group
• One group pretest-posttest design

2
Block Randomization

• Block randomization (BR) is used to form groups
  of equal sizes
• First you create groups (called blocks)
• Then you randomly assign members of a block to
  your experimental treatments

3
Block Randomization

• in each block of treatments
• e.g., 4 treatments ? 4 subjects per block
• In that case, first 4 subjects to sign up would
  form Block 1, second 4 subjects to sign up would
  form Block 2, and so on.
• Subjects in each block now randomly assigned to
  treatments

4
Block Randomization

• Block randomization yields treatment groups which
  all have the same size.
• This is important for many statistical tests
• Equal ns mean (roughly) equal variances and thus
  comparable reliability
• Plus, BR will cause history effects to affect
  all groups equivalently

5
Block Randomization

• BR will eliminate confounding history effects
• changes in experimenter
• changes in the population (e.g., 1st vs. 2nd
  semester of Psych 020)
• actual historic events imagine if you had run
  your control group in the week September 3 7,
  2001 and treatment September 10 14, 2001
• block randomization will eliminate such
  confounds, at the expense of greater error
  variance

6
2. Natural Groups Designs

• Natural groups designs are those in which
  individual difference variables are selected
  rather than manipulated.
• A simple example is when you use age or sex as an
  independent variable you cannot randomly assign
  people to the conditions young or old, or to
  female or male.

7
2. Natural Groups Designs

• We also use natural groups designs when ethical
  constraints keep us from assigning people to
  groups
• E.g., you could assign people to divorce and
  no divorce treatments, and perhaps even pay
  people to get divorced or stay married. But to do
  so would be unethical
• Instead we would compare people who have chosen
  to get divorced to people who have chosen not to
  a natural groups design

8
2. Natural Groups Designs

• Natural groups designs are useful for
• Description
• Do divorced people receive psychiatric care at a
  higher rate than those who are married?
• Prediction
• If so, we can predict that a new set of divorced
  people is more likely than a new set of married
  people to need psychiatric care

9
2. Natural Groups Designs

• But natural groups designs cannot be used to make
  inferences about cause!
• Natural groups designs are correlational studies,
  not experiments
• You must NOT draw causal inferences from studies
  that use natural groups designs (that is, do not
  offer opinions about what causes any differences
  on your dependent variable between the groups).

10
2. Natural Groups Designs

• Since you did not establish equivalence of your
  groups at the beginning of your study (you did
  not randomly assign people to groups), you have
  not eliminated plausible alternatives to any
  causal account that you might offer.
• E.g., do divorced people need more psychiatric
  care because of the stress of divorce? Or do
  people who need more psychiatric care place more
  strain on their relationships or choose a mate
  unwisely in the first place?

11
Subject loss

• For a between-groups experiment to be internally
  valid, we need the two groups to be equivalent
  not only at the beginning of the experiment, but
  also at the end.
• If more subjects drop out of one group than out
  of another, the two groups may no longer be
  comparable.

12
Subject loss

• Two kinds of subject loss
• Mechanical subject is lost from the experiment
  because of equipment failure.
• This is probably a random effect thus, will
  not produce systematic differences between the
  two groups.

13
Two kinds of subject loss

• B. Selective this is when some characteristic
  of either the subject or the treatment is
  responsible for the loss
• e.g., treatment involves a difficult or
  unpleasant task, but control condition does not
• clinically depressed subjects compared with
  sub-clinically depressed controls the most
  severely depressed subjects in the former group
  may be the most likely to drop out

14
Two kinds of subject loss

• B. Selective what can you do?
• If you notice this loss after the fact, nothing.
• If you anticipate such loss, you may be able to
  screen people on some variable that will let you
  predict loss, and then select subjects on that
  basis at a cost to generalizability.

15
But what about external validity?

• Random assignment in Loftus Burns study
  guaranteed internal validity the group
  difference in performance could not have been
  caused by anything other than the treatment.
• But what about external validity?
• Would the same effects be found with a real-life
  bank robbery instead of one on film?
• Would the same effects be found with people
  other than young university students?

16
But what about external validity?

• As Stanovich points out, the answer is often,
  who cares?
• we often do an experiment to test a particular
  theory, not to find out what the ordinary person
  would do in the real world
• Often, any kind of subject will do to test our
  theory, so long as they are competent in our
  experimental task

17
But what about external validity?

• Of course, sometimes generalizability matters.
• if so, then try for representative samples
  situations
• when you cant do that, at least use several
  different types of people, stimuli, and
  situations
• or replicate partial or complete replication
• or use meta-analysis review of published papers
• Set criteria for inclusion of papers in your
  review
• Select a procedure for amalgamating findings

18
Some unsatisfactory alternatives to experiments

• All of the following fail to control for
  important threats to the validity of a
  conclusion
• One group posttest only design
• Cant tell if treatment changed behavior if you
  dont know what behavior was like to start with.

19
Some unsatisfactory alternatives to experiments

• Posttest only with non-equivalent control group
• Control treatment groups are not equated at
  the start.
• Differences between treatment and control groups
  could be due to treatment or to other things
  (since control group is not equivalent).

20
Some unsatisfactory alternatives to experiments

• One group pretest-posttest design
• Change in behavior may have been caused by
  variables other than the one you think produced
  it. (E.g., maturation, attention, change in the
  weather)