Design

1
Design

• Experimental Control

2
Experimental control allows causal inference
(IV caused observed change in DV)

• Experiment has internal validity when it fulfills
  3 conditions for causal inference
• 1) covariation
• 2) time-order relationship
• 3) elimination of plausible alternatives

3
Specify variables to be controlled

• Controlling extraneous variables
• 1) elimination
• 2) holding conditions constant
• 3) randomization/balancing
• 4) counterbalance

4
1) Elimination

• If possible eliminate the extraneous variable
• Eg noise
• As a confound group A measured during high
  traffic Group B low traffic noises
• Nuisance variable (may not be a confound). Random
  noises from heating system.

5
2) Hold conditions constant

• Minimize variability
• Time of day
• Lighting
• Instructions
• Stimuli
• Procedure.

6
Loftus and Burns ( 1982)

• Two groups both saw a film of a bank robbery.
  Only the ending differed.
• Group A violent ending
• Group B nonviolent
• Both groups asked questions about events that
  happened prior to end scenes
• Eg the number on a t-shirt worn by a bystander
• Correct recall group A 4 Group B 28
• Same film, same instructions, same questions,
  same room
• Did not control same temperature or weather
• Only factors thought to impact DV

7
3) Randomization/Balance

• Especially useful if unsure what extraneous
  variables may be operating

8
Between Subjects Design

• only choice if
• a) subject variable eg smoker and non-smoker
• b) if manipulation of IV makes repeats impossible
  or undesirable (deception or carryover effects)
• the number of groups the number of levels of IV

9

• disadvantages
• many subjects needed
• individual variation and selection effects
• statistical tests
• compare variability between groups to variability
  within groups
• sources of variability are
• a) the IV
• b) confounds systematic
• c) error unsystematic (individual variability)

10
Design problems

• The equivalent group

11
Equivalent Groups

• - try to compensate for selection effect
• - groups are equal to each other in important
  ways
• - the number of groups the number of levels of
  IV

12
Random Assignment

• Every participant has equal chance of being in
  each group, the individual variation is spread
  through the groups evenly
• this works well with big N

13

• b) Block Randomization
• use random number table to assign order
• if have 5 groups then use numbers 1-5
• list the numbers in the order they appear must
  finish sequence before repeating a number
• c) Matching
• if small N then a few individuals assigned by
  chance can have a big impact
• test participants on a variable and pair scores
  each group gets similar scores
• -you need a priori reason to match on a variable
• -it adds logistical complexity
• -may give away hypothesis ( bias and reactivity
  problem)

14
Example

• weights
• 156 167 183 170 145
• 143 152 145 181 162
• 175 159 169 174 161
• order
• 143 145 145 152 156 159 161 162 167 169 170 174
  175 181 183

15
Matching
16
Block randomization

• 167 183 170 145
• 143 152 145 181 162
• 175 159 169 174 161

2 1 1 1 3 1 3 3 3 2 3 2 2 2 1
17
Balancing

• Cannot control characteristics of participants.
• Try to evenly spread the individual differences
  between the levels of IV
• Random assignment
• Eg if in the Loftus and Burns study groups
  differed in attention or memory then problem

18
Between SubjectsDesign problems

• The equivalent group
• Solution randomize or balance

19
Within Subjects Design (repeated measures)

• Each participant exposed to each level of the IV
• Fewer people needed (economical)
• Individual variability removed as source of error
  (more power in testing)
• Great for rare events/species/diseases

20

• BUT sequence or order effects can be problematic
• Progressive effects
• Practice improves performance
• Fatigue worsens performance
• Carryover effects
• Doing task A has bigger impact on task B than
  the reverse
• Uneven impact

21
Within SubjectsDesign problem

• Sequence effects

22
4) Counterbalance

• a) complete counterbalancing use all possible
  sequences of orders at least once
• good if few conditions (3 or less) (n! possible)
• 3 groups gives ? possible combinations
• 4 groups ? possible.

23
4) Counterbalance

• a) complete counterbalancing use all possible
  sequences of orders at least once
• good if few conditions (3 or less) (n! possible)
• 3 groups gives 6 possible combinations
• 4 groups 24 possible.

24

• b) partial counterbalancing
• - take random sample of all possible sequences ,
  reduces systematic bias

25

• c) Latin squares
• every condition appears equally often in every
  sequential position
• - if balanced Latin square then each condition
  precedes and follows every other once

26
Latin Squares
27
Latin Squares
28
Latin Squares
29
Latin Squares
30
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2
,5.Second row add one
31
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2
,5.Second row add one
32
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2
,5.Second row add one
33
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2
,5.Second row add one
34
Within SubjectsDesign problem

• Sequence effects
• Solution - counterbalance

35
Experimental Control

• Dependant Variable
• validity
• reliability
• multiple measures

36
Independent Variable

• Vary in a systematic way
• Control confounds related to IV
• Eliminate
• Hold constant
• Balance (groups)
• Counterbalance (order)
• Randomize
• Plan for experimenter bias

37
Participant Effects

• Random assignment
• Pilot measures for social desirability
• Consider floor/ceiling
• Yes/no bias

38
Single group

• A single group threat includes history,
  maturation, testing, instrumentation, mortality
  and regression to mean threats.

39
Multiple Groups

• These multiple group threats are called a
  selection bias or selection threat.
• These include selection history, selection
  maturation, selection testing, selection
  instrumentation, selection mortality and
  selection regression threats

40
Double pretest
The design includes two measures as denoted by
two "Os" prior to the program. This design can
rule out selection maturation threat and a
selection regression threat. It will help to make
sure that the two groups are comparable before
the treatment
41
Switching Replication Design
Good at solving the social threats to internal
validity compensatory rivalry, compensatory
equalization, resentful demoralization. Both
groups get same program so no inequity
42

• control group assumes extraneous variables
  operate on both experimental and control equally
• more than one control group can be used to assess
  different variables

43
Single Group
Multiple Groups
44
Solomon 4 group design
testing threat The design consists of four
groups of randomly assigned. Two of them receive
the treatment as denoted by " X" and the other
two do not.
45
Determine extraneous variables
Will not influence DV
Might influence DV
Cannot be controlled
Can be controlled
ignore
Cannot randomize
Randomize
Continue experiment
Continue experiment
Continue experiment
Abandon experiment