Experimental Research Designs

1
Experimental Research Designs
2
Experimental Design

• Advantages
• Best establishes cause-and-effect relationships
• Disadvantages
• Artificiality of experiments
• Feasibility
• Unethical

3
Causality

• Temporal precedence
• Covariation between IV and DV
• Eliminate alternative explanations

4
Types of Experimental Designs

• Simple True Experimental
• Complex True Experimental
• Quasi-Experimental

5
Types of Experimental Designs

• Simple True Experimental
• Complex True Experimental
• Quasi-Experimental

6
Simple True Experimental

• Characteristics
• Types
• Variations

7
Characteristics of True Designs

• Manipulation (treatment)
• Randomization
• Control group
• Characteristics of simple true designs
• One IV with 2 levels (T, C)
• One DV

8
Types

• Randomized posttest control group design
• Randomized pretest-posttest control group design

9
Randomized posttest control group design

• R T Post
• R C Post

10
Randomized pretest-posttest control group design

• R Pre T Post
• R Pre C Post

11
Advantages Disadvantages

• Advantages of pretest design
• Equivalency of groups
• Can measure extent of change
• Determine inclusion
• Assess reasons for and effects of mortality
• Disadvantages of pretest design
• Time-consuming
• Sensitization to pre-test

12
Solomon four-group design

• R Pre T Post
• R Pre C Post
• R T Post
• R C Post

13
Variations

• Independent groups (between groups)
• Repeated measures (within groups)

14
Repeated Measures Design

• Advantages
• Fewer subjects needed (less costly)
• Sensitive to finding statistical differences
• Disadvantages
• Order effect (practice, fatigue, carry-over)

15
Dealing with Order Effects

• Counterbalancing
• n!
• Latin squares

16
Latin Squares
17
Dealing with Order Effects

• Counterbalancing
• n!
• Latin squares
• Randomized blocks
• Time interval between treatments

18
Variations

• Independent groups (between) vs. repeated
  measures (within) designs

Consider external validity when deciding which
design to use.
19
Types of Experimental Designs

• Simple True Experimental
• Complex True Experimental
• Quasi-Experimental
• Pre-Experimental

20
Characteristics of True Designs

• Manipulation (treatment)
• Randomization
• Control group
• Characteristics of simple true designs
• One IV with 2 levels (T, C)
• One DV

21
Complex True Experimental

• Randomized matched control group design
• Increased levels of IV
• Factorial design
• Multiple DVs

22
Complex True Experimental

• Randomized matched control group design
• Increased levels of IV
• Factorial design
• Multiple DVs

23
Randomized matched control group design

• M R T Post
• M R C Post

• Used in small samples
• ? cost in time money

24
Complex True Experimental

• Randomized matched control group design
• Increased levels of IV
• Factorial design
• Multiple DVs

25
Increased Levels of IV

• Provides more complete information about the
  relationship between the IV DV
• Detects curvilinear relationships
• Examines effects of multiple treatments

26
DV
Performance level ( complete)
IV
Amount of reward promised ()
27
Increased Levels of IV
DV
Performance level ( complete)
IV
Amount of reward promised ()
28
Complex True Experimental

• Randomized matched control group design
• Increased levels of IV
• Factorial design
• Multiple DVs

29
Factorial Design

• gt1 IV (factor)
• Simultaneously determine effects of 2 or more
  factors on the DV (real world)
• Between Factor vs. Within Factor
• IDd by of factors and levels of factors

2X2
30
Do differing exercise regimens (hi, med, lo
intensity) have the same effect on men as they do
on women?

• 3 X 2 (Exercise Regimen X Gender)
• 2 factors
• Exercise Regimen 3 levels
• Gender 2 levels
• Between factors
• DV?
• Experimental IVs or Participant IVs?

31
(No Transcript)
32
Do strength gains occur at the same rate in men
as they do in women over a 6 mo. training period?
Measurements are taken at 0, 2, 4, 6 mo.

• 2 X 4 (Gender X Time)
• ? factors
• Time 4 levels
• Gender 2 levels
• Between or within factors?
• DV?
• Experimental IVs or Participant IVs?

33
(No Transcript)
34
Cell means, Margin meansMain Effects,
Interactions

35
Interaction Main Effects

• Interaction - combined effect of the factors on
  the dependent variable
• Main effect the deviation of two or more
  treatment means from the grand mean

36
(No Transcript)
37
Parallel lines indicate no interaction.
Is there a main effect?
38
Is there a main effect?
39
Non-parallel lines indicate an interaction.
Is there a main effect?
40
Is there a main effect?
41
Interpretation

• Always interpret the interaction first
  (graphical)
• If no significant interaction, interpret main
  effects

42

• Advantages of factorial designs
• Greater protection against Type I error
• More efficient
• Can examine the interaction
• Disadvantages
• ? subject for between factor designs

Consider external validity when deciding which
design to use.
43
IV A Exposure to Violence violent vs.
nonviolent videoIV B Gender male vs.
femaleDV ads recalled (0-8)
B1
B2
9
B
A
5
1
1
2
A
44
IV A Exposure to Violence violent vs.
nonviolent videoIV B Gender male vs.
femaleDV ads recalled (0-8)
B1
B2
9
A Yes B No AxB Yes
5
1
1
2
A
45
Complex True Experimental

• Randomized matched control group design
• Increased levels of IV
• Factorial design
• Multiple DVs

46
Do strength gains occur at the same rate in men
as they do in women over a 6 mo. training period?
Measurements are taken at 0, 2, 4, 6 mo.
47
Types of Experimental Designs

• Simple True Experimental
• Complex True Experimental
• Quasi-Experimental

48
Characteristics of True Designs

• Manipulation (treatment)
• Randomization
• Control group
• Less control
• More real-world
• Program evaluation

49
Randomized posttest control group design

• R T Post
• R C Post

50
Randomized pretest-posttest control group design

• R Pre T Post
• R Pre C Post

51
Quasi-experimental Designs

• One group posttest-only design
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

52
Quasi-experimental Designs

• One group posttest-only design
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

53
Randomized posttest control group design

• R T Post
• R C Post

54
One group posttest-only design (One shot study)

• T Post

No control of IV threats Use?
55
Quasi-experimental Designs

• One shot study
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

56
Randomized pretest-posttest control group design

• R Pre T Post
• R Pre C Post

57
One group pretest-posttest design

• Pre T Post

• History
• Maturation
• Testing
• Instrument decay
• Regression

Use control group
58
Quasi-experimental Designs

• One shot study
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

59
Randomized posttest control group design

• R T Post
• R C Post

60
Non-equivalent control group design (Static
group comparison design)

• T Post
• C Post

• Selection bias

61
Quasi-experimental Designs

• One shot study
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

62
Randomized pretest-posttest control group design

• R Pre T Post
• R Pre C Post

63
Non-equivalent control group pretest-posttest
design

• Pre T Post
• Pre C Post

• Can check selection bias

64
Quasi-experimental Designs

• One shot study
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

65
Time series

• Pre Pre Pre Pre T Post Post Post Post

66
Quasi-experimental Designs

• One shot study
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

67
Quasi-experimental Designs

• One shot study
• One group pretest-posttest design
• Non-equivalent control group design
• Non-equivalent control group pretest-posttest
  design
• Time series
• Single subject designs (Case study)
• Developmental designs

68
Developmental Research Designs

• Longitudinal
• Powerful (within subject)
• Time consuming
• Attrition
• Testing effect

• Cross Sectional
• Less time consuming
• Cohorts problem

69
Choosing a Research Design

• Best addresses the problem
• Ethics
• Cost in time and money
• Validity (internal external)