RESEARCH DESIGN

1
RESEARCH DESIGN
I. Design Elements II. Threats to
Validity III. Research Designs IV. Design
Tradeoffs
2
DESIGN ELEMENTS

• Comparison Groups
• Randomization
• Treatments
• Time
• Observation/Measures

3
DESIGN ELEMENTS

• Comparison Groups

4
Comparison Groups

• Comparable groups of research participants
• Treatment Group
• Multiple Treatment Groups
• Control Group
• Placebo Group

5
DESIGN ELEMENTS

• Randomization

6
Randomization

• Separate from sampling issue
• Best way to get equivalence
• Other Means of Creating Comparison Groups
• Cluster
• Matched
• Cohort
• Convenience

7
DESIGN ELEMENTS

• Treatments

8
DESIGN ELEMENTS

• Time

9
DESIGN ELEMENTS

• Observation/Measures

10
Observations or Measures

• Pre-tests
• Post-tests
• Proxy Pre-tests
• Multiple Assessments

11
Design Elementsand Types of Design
12
Design Elementsand Types of Design

• An industrial psychologist wants to study the
  effects of soft classical music on the
  productivity of a group of bank loan processors
  working in an open office. At the beginning of
  the month, the psychologist meets with the
  employees to explain the study, gets their
  consent to play the music, and then begins to
  have music piped into the office. At the end of
  the month, the department head reports a 30.7
  increase in the number of loans processed over
  the prior month. "Soft music increases
  productivity," the psychologist concludes.

13
Internal Validity

• How confident are we that the conclusions reached
  in this study reflect the truth? Given our
  research design and measures, how confident are
  we in our results?
• Did the independent variable variable cause the
  dependent variable or are there potential
  alternative explanations?

14
Threats to Internal Validity

• Selection
• History
• Instrumentation

15
Threats to Internal Validity

• Testing
• Interactions with selection
• Maturation

16
Threats to Internal Validity

• Regression
• Equalization of treatments
• Differential Mortality

17
NONEXPERIMENTALDESIGNS

• Descriptive Research
• Archival Research
• O O O O
• or or
• O

18
PRE-EXPERIMENTALDESIGNS

• Post-test Only
• X O
• Pre-test/Post-test
• O X O

19
EXPERIMENTAL DESIGNS

• Post-test Only w/Control
• R X O
• R O

20
EXPERIMENTAL DESIGNS

• Post-test Only w/ Control
• Factorial Designs
• R X11 O
• R X12 O
• R X21 O
• R X22 O
• Factor 1 Timing of Orientation Training
  Factor 2 Delivery Method
• Level 1 First day of employment Level 1
  Classroom
• Level 2 Fourth day of employment Level 2
  Computer-based

21
2X2 Factoral Design
22
EXPERIMENTAL DESIGNS

• Pre-test/Post-test w/ Control

R O X O R O O
23
(No Transcript)
24
EXPERIMENTAL DESIGNS

• Solomon Four Group
• R O X O
• R O O
• R X O
• R O

25
Solomon 4 Group
26
SOLOMON FOUR-GROUP
27
EXPERIMENTAL DESIGNS
28
QUASI-EXPERIMENTALDESIGNS

• Nonequivalent Experimental Designs
• Proxy Pretest
• Separate Pre-Post Samples
• Cohort Design
• Double Pre-test Design
• Time Series Design
• Multiple Time Series Design

29
QUASI-EXPERIMENTALDESIGNS

• Any Experimental Design without Random Assignment
  to Groups
• e.g., Pre-test posttest w/ control
• N O X O
• N O O
• Because group assignment is nonrandom, the groups
  may be different and those group differences may
  affect outcomes.
• Non-Random assignment methods
• Cluster, Matched, Cohort, Convenience

30
QUASI-EXPERIMENTALDESIGNS

• Proxy Pretest
• N O1 X O2
• N O1 O2
• Separate Pre-Post Samples
• N O
• N X O
• N O
• N O

31
Cohort Design

• O X1 O 1st cohort
• O X2 O 2nd cohort
• O X2 O 3rd

32
QUASI-EXPERIMENTALDESIGNS
33
TIME SERIES DESIGNS

• O O O O X O O O O

34
Multiple Time Series

• N O O O O X O O O O
• N O O O O O O O O

35
Eliminating Validity Threats

• Design
• Measurement
• Post hoc Matching
• Qualitative Data
• Non-Equivalent DVs
• Statistical Analysis

36
Eliminating Validity Threats
37
Pre-test, Post-test, Non-Random Comparison
Group, Non-Equivalent Dependent Variables
38
What Is a Good Design?

• Theory-grounded
• Situational (adapted)
• Feasible
• Redundant
• Efficient

39
Design Considerations

• Purpose of Study
• Importance of Problem
• Constraints