Seth M' Noar, Ph'D'

1
Experiments

• Seth M. Noar, Ph.D.
• CJT665
• Department of Communication
• University of Kentucky

2
2 Broad Types of Research

• Surveys Observations that occur in natural
  settings
• Often operate in real world settings
• But, causality is difficult to demonstrate
• Experiments Studies in which data are collected
  under conditions where 1 or more variables are
  manipulated
• Often operate in an artificial setting
• But, causality is easier to demonstrate

3
2 Broad Types of Research (contd)

• These two types are complementary in many ways
  ideal program of research takes advantage of both
• Research design similar to statistics should
  choose the best and most appropriate research
  design
• However, resources and other factors may help to
  dictate what design is feasible
• Large randomized trials cost a lot of money
• Some perfect experiments are unethical to
  conduct

4
Experiments

• True experiment involves
• IV manipulated variable
• DV variable expected to be influenced by the
  manipulation of the IV
• Pool of participants who are randomly assignment
  to groups (e.g., control and experimental)
• Note Random assignment distinguishes
  experimental from quasi-experimental designs

5
Experimental Designs

• 5 Types of Designs
• Pretest-Posttest Control Group Design
• Posttest Only Control Group Design
• Solomon Four Group Design
• Factorial Designs
• Blocking Designs
• There are also many variations on these designs

6
Pretests

• Pretest advantages
• Demonstrate that randomization worked that
  groups are equivalent at baseline
• If groups are different in some way, can
  statistically control for this since we have
  pretest data
• Gives one data to include/exclude individuals
  from experiment, and use in other ways
• However, are their some cases where having no
  pretest is better?

7
(No Transcript)
8
(No Transcript)
9
Pretests (contd)

• Disadvantages
• Can clue participants as to the purpose of the
  study
• Can sensitize individuals to the IV
• Opens up more potential threats to internal
  validity

10
Solomon 4 Group Design

• Combines the previous 2 designs together
• Can evaluate several effects within 1 experiment
• Allows one to examine effects of the IV
  manipulation (treatment)
• Allows one to see if the pretest had an effect
• Allows one to examine effects of the IV
  manipulation (treatment) above and beyond any
  pretest effects (interaction)
• Note One is essentially treating the pretest as
  an additional IV

11
(No Transcript)
12
Solomon as 2 x 2 Factorial
Pretest
No Pretest
Treatment
Treatment Main Effect
No Treatment
Pretest Main Effect
13
Factorial Designs

• Factorial Designs involve more than 1 IV
• For example, a 2 X 2 design
• Sensation Value Low, High
• Need for cognition Low, High
• What analysis tool does one use for a factorial?

14
2 x 2 Factorial
High NFC
Low NFC
High SV
Low SV
15
Blocked / Mixed Designs

• Blocked or mixed designs are appropriate when we
  have factors we are interested in that we cannot
  manipulate
• Examples IQ, Gender, Race
• Key feature is this participants are sorted into
  categories (blocked) according to some variable
  and then are assigned to groups

16
(No Transcript)
17
Validity

• 2 major types of validity in experiments
• Internal validity extent to which one can make
  causal inferences about the IV-DV relationship
• External validity The extent to which one can
  generalize findings of the study to the
  population and to other contexts
• These are generally mutually exclusive, and have
  mutually exclusive threats

18
Threats to External Validity

• Key issue here is generalizability extent to
  which findings can be expected to be replicated
  with other samples, at other times and places,
  and under other conditions
• Thus, non-random sampling, interactions between
  IVs and the context in which they are delivered,
  and other factors can affect external validity
• Often we do NOT have true random samples
• Best we can do is attempt to replicate our
  findings in many samples as they become available

19
Threats to Internal Validity

• History An event takes place between pre and
  post test (MM example)
• Maturation Individuals mature in some way
  between pre and post test (Anti-marijuana
  example)
• Testing Scores on post test change because of
  exposure to pretest
• Instrumentation Scores on post test change
  because of unreliable measurement (not actual
  change). (This happens more with subjective
  measures).
• Statistical Regression Unreliability or
  measurement error results in scores away from the
  mean at time 1, and then back to the mean at time
  2

20
Threats to Internal Validity (contd)

• 6. Selection If posttest scores differ
  across groups, this may be due to selection
  factors that went into creating the groups
  (gender diffs w/ college students)
• 7. Experimental Mortality Differences that
  occur in the posttest that are due to
  differential attrition across groups
• 8. Selection Interactions If selection
  factors went into creating the groups, these
  specially selected groups may encounter
  differential history, maturation, testing, or
  other effects.
• Note These threats, when they occur, make it
  very difficult to tell if it was in fact the IV
  that lead to changes in the DV