Quasiexperimental Designs, Experimental Designs, and SingleSubject Designs

1
Quasi-experimental Designs, Experimental Designs,
and Single-Subject Designs

• OT 667
• July 1, 2002

2
Experimental Design

• Control Group
• Random Assignment
• Manipulate a variable
• (treat one group but not the other, OR use 2
  different treatments)

3
Control Groups

• Assume equivalence between control group and
  treatment group
• Help rule out extraneous effects of treatment
  which can affect change rather than the treatment
  itself (getting out of the house, receiving
  attention)
• When a control group is not possible..
• Compare treatments (problem when treatment of
  comparison has not been previously assessed using
  a control group)

4
Random Assignment

• Subjects have an equal change of being assigned
  to the control or treatment group
• Assures group equivalence
• What do we mean by group equivalence????

5
Intent to Treat

• A principle used when subjects are randomly
  assigned to groups but then, for various reasons,
  do not stay in their assigned group
• Can use on-protocol assessment, where the data
  are assessed including the patients in their
  respective groups.
• Group sizes are unequal
• Bias in favor of the treatment variable under
  study

6
An intent to treat analysis analyzes the data
with subjects outcomes included in the group to
which they were assigned. Makes it much harder
to find significant differences
7
Many researchers do both in protocol and intent
to treat analyses and compare the analyses. If
the results are the same, all is good. If not,
you have spent a lot of time and money for
equivocal results..
8
Manipulation of Variables

• Variable - predetermined treatment or
  intervention
• Active variable one that is manipulated by the
  researcher
• Attribute variable traits of subjects, such as
  age, diagnosis, gender, etc cannot be
  manipulated by the researcher
• Can manipulate one or more active variables at
  the same time
• Can also assign people with various attribute
  levels to receive the active variable

9
Blinding in Research
The purpose of blinding is to minimize bias in
the process of recording or collecting
information about the outcome of the treatment.
10
So who gets blinded?

• Subjects
• Persons administering the treatment
• Persons collecting the data
• Persons analyzing the data

11
Controlling for Differences within a Study

• Selection of homogenous subjects
• Blocking
• Matching
• Subjects as their own control
• Analysis of co-variance

12
Does the experimental treatment really cause the
observed change in the dependent variable?This
question addresses the internal validity of a
study.
13
Threats to Internal Validity

• History
• Maturation
• Attrition
• Testing
• Instrumentation
• Statistical Regression
• Selection
• Ambiguity about Direction of Causal Influence
• Treatment Diffusion
• Demoralization of respondents receiving treatment
  assumed to be less desirable

14
Can the results of the study be generalized to
the population of whom the sample is
representative in a given study? This question
asks about the external validity of a study.
15
Threats to External Validity

• Interaction of Treatment and Selection
• Interaction of Treatment and Setting
• Interaction of Treatment and History

16
Quasi-experimental Research
17
Quasi-experimental designs are those studies
which do not use random assignment for
comparisons. Many such studies also do not use a
control group
18
Rationale for Using Quasi-Experimental Design

• Clinical limitations space, time, money,
  equipment/personnel needed to provide
  treatments.
• Use of quasi-experimental design can be a step to
  collect data to apply for funding for
  experimental design
• Ethical issues random assignment and use of a
  control group are not always acceptable to the
  population or sample under study

19
Kinds of Quasi-Experimental Designs

• One group pretest-posttest design
• Time Series design
• Nonequivalent pretest-posttest design
• Nonequivalent posttest only design

20
Single-Subject Design
21
Single subject designs are a controlled
experimental approach to the study of a single
case or small group of subjects. The subject is
used as his/her own control to demonstrate the
effect of a treatment over time.
22
Characteristics of Single Subject Designs

• Clear definition of the independent and dependent
  variables
• The design is broken into phases
• Data is collected in a series of repeated
  measures across the phases

23
When are Single Subject Designs Used?

• To study treatment of rare conditions
• To study a new treatment protocol
• To document the outcomes of treatment in a
  systematic manner
• When preparing a new treatment protocol to go on
  to a larger study and you need to work out the
  problems with the intervention

24
Phases of Studies

• Baseline phase
• Looking for stability
• Looking for trends
• Intervention phases
• Phases where treatment is administered

25
The target behavior

• The target behavior is the characteristic you
  are attempting to change via the independent
  variable.
• Measurement of the target behavior can be done
  via counts of observed behavior, via tests such
  as ROM, BP, etc.

26
Types of Measures

• Frequency of observed behavior
• Duration of observed behavior
• Magnitude or degree of change in a behavior

27
Kinds of Single Subject Designs

• Withdrawal designs
• Alternating treatment designs
• Changing criterion designs
• Multiple treatment designs
• Multiple baseline designs
• Across subjects
• Across settings
• Across behaviors

28
Withdrawal Designs

• Designs where the treatment is not given or
  withdrawn in specified phases
• Baseline (no treatment) is measure first
• A treatment phase occurs where behavior is still
  measured
• The treatment is then withdrawn and behavior
  measured
• AB, ABA, or ABAB are common forms

29
Multiple Baseline Designs

• Collection of data across at least three series
  of events
• Events can be 3 subjects, 3 settings, 3
  behaviors, and so forth
• Baseline is gathered in all 3 conditions.
• When baseline behaviors are stable across all
  conditions, intervention starts in one condition
• Institution of the intervention is initiated
  sequentially across the other two conditions

30
Data Analysis

• Is done by visual inspection of the data
• Changes in trend, level and slope of the behavior
  are inspected
• Evidence of stability or lack of it affects the
  interpretation of how the treatment worked

31
Statistical Analyses

• Although visual inspection is the most common
  method of data analysis, some statistical
  analyses are done as well
• Mean scores, two standard deviation band method
  and others may be used to analyze data

32
Reliability Issues

• Inter-rater reliability is collected across all
  phases of data
• Inter-rater agreement is calculated by various
  methods, including total reliability, point by
  point reliability, and others
• Reliability calculations are critical to insure
  the behaviors being counted are truly present

33
Concerns about Single Subject Designs

• Hard to generalize from such a small sample of
  subjects
• Some designs present confounds of effects, such
  as multiple or alternating treatment designs