Research Methods in Communication Disorders

1
Research Methods in Communication Disorders

• Slide Set 3

2
V. Research Design Principles

• There are as many research designs as there are
  hypotheses to be tested.
• Two major classes of research design that have
  broad applicability in communicative disorders
  research include group designs and single-subject
  designs.
• For both experimental and descriptive research,
  group designs can be classified as
  between-subjects, within-subjects, or mixed.

3
A. Group Designs

• In group designs, one or more groups of subjects
  are exposed to one or more levels of the
  independent variable, and the average performance
  of the group of subjects on the dependent
  variable is examined to determine the
  relationship between the independent and the
  dependent variables.

4
A. Group Designs

• In between-subject designs, different groups of
  subjects are compared to each other.
• In within-subject designs the same group of
  subjects is compared in different situations.
• Mixed designs include both types of comparisons
  in the same study.

5
1. Between-Subjects Designs Experimental

• In experimental between-subject designs,
  different groups of subjects are exposed to
  different treatments or levels of the independent
  variable.
• The independent variable or experimental
  treatment is applied to one group of subjects
  (the experimental group) but not applied to
  another group of subjects (the control group).
• The difference between the performance of the two
  groups is taken as an index of the effect of the
  independent variable on the dependent variable.

6
a. Random Selection Designs

• There are four types of between-subject designs
  random selection designs, random assignment
  designs, matched group, and natural group
  designs.
• The natural group design is the most commonly
  used in communication disorders research.
• With random selection designs, two groups are
  randomly selected from the same population.
• Variables such as age, gender, and education are
  controlled by random selection of subjects.

7
b. Random Assignment Designs

• Random assignment designs overcome the difficulty
  of access to an entire population.
• When only a restricted population of subjects is
  available, they can be randomly assigned to one
  group or another.
• Like the random selection procedure, subject
  variables are controlled by allowing them to vary
  randomly.
• Except for the population from which subjects are
  selected, there is no difference between random
  selection and random assignment designs.

8
c. Matched Group Designs

• The subject variability that may be a problem in
  random assignment designs can be overcome by
  matched group designs.
• In a simple matched group design, one or more
  variables that may affect the dependent variable
  can be held constant between groups by matching
  the groups on those variables.
• The two groups can be matched on the dependent
  variable prior to treatment.
• The two groups can be matched also on the
  independent variable prior to training.

9
c. Matched Group Designs

• When matching to either a dependent or
  independent variable, it is important that the
  groups have the same mean and distribution of the
  scores, or ages, around the mean.
• Restrictions in the range of variation of matched
  variables will enhance the observed effects of
  the independent variable.

10
d. Naturalistic Group Designs

• In this design, the groups are selected from two
  different populations.
• In the previous designs, the two groups were
  selected from the same population and the
  independent variable was different treatment of
  the two groups.
• In natural group design, the independent variable
  is a difference between groups created by
  nature that exists prior to the selection of the
  groups.
• The effect of this independent variable is
  studied.

11
e. Combined Natural Group and Matched Group Design

• In studies in which communication disorder groups
  are compared with normal control groups, the
  natural group design must be used because the two
  groups come from different populations.
• This is also true of comparisons between
  communication disorder groups that differ in type
  or severity of disorder, or in other variables
  such as age and gender.

12
e. Combined Natural Group and Matched Group Design

• Natural/matched group designs can also be used to
  compare two groups of communication disorders
  where the natural groupings are defined by
  differences in experience.
• Some limitations of natural/matched group designs
  are that matching groups restricts the
  applicability of the results to groups with the
  same characteristics, it is difficult to match
  natural groups on all relevant variables, and a
  difference between natural groups does not prove
  that the independent variable has caused the
  difference.

13
2. Between-Subjects Designs Descriptive

• Between-subject designs are also common in
  descriptive research.
• In descriptive between-subject designs, different
  groups of subjects are compared with each other
  with regard to their performance on some
  criterion variable.
• Examples of between-subject descriptive research
  include comparative research, cross-sectional
  developmental research, and surveys that compare
  the responses of different groups.

14
2. Between-Subjects Designs Descriptive

• Comparative research involves the description of
  dependent variable differences between groups of
  subjects who differ with respect to some
  classification variable (e.g., children with
  palatal clefts vs. children without palatal
  clefts).
• Cross-sectional development research uses a
  between-subject design because separate groups of
  subjects who differ with respect to age are
  compared.

15
2. Between-Subjects Designs Descriptive

• Some surveys are conducted for the purpose of
  comparing the interview or questionnaire
  responses of subjects who fall into different
  classifications (e.g., hearing-aid users vs.
  nonusers).
• Between-subject descriptive research designs may
  be bivalent, in which cases the classification
  variable is broken down into two mutually
  exclusive categories (e.g., laryngectomees vs.
  speakers with normal larynges).

16
2. Between-Subjects Designs Descriptive

• Between-subject descriptive designs can also be
  multivalent, in which case the classification
  variable is divided into categories that are
  ordered along some continuum (e.g., mild vs.
  moderate vs. severe hearing loss).
• Between subject descriptive designs can also
  include comparisons of subjects who are
  simultaneously categorized with respect to more
  than one classification variable (e.g., male vs.
  female mild vs. moderate vs. severe mental
  retardation).

17
2. Between-Subjects Designs Descriptive

• The first step in between-subject descriptive
  research is to define criteria for selecting
  subjects from each category of the classification
  variable.
• Classifications must be constructed that are
  mutually exclusive, that is, subjects should fall
  into only one category with regard to each
  classification variable.
• Sometimes it may be necessary to use several
  measures in a battery of selection test in order
  to classify subjects.

18
2. Between-Subjects Designs Descriptive

• The second step to between-subject descriptive
  research is the attempt to equate subjects on
  extraneous variables.
• Because subjects cannot be assigned randomly to
  the various classification, equivalence of groups
  on all extraneous variables is quite difficult to
  achieve.
• The best alternative is to try to minimize group
  differences on extraneous variables known to
  correlate with the dependent variable.

19
1. Between-Subjects Designs Descriptive

• A common way to do this is to match the various
  groups on the extraneous variables known to be
  most highly correlated with the dependent
  variable.
• Both overall matching or pair matching can be
  used.

20
2. Within-Subjects (Repeated Measures) Designs

• In repeated measures design, the levels of the
  independent variable are varied within a single
  group of subjects.
• They are used in communication disorders research
  when there are not enough subjects available for
  two independent groups, when it is difficult to
  match relevant variables in two independent
  groups, or when it is more efficient to carry out
  the experimental procedures with one group.

21
2. Within-Subjects (Repeated Measures) Designs

• The basic rules of the within-subjects design are
  to assess the dependent variable twice in a
  single group of subjects.
• The difference between two assessments
  demonstrates the effect of the independent
  variable.
• Subject variables such as age, gender, and
  education do not have to be controlled because
  the same group is used of both values of the
  independent variable.

22
2. Within-Subjects (Repeated Measures) Designs

• Limitations of repeated measurement designs
  include order effects.
• Order effects may take the form of a practice
  effect that improves performance or a fatigue or
  boredom effect that impairs performance.
• When repeated measurement designs are used to
  assess the effects of training, there is a lack
  of control for the possibility that improvement
  might have occurred without training.

23
2. Within-Subjects (Repeated Measures) Designs

• A control procedure for order effects is to
  counterbalance the order of presentation.
• However, counterbalancing cannot be used to
  control for order effects in training studies
  because the pretraining must always be the first
  measure.
• To control for the possibility of improvement
  without training, the independent group and
  repeated measure designs can be combined, with a
  trained and untrained group tested before and
  after training.

24
3. Between-Group and Within-Subjects (Mixed)
Designs

• In many research studies, more than one
  independent variable is considered.
• The effects of two or more independent variables
  on a dependent variable may be examined.
• In other cases, one independent variable is
  studied with a between-subjects comparison, and
  the other independent variable is studied with a
  within-subject comparison.
• This mixed design incorporates both the
  between-group and the within-subjects tactics.

25
B. Single-Subject Designs

• Single-subject designs focus on the individual
  behavior of subjects rather than considering the
  average performance of a group of subjects.
• Single case research designs are appropriate for
  communication disorders research because they are
  intended to demonstrate that interventions cause
  changes in behavior.
• Single-subject designs can examine the behavior
  of more than one person, but the data of each
  person will be evaluated individually rather than
  as part of a group average.
• A number of different designs are available.

26
1. ABA, ABAB Designs

• The most basic single subject designs are
  withdrawal and reversal designs.
• After several baseline (A) measures, a treatment
  is given until the target behavior (B) changes.
• Then the treatment is taken away (withdrawal
  design) or nontarget behaviors are reinforced
  (reversal design) until the target behavior
  returns to baseline, and the treatment (B) is
  usually given once more.
• This procedure is designed to prove that the
  treatment caused the change in behavior.

27
1. ABA, ABAB Designs

• When there is baseline, treatment, and either
  withdrawal or reversal , it is an ABA design.
• When the treatment is is given again after the
  withdrawal or reversal, it is an ABAB design,
  which provides more complete proof of the
  effectiveness of treatment.
• These designs should only be used in
  communication disorders research only when
  treatment effects may be temporary or can be
  reversed, and may be ethically inadvisable.

28
2. Multiple Baseline Designs

• Another way to demonstrate the effect of a
  treatment is the multiple baseline design.
• Treatment effects are first demonstrated for one
  dependent variable, and then for two or more
  additional dependent variables.
• The additional variables can be target behaviors,
  conditions, or subjects.

29
2. Multiple Baseline Designs

• In the multiple baseline design across behaviors,
  three or more target behaviors (e.g., the
  articulation of three different phonemes) are
  selected.
• Baseline measures (A) are taken for all three
  behaviors.
• Then treatment of the first target behavior (B)
  is begun, while baseline measures are continued
  for the other two behaviors.

30
2. Multiple Baseline Designs

• When the treatment effect for B has reached the
  desired level, treatment of the second target
  behavior (C) is begun, while baseline measures
  are continued for the other behavior.
• Finally, when target behavior C reaches the
  desired level, training of a third target
  behavior (D) can be begun, and continued until it
  reaches the desired level.
• If the baseline behaviors change only when the
  appropriate treatment is introduced, there is
  evidence of a causal relationship.

31
2. Multiple Baseline Designs

• In the multiple baseline design across
  conditions, a single target behavior is trained
  in three or more different training conditions.
• For example, the treatment of nonfluencies might
  be carried out in a research laboratory (B), at
  home (C), and in a public place (D).
• The sequence of changes from baseline measures
  (A) to the three conditions follows exactly the
  same sequence as those in the multiple baseline
  design across behaviors.

32
2. Multiple Baseline Designs

• In the multiple baseline design across subjects,
  a baseline (A) is established for three or more
  subjects (B, C, and D), and then the treatment of
  a target behavior is introduced at different
  times for the subjects, following the sequence
  described for the multiple baseline design across
  behaviors.
• The multiple baseline design across subjects is
  frequently used in communication disorders
  research.

33
3. Multiple Treatment Designs

• The effects of two or more different treatments
  can be compared with several single subject
  designs.
• In some cases the treatments are given one after
  the other, and in other cases the treatments are
  trained at the same time.
•  The simple ABAB design can be extended to ABABC,
  ABABAC, ABABCD, and so forth, where new
  treatments are introduced after training with the
  first treatment.

34
3. Multiple Treatment Designs

• Such designs are called multi-treatment designs.
• The usual multi-treatment design involves a
  preplanned comparison of methods.
• Baselines can be taken between each treatment
  (ABACAD), and a theoretically optimal sequence of
  different treatments can be presented, such as
  training in imitating speech sounds followed by
  training in naming the speech sounds in words
  represented by pictures.

35
3. Multiple Treatment Designs

• The alternating treatment design, also called the
  multiple schedules design, presents the
  treatments (usually only two) in each session in
  counterbalanced order, or in alternating
  sessions.
• It is not necessary to take baseline measures
  because the treatment effects are compared, but
  it is advisable to take the baseline measures to
  demonstrate the magnitude of the effects.
• The simultaneous treatment design, also called
  the concurrent schedule design, is difficult to
  conceptualize.  

36
3. Multiple Treatment Designs

• The difficulty with multiple treatment designs in
  communication disorder research is assessing the
  possible carryover effects from one treatment to
  another in the treatment of nonreversible
  behaviors.
• This difficulty may be overcome by using
  different target behaviors for each treatment,
  e.g., treatment B for one misarticulated phoneme
  and treatment C for a second misarticulated
  phoneme.

37
4. Generalization Designs

• In multiple baseline designs across behaviors and
  conditions, generalization from one target
  behavior to another or from one condition to
  another is undesirable, because the baselines for
  the untrained behaviors or conditions will
  change. In practical training studies, however,
  it is hoped that training effects will not be
  confined to the exact targets used in training,
  but will generalize to nontrained behaviors
  (e.g., a phoneme correctly articulated in a set
  of training words will be articulated correctly
  in non-trained words).

38
4. Generalization Designs

• There are several ways of assessing
  generalization to nontrained behaviors.
• One simple method is to probe nontrained
  behaviors during baseline, at intervals during
  training, and after training.
• If training is done in a laboratory setting, it
  is important to assess generalization, that is,
  carryover, to a normal conversational setting.

39
B. Single-Subject Designs

• Single subject designs have some of the
  advantages of group designs and some of the
  advantages of observational and case study
  designs.
• They provide direct quantitative measures of the
  behaviors studied, being averaged neither across
  subjects in groups nor across studies.
• Experimental conditions are rigorously controlled
  to obtain information about causal relationships
  between independent and dependent variables.

40
B. Single-Subject Designs

• Large groups of subjects are not needed.
• Information is obtained for individuals rather
  than groups.
• The information may have direct practical
  applications. 
• Single subject designs have their own inherent
  limitations.
• Stable baselines may be difficult to establish.
• If the treatment is not immediately effective in
  changing the baseline behavior, it may be
  difficult to demonstrate causal relationships.

41
B. Single-Subject Designs

• Reversal designs cannot be used if treatment
  effects do not or should not reverse.
• Multiple baseline designs cannot be used if
  treatment effects generalize to nontreated
  behaviors.
• Multiple treatment designs may yield ambiguous
  results if treatment effects carry over and if
  differences between treatment effects are not
  clear-cut.

42
B. Single-Subject Designs

• The very rigid specification of target behaviors,
  treatments, and control procedures may make the
  treatment too artificial for direct application
  to clinical intervention.
• The treatment may change the target behaviors
  only in the experimental condition and not in
  natural communication situations.
• Finally, there is the problem of generalizing the
  results.
• There is no way of predicting that all subjects
  of the same type will show the same treatment
  effects.
•  
•