Methodology

1
Methodology
2
Research Methodology

• Research methods are specific procedures used to
  gather and analyze research data of a particular
  paradigm (e.g. experimental versus descriptive
  quantitative versus qualitative).
• They are concerned with the protection of human
  participants, research design principles,
  sampling, data collection, instrumentation, and
  statistical analysis.
• Research participants are essential to the
  conduct of research and to the progress and
  discoveries researchers make in a variety of
  fields.

3
Research Ethics

• The relationship between researchers and
  participants is critical and should be based on
  accurate information, trust, and respect.
• Throughout history, harmful research has been
  conducted on unwilling human participants.
• Over the years, public outrage has led to the
  development of a number of advisory commissions
  and codes of research ethics.
• As recently as 2000, the Office of Human Research
  Protections (OHRP) was established within the US
  Department of Health and Human Services to
  provide leadership for all 17 Federal agencies
  that carry out research involving humans.

4
Guiding Principles

• The three fundamental ethical principles that
  guide the ethical conduct of research involving
  human participants are
• respect for persons (autonomy)
• beneficence
• justice
• You will learn more about human subjects
  protection when you complete your tutorial.

5
Respect for Persons

• Two ethical standards are contained within this
  principle
• Individuals should be treated as autonomous
  agents
• Persons with diminished autonomy may need
  additional protections.
• An autonomous individual is one capable of
  deliberation about personal goals and acts, as
  well as opinions and choices.

6
Respect for Persons

• Prospective research participants must be given
  the information needed to determine whether or
  not to participate in a research study.
• There should be no pressure to participate and
  ample time to decide.
• Individuals must enter into research voluntarily
  and with adequate information.
• This is called informed consent.

7
Respect for Persons

• When an individual or a class of participants is
  considered incapable of informed decision making
  (e.g., children, people with severe developmental
  disorders or dementias), respect for persons
  requires giving him/her the opportunity to
  choose, to the extent they are able, to
  participate or not participate in the research
  activity.
• In some instances, respect for persons may
  require seeking permission of other parties, such
  as a parent or legal guardian.

8
Beneficence

• Another ethical standard has to do with efforts
  to secure the well-being of human participants
  and to protect them from harm.
• The principle of beneficence obligates the
  researcher to maximize possible benefits and
  minimize possible harm.
• The researcher must decide when it is justifiable
  to seek certain benefits despite inherent harms
  or risks.
• No individual should be intentionally injured.

9
Justice

• The ethical considerations of risks versus
  benefits leads to the question of justice.
• This principles requires that participants be
  treated fairly and without bias.
• The concept of justice may be questioned when
  deciding who will be given an opportunity to
  participate, who will be excluded, and the
  reasons for exclusion.
• When selecting participants for research,
  researchers are responsible for ensuring that
  selection is equitable.

10
Justice

• Research should only involve persons from groups
  that are directly related to the problem being
  studied and that are likely to benefit from the
  research.
• To be fair, the researcher must ask Are some
  classes of persons being selected simply because
  of their availability, their compromised
  position, or their vulnerability.
• Vulnerable research participants are persons who
  are relatively or absolutely incapable of
  protecting their own interests.
• Vulnerable populations include

11
Justice

• Children
• Individuals with questionable capacity to consent
• Prisoners
• Fetuses and pregnant women
• The terminally ill
• Students/employees
• Comatose patients
• The researcher should be cognizant of the special
  problems of research involving vulnerable
  populations.

12
Justice

• It is important to justify the proposed
  involvement of these populations in the research,
  and include additional safeguards for their
  safety and welfare.
• Researchers must also be careful not to
  overprotect vulnerable populations and exclude
  them from participating in research in which they
  wish to participate, particularly where the
  research involves therapies for conditions in
  which there are not available treatments.

13
Informed Consent

• Once the researcher has a carefully defined
  research question, a valid design, and protocol
  for a research project, it is time to plan for
  the informed consent of those invited to
  participate.
• Informed consent is a legal, regulatory, and
  ethical concept based on respect for the
  individual and the individuals autonomy and
  right to define his/her own goals and make
  choices designed to achieve those goals.
• This right applies to all types of medical
  intervention and clinical research.

14
Informed Consent

• In research, however, informed consent is more
  than just obtaining a signature of the potential
  research participant.
• It is a process that involves
• conveying accurate information about the study
  and its purpose
• disclosing known risks, benefits, alternatives,
  and procedures
• answering questions and
• enabling the potential participant to make an
  informed decision about whether to participate.

15
Informed Consent

• Finally, participants must be informed that even
  after they have made a voluntary agreement to
  participate in the study, they may withdraw such
  agreement at any time without penalty.
• If a person is unable to provide his/her own
  consent, a legally authorized representative can,
  in some cases, give permission for participation
  in research.
• A legally authorized representative is a legal
  guardian a parent (for children only) and in
  some cases, a validly designated durable power of
  attorney for health care.

16
Informed Consent

• Because children have not attained an age at
  which they can consent to research or treatment,
  the parent or guardian may provide permission
  for the child to participate in the study.
• In most cases, the child must indicate
  willingness to participate by assenting to the
  study.
• Edinboro University has a Human Subjects Review
  Board (HSRB) which reviews research applications
  of planned research studies by faculty, staff,
  and students.
• The policy, the application form, the informed
  consent guidelines, and the informed consent
  outline form are available at http//www.edinboro.
  edu/departments/provost/human_subjects_review_boar
  d.dot

17
Research Design Principles

• There are as many research designs as there are
  hypotheses to be tested.
• Two major classes of research design that have
  broad applicability in communicative disorders
  research include group designs and single-subject
  designs.
• For both experimental and descriptive research,
  group designs can be classified as
  between-subjects, within-subjects, or mixed.

18
Groups Designs

• In group designs, one or more groups of subjects
  are exposed to one or more levels of the
  independent variable, and the average performance
  of the group of subjects on the dependent
  variable is examined to determine the
  relationship between the independent and the
  dependent variables.
• In between-subject designs, different groups of
  subjects are compared to each other.

19
Group Designs

• In within-subject designs the same group of
  subjects is compared in different situations.
• Mixed designs include both types of comparisons
  in the same study.
• In experimental between-subject designs,
  different groups of subjects are exposed to
  different treatments or levels of the independent
  variable.

20
Between-Subjects Designs

• The independent variable or experimental
  treatment is applied to one group of subjects
  (the experimental group) but not applied to
  another group of subjects (the control group).
• The difference between the performance of the two
  groups is taken as an index of the effect of the
  independent variable on the dependent variable.
• There are four types of between-subject designs
  random selection designs, random assignment
  designs, matched group, and natural group
  designs.

21
Random Selection Designs

• With random selection designs, two groups are
  randomly selected from the same population.
• Variables such as age, gender, and education are
  controlled by random selection of subjects.
• For example, two groups of 100 children are
  randomly selected from the total population of
  children with language disorders.
• The independent variable, or experimental
  treatment, is vocabulary training.
• The two levels of the treatment are vocabulary
  training and no vocabulary training.

22
Random Selection Designs

• The experimental group is given extensive
  training in which new words are introduced in the
  context of stories.
• The other group, the control group, is given no
  training.
• At the completion of the training, both groups
  take a vocabulary test that includes new words.
• The score on the vocabulary test, or the measure
  of the effect of treatment, is the dependent
  variable.
• Subject variables such as age, gender, and
  education of the children are controlled by
  allowing them to vary randomly.

23
Random Selection Designs

• Situation variables such as testing room, time of
  testing, and the person giving the vocabulary
  test are held constant.
• The difference between the two groups in
  vocabulary test scores is evaluated by
  statistical analysis to estimate the effect of
  training in defining words on the vocabularies of
  children with language disorders.

24
Random Assignment Designs

• Random assignment designs overcome the difficulty
  of access to an entire population.
• When only a restricted population of subjects is
  available, they can be randomly assigned to one
  group or another.
• Like the random selection procedure, subject
  variables are controlled by allowing them to vary
  randomly.
• Except for the population from which subjects are
  selected, there is no difference between random
  selection and random assignment designs.

25
Matched Group Designs

• The subject variability that may be a problem in
  random assignment designs can be overcome by
  matched group designs.
• In a simple matched group design, one or more
  variables that may affect the dependent variable
  can be held constant between groups by matching
  the groups on those variables.
• The two groups can be matched on the dependent
  variable prior to treatment.
• The two groups can be matched also on the
  independent variable prior to training.

26
Matched Group Designs

• For example, in the vocabulary training study,
  the two groups could be matched on the basis of a
  vocabulary test given prior to training.
• Then, any differences between groups in
  vocabulary after training can be attributed to
  the independent variable, training versus no
  training.
• If two groups could not be matched on vocabulary
  prior to training, they could be matched in age,
  gender, education, and/or cultural background to
  control the effects of these variables on the
  dependent variable.

27
Matched Group Designs

• When matching to either a dependent or
  independent variable, it is important that the
  groups have the same mean and distribution of the
  scores, or ages, around the mean.
• The matched group design is very appropriate for
  communication disorders research, because the
  relatively small populations available to most
  researchers make it very difficult to select
  comparable groups by random assignment.
• What is sacrificed in the simple matched group
  design, as compared with the random selection
  design, is the ability to generalize findings to
  the entire population.

28
Naturalistic Group Designs

• In the previous designs, the two groups were
  selected from the same population and the
  independent variable was different treatment of
  the two groups.
• In this design, the groups are selected from two
  different populations.
• The independent variable is a difference between
  groups created by nature that exists prior to
  the selection of the groups.
• The effect of this independent variable is
  studied.

29
Naturalistic Group Designs

• For example, a randomly selected group of boys
  can be compared with a randomly selected group
  for girls in their performance on a vocabulary
  test to determine whether girls have better
  vocabularies than boys.
• The independent variable is gender, and dependent
  variable is once again vocabulary score, and all
  other variables are uncontrolled.

30
Natural Group Matched Group Design Combined

• In studies in which communication disorder groups
  are compared with normal control groups, the
  natural group design must be used because the two
  groups come from different populations.
• This is also true of comparisons between
  communication disorder groups that differ in type
  or severity of disorder, or in other variables
  such as age and gender.
• Since the combined matched group and natural
  group design is so important in our discipline,
  several examples will be given.

31
Natural Group Matched Group Design Combined

• Lets say it is important to determine whether
  children with phonological disorders also have
  higher level language disorders.
• Higher level language skills of a natural group
  of children with phonological disorders and a
  natural group of children without phonological
  disorder would be compared.
• The independent variable would be the presence of
  phonological disorders.

32
Natural Group Matched Group Design Combined

• The dependent variable would be a measure of
  higher level language skill (.e.g., a vocabulary
  test or a grammar test).
• The control group of children without
  phonological disorders would be selected to match
  the phonological disorder group in relevant
  variables such as age, gender, education, and
  cultural background.

33
Natural Group Matched Group Design Design

• With all relevant variables controlled, a
  difference between groups on the measure of
  higher level language skill would provide
  evidence that children with phonological
  disorders do have higher level language
  disorders.
• However, because the difference could be
  attributable to variables other than phonology
  per se (i.e., organic or environmental factors),
  one could not conclude that there is a cause and
  effect relationship between phonological
  disorders and higher level language disorders.

34
Natural Group Matched Group Design Design

• All that can be concluded is that children with
  phonological disorders tend to have higher level
  language disorders, for whatever reason.
• Natural/matched group designs can also be used to
  compare two groups of communication disorders
  where the natural groupings are defined by
  differences in experience.
• A study might be designed to assess the effects
  of different methods of training children with
  congenital sensorineural hearing impairment.

35
Natural Group Matched Group Design Combined

• A group trained by auditory oral methods and a
  group trained by total communication methods
  would be compared in educational achievement.
• The independent variable would be the method of
  language training, and the dependent variable a
  measure of educational achievement especially
  adapted for children who are hearing impaired.
• Variables controlled by matching might include
  age, gender, years of education, cultural
  background, amount of hearing impairment, and
  time of beginning language training.

36
Natural Group Matched Group Design Combined

• However, it is often difficult to match natural
  groups on all relevant variables and
• A difference between natural groups does not
  prove that the independent variable has caused
  the difference.

37
Descriptive Between-Subjects Designs

• Between-subject designs are also common in
  descriptive research.
• In descriptive between-subject designs, different
  groups of subjects are compared with each other
  with regard to their performance on some
  criterion variable.
• Examples of between-subject descriptive research
  include comparative research, cross-sectional
  developmental research, and surveys that compare
  the responses of different groups.

38
Descriptive Between-Subjects Designs

• Comparative research involves the description of
  dependent variable differences between groups of
  subjects who differ with respect to some
  classification variable (e.g., children with
  palatal clefts vs. children without palatal
  clefts).
• Cross-sectional development research uses a
  between-subject design because separate groups of
  subjects who differ with respect to age are
  compared.

39
Descriptive Between-Subjects Designs

• Between-subject descriptive research designs may
  be bivalent, in which cases the classification
  variable is broken down into two mutually
  exclusive categories (e.g., laryngectomees vs.
  speakers with normal larynges).
• Between-subject descriptive designs can also be
  multivalent, in which case the classification
  variable is divided into categories that are
  ordered along some continuum (e.g., mild vs.
  moderate vs. severe hearing loss).

40
Descriptive Between-Subjects Designs

• Between subject descriptive designs can also
  include comparisons of subjects who are
  simultaneously categorized with respect to more
  than one classification variable (e.g., male vs.
  female mild vs. moderate vs. severe mental
  retardation).
• The first step in between-subject descriptive
  research is to define criteria for selecting
  subjects from each category of the classification
  variable.

41
Descriptive Between-Subjects Designs

• Classifications must be constructed that are
  mutually exclusive, that is, subjects should fall
  into only one category with regard to each
  classification variable.
• For example, in a comparison of patients with
  cochlear hearing loss and patients with
  conductive hearing loss, all subjects must fit
  the definition of only one of the two groups.
• Patients who were found to have both a cochlear
  and a conductive component to their losses would
  have to form a third comparison group, that is,
  patients with mixed hearing losses.

42
Descriptive Between-Subjects Designs

• The second step to between-subject descriptive
  research is the attempt to equate subjects on
  extraneous variables.
• Because subjects cannot be assigned randomly to
  the various classification, equivalence of groups
  on all extraneous variables is quite difficult to
  achieve.
• The best alternative is to try to minimize group
  differences on extraneous variables known to
  correlate with the dependent variable.

43
Within-Subjects (Repeated Measures) Designs

• In repeated measures design, the levels of the
  independent variable are varied within a single
  group of subjects.
• They are used in communication disorders research
  when there are not enough subjects available for
  two independent groups, when it is difficult to
  match relevant variables in two independent
  groups, or when it is more efficient to carry out
  the experimental procedures with one group.
• The basic rules of the within-subjects design are
  to assess the dependent variable twice in a
  single group of subjects.

44
Within-Subjects (Repeated Measures) Designs

• The difference between two assessments
  demonstrates the effect of the independent
  variable.
• Subject variables such as age, gender, and
  education do not have to be controlled because
  the same group is used of both values of the
  independent variable.
• Limitations of repeated measurement designs
  include order effects.

45
Within-Subjects (Repeated Measures) Designs

• Order effects may take the form of a practice
  effect that improves performance or a fatigue or
  boredom effect that impairs performance.
• When repeated measurement designs are used to
  assess the effects of training, there is a lack
  of control for the possibility that improvement
  might have occurred without training.
• A control procedure for order effects is to
  counterbalance the order of presentation.

46
Within-Subjects (Repeated Measures) Designs

• However, counterbalancing cannot be used to
  control for order effects in training studies
  because the pretraining must always be the first
  measure.
• To control for the possibility of improvement
  without training, the independent group and
  repeated measure designs can be combined, with a
  trained and untrained group tested before and
  after training.

47
Between-Group and Within-Subjects (Mixed) Designs

• In many research studies, more than one
  independent variable is considered.
• The effects of two or more independent variables
  on a dependent variable may be examined.
• In other cases, one independent variable is
  studied with a between-subjects comparison, and
  the other independent variable is studied with a
  within-subject comparison.
• This mixed design incorporates both the
  between-group and the within-subjects tactics.

48
Single-Subject Designs

• Single-subject designs focus on the individual
  behavior of subjects rather than considering the
  average performance of a group of subjects.
• Single case research designs are appropriate for
  communication disorders research because they are
  intended to demonstrate that interventions cause
  changes in behavior.
• Single-subject designs can examine the behavior
  of more than one person, but the data of each
  person will be evaluated individually rather than
  as part of a group average.
• A number of different designs are available.

49
ABA, ABAB Designs

• The most basic single subject designs are
  withdrawal and reversal designs.
• After several baseline (A) measures, a treatment
  is given until the target behavior (B) changes.
• Then the treatment is taken away (withdrawal
  design) or nontarget behaviors are reinforced
  (reversal design) until the target behavior
  returns to baseline, and the treatment (B) is
  usually given once more.

50
ABA, ABAB Designs

• This procedure is designed to prove that the
  treatment caused the change in behavior.
• When there is baseline, treatment, and either
  withdrawal or reversal , it is an ABA design.
• When the treatment is is given again after the
  withdrawal or reversal, it is an ABAB design,
  which provides more complete proof of the
  effectiveness of treatment.
• These designs should only be used in
  communication disorders research when treatment
  effects may be temporary or can be reversed, and
  may be ethically inadvisable.

51
Multiple Baseline Designs

• Another way to demonstrate the effect of a
  treatment is the multiple baseline design.
• Treatment effects are first demonstrated for one
  dependent variable, and then for two or more
  additional dependent variables.
• The additional variables can be target behaviors,
  conditions, or subjects.

52
Multiple Baseline Designs Behavior

• In the multiple baseline design across behaviors,
  three or more target behaviors (e.g., the
  articulation of three different phonemes) are
  selected.
• Baseline measures (A) are taken for all three
  behaviors.
• Then treatment of the first target behavior (B)
  is begun, while baseline measures are continued
  for the other two behaviors.
• When the treatment effect for B has reached the
  desired level, treatment of the second target
  behavior (C) is begun, while baseline measures
  are continued for the other behavior.

53
Multiple Baseline Designs

• Finally, when target behavior C reaches the
  desired level, training of a third target
  behavior (D) can be begun, and continued until it
  reaches the desired level.
• If the baseline behaviors change only when the
  appropriate treatment is introduced, there is
  evidence of a causal relationship.

54
Multiple Baseline Designs Conditions

• In the multiple baseline design across
  conditions, a single target behavior is trained
  in three or more different training conditions.
• For example, the treatment of nonfluencies might
  be carried out in a research laboratory (B), at
  home (C), and in a public place (D).
• The sequence of changes from baseline measures
  (A) to the three conditions follows exactly the
  same sequence as those in the multiple baseline
  design across behaviors.

55
Multiple Baseline Designs Subjects

• In the multiple baseline design across subjects,
  a baseline (A) is established for three or more
  subjects (B, C, and D), and then the treatment of
  a target behavior is introduced at different
  times for the subjects, following the sequence
  described for the multiple baseline design across
  behaviors.
• The multiple baseline design across subjects is
  frequently used in communication disorders
  research.

56
Multiple Treatment Designs

• The effects of two or more different treatments
  can be compared with several single subject
  designs.
• In some cases the treatments are given one after
  the other, and in other cases the treatments are
  trained at the same time.
• The simple ABAB design can be extended to ABABC,
  ABABAC, ABABCD, and so forth, where new
  treatments are introduced after training with the
  first treatment.

57
Multiple Treatment Designs

• Such designs are called multi-treatment designs.
• The usual multi-treatment design involves a
  preplanned comparison of methods.
• Baselines can be taken between each treatment
  (ABACAD), and a theoretically optimal sequence of
  different treatments can be presented, such as
  training in imitating speech sounds followed by
  training in naming the speech sounds in words
  represented by pictures.

58
Multiple Treatment Designs

• The alternating treatment design, also called the
  multiple schedules design, presents the
  treatments (usually only two) in each session in
  counterbalanced order, or in alternating
  sessions.
• It is not necessary to take baseline measures
  because the treatment effects are compared, but
  it is advisable to take the baseline measures to
  demonstrate the magnitude of the effects.

59
Multiple Treatment Designs

• The difficulty with multiple treatment designs in
  communication disorder research is assessing the
  possible carryover effects from one treatment to
  another in the treatment of nonreversible
  behaviors.
• This difficulty may be overcome by using
  different target behaviors for each treatment,
  e.g., treatment B for one misarticulated phoneme
  and treatment C for a second misarticulated
  phoneme.

60
Generalization Designs

• In multiple baseline designs across behaviors and
  conditions, generalization from one target
  behavior to another or from one condition to
  another is undesirable, because the baselines for
  the untrained behaviors or conditions will
  change.
• In practical training studies, however, it is
  hoped that training effects will not be confined
  to the exact targets used in training, but will
  generalize to nontrained behaviors (e.g., a
  phoneme correctly articulated in a set of
  training words will be articulated correctly in
  non-trained words).

61
Generalization Designs

• There are several ways of assessing
  generalization to nontrained behaviors.
• One simple method is to probe nontrained
  behaviors during baseline, at intervals during
  training, and after training.
• If training is done in a laboratory setting, it
  is important to assess generalization, that is,
  carryover, to a normal conversational setting.

62
Single-Subject Designs

• Single subject designs have some of the
  advantages of group designs and some of the
  advantages of observational and case study
  designs.
• They provide direct quantitative measures of the
  behaviors studied, averaged neither across
  subjects in groups nor across studies.
• Experimental conditions are rigorously controlled
  to obtain information about causal relationships
  between independent and dependent variables.

63
Single-Subject Designs

• Large groups of subjects are not needed.
• Information is obtained for individuals rather
  than groups.
• The information may have direct practical
  applications. 
• Single subject designs have their own inherent
  limitations.
• Stable baselines may be difficult to establish.
• If the treatment is not immediately effective in
  changing the baseline behavior, it may be
  difficult to demonstrate causal relationships.

64
Single-Subject Designs

• Reversal designs cannot be used if treatment
  effects do not or should not reverse.
• Multiple baseline designs cannot be used if
  treatment effects generalize to nontreated
  behaviors.
• Multiple treatment designs may yield ambiguous
  results if treatment effects carry over and if
  differences between treatment effects are not
  clear-cut.

65
Single-Subject Designs

• The very rigid specification of target behaviors,
  treatments, and control procedures may make the
  treatment too artificial for direct application
  to clinical intervention.
• The treatment may change the target behaviors
  only in the experimental condition and not in
  natural communication situations.
• Finally, there is the problem of generalizing the
  results.
• There is no way of predicting that all subjects
  of the same type will show the same treatment
  effects.
•  
•