Evidence-Based Practice

1
Evidence-Based Practice
2
Summary of Presentation

• Evaluating Interventions
• Recommendations for Parents
• Fads

3
What Interventions Should We Use?

• The Right to Effective Behavioral Treatment
• ABA Task Force (1987)
• Published in JABA (1988)
• http//www.abainternational.org/ABA/statements/tre
  atment.asp
• Individuals who are recipients of treatment
  designed to change their behavior have the right
  to
• A therapeutic environment
• Services whose overriding goal is personal
  welfare
• Treatment by a competent behavior analyst
• Programs that teach functional skills
• Behavioral assessment and ongoing evaluation
• The most effective treatment procedures available

4
The Most Effective Treatment Procedures Available

• An individual is entitled to effective and
  scientifically validated treatment
• In turn, the behavior analyst has an obligation
  to use only those procedures demonstrated by
  research to be effective.

5
Behavior Analyst Certification Board (BACB)
Guidelines for Responsible Conduct

• Section 2.09 Treatment Efficacy
• http//www.bacb.com/consum_frame.html
• The behavior analyst always has the
  responsibility to recommend scientifically
  supported most effective treatment procedures.
  Effective treatment procedures have been
  validated as having both long-term and short-term
  benefits to clients and society.
• Clients have a right to effective treatment
  (i.e., based on the research literature and
  adapted to the individual client).
• Behavior analysts are responsible for review and
  appraisal of likely effects of all alternative
  treatments, including those provided by other
  disciplines and no intervention.

6
As Behavior Analysts, we

• Maintain a healthy skepticism
• Differentiate opinions, beliefs, and speculations
  from facts
• Dont make claims without supporting objective
  data
• Skepticism ? cynicism
• Being open and being skeptical might seem
  contradictory, but theyre not. This is the way
  to think scientifically.
• Some phenomena that seem outlandish are valid
• Lucid dreaming, extraordinary feats of human
  memory and appropriate clinical uses of hypnosis
  (as opposed to the scientifically unsupported use
  of hypnosis for memory recovery).
• We must keep our minds open but not so open that
  our brains fall out

7
Ghezzi, Williams, and Carr (1999)

• Preface
• Those who fall in love with practice without
  science are like a sailor who enters a ship
  without a helm or compass, and who never can be
  certain whither he is going (Leonardo da Vinci)

8
Green (1996)

• Behavior analysts use a number of practices to
  increase the objectivity of evidence about
  treatment effects
• Operational definitions (specific and observable
  descriptions of behavior)
• Measurement procedures are clearly specified
• Interobserver agreement data are collected
• Multiple measures of the effects of tx can be
  obtained for example,
• Direct observation
• Standardized tests
• Parent ratings of behavior

9
How Do Parents Choose a Treatment for Their Child
with Autism?

• Recommendation by pediatrician or other doctor
• School
• Other parent
• Internet, book
• Do you think these sources reliably recommend
  interventions based on the objectivity of the
  evidence?

10
Fads in Autism Treatment

• Metz, Mulick, and Butter (2005) Google search -
  autism and treatment 65 distinct interventions
  sold as effective for treating autism
• Telepathy, injection of sheep stem cells, thyme,
  swimming with dolphins
• Fads in the media
• In autism treatment, fads tend to be harmful
• Waste time
• Waste money
• Falsely raise hopes and expectations
• Distract from effective treatments
• In some cases, harm children and families

11
Why Do Ineffective/Unproven Treatments Become
Fads? (Vyse, 2005)

• Theyre presented as relatively easy and with
  immediate effects by people who appear warm,
  sincere, and attentive
• Best treatment is deemed distasteful or is hard
  to get
• Alternate treatments are supported by popular
  culture, feel right, seem to make sense
• Professionals or other people recommend them
• Most professionals are not trained how to
  evaluate treatments
• Autism treatment is a commercial enterprise

12
Why Do Ineffective/Unproven Treatments Become
Fads? (Metz et al., 2005)

• Parents are in a vulnerable position
• Doing SOMETHING feels better than doing nothing
• Grieving process
• Avoidance of guilt
• Urgency
• Qualities that make parents dedicated and
  enthusiastic make them vulnerable to accept
  claims without close scrutiny
• Nature of Autism
• Cause is unknown
• Autism is mysterious myth of the hidden inner
  child

13
Even Well-Meaning Professionals Use
Ineffective/Unproven Interventions
(Metz et al., 2005)

• Example of facilitated communication
• Rationale provided for the therapy may be logical
  or sound convincing
• But many are based on flawed theories about the
  cause of autism
• For the overwhelming majority of autism
  treatments, anecdotes and testimonials are the
  only supporting evidence
• Almost none stand up to reasonably rigorous
  scientific evaluation
• Many therapies for autism in widespread use today
  have been shown to be ineffective in scientific
  studies
• Some have been shown to be harmful
• Still others have not been subjected to any
  rigorous evaluations

14
Where can we refer parents?

• Association for Science in Autism Treatment
• www.Asatonline.org

15
Evidence-Based Practice

• Evidenced-based interventions
• Evidence-based practices
• Empirically supported treatments
• Best practices

16
Evidence-Based Practice (EBP)ODonohue and
Ferguson (2006)

• the integration of the best available research
  with clinical expertise in the context of patient
  characteristics, culture, and preferences (APA
  Presidential Task Force on Evidence-Based
  Practice, 2006, p. 273)
• http//www.apa.org/practice/ebp.html
• Other similar terms
• Empirically-validated therapies (EVT)
• Empirically-supported therapies (EST)

17
Development of EBPODonohue and Ferguson (2006)

• 20 years ago - Clinical practice guidelines
  developed in medicine to
• Help standardize decision-making in treatment
• Encourage use of empirically sound treatments
• Improve quality of medical services and reduce
  errors
• Early 90s American Psychiatric Association
  developed their own clinical practice guidelines
• Intended to assist psychiatrists in
  decision-making in treatment
• Tended to recommend pharmacotherapy over
  non-pharm treatments
• Mid 90s American Psychological Association
  (APA) Task Force publishes their own clinical
  practice guidelines

18
The Chambless Criteria

• Division 12 of the American Psychological
  Association (APA) Clinical Psychology
• Established a Task Force (headed by Diane
  Chambless) to identify and promote empirically
  supported psychological treatments in 1993.
• Chambless et al. (1998) available at
  http//www.apa.org/divisions/div12/est/97report.pd
  f

19
(No Transcript)
20
(No Transcript)
21
Weaknesses of Chambless Criteria ODonohue and
Ferguson (2006)

• Do not take clinical significance into account
• Focus on efficacy, not effectiveness
• Will the same results be obtained in natural
  settings? Is the treatment practical to
  implement in terms of staffing, funds, expertise?
• Most studies on the EBP list do not include
  participants with comorbid disorders
• Are biased toward group designs and inferential
  statistics
• Only 2 good group design exps, but at least 9
  single-case exps
• Many single-case designs dont compare 2
  treatments

22
Individuals with Disabilities Education
Improvement Act (IDEA) No Child Left Behind
(NCLB)

• NCLB of 2001 interventions should be based on
  scientific research
• IDEA of 2004 scientifically based institutional
  practices for those in spec eds
• Services in a childs IEP should be based on
  peer reviewed research the extent practicable
  (IDEA, 2004)

23
Professional Organizations

• APA
• Established scientific and professional
  knowledge of the discipline
• National Association of School Psychology (NASP)
• Practice should be based on scitific research

24
Horner, Carr, Halle, McGee, Odom, and Wolery
(2003)

• Single-subject research documents a practice as
  evidence-based when
• The practice is operationally defined
• The outcomes and context in which the practice is
  to be used are defined (target behaviors
  affected, setting, age, skills, diagnosis,
  implementer)
• The practice is implemented with fidelity (tx
  integrity data)
• The change is the DV is shown to result from the
  IV experimental control!
• Effects of the practice are
  replicated across a sufficient
  number of
  studies

25
Make a table
APA NY State Dept of Heath (1999) CEC NASP CASP rating system for single case designs Gina Greens Gold standards National Standards Projects
Well-established Tx Efficacy demonstrated (gt placebo or to established Tx) in at least 2 good b/w group designs expos OR large series of s-c designs exps with good design and comparison to another Tx
Probably Efficacios Tx
Experimental Tx

26
Script and Script Fading Script and Script Fading Script and Script Fading Script and Script Fading Script and Script Fading Script and Script Fading
APA NY State CEC NASP NASP rating system Gina Greens Gold Standards NSP






27
Problems with Current Clinical Guidelines

• Studies examined were before 1999
• Process used by each of the task forces were not
  made public
• Not comprehensive

28
National Standards Project-examined empirical
evidence supporting interventions that could be
used in school setting for those younger than 22

• Exclusion criteria
• Medical/complementary/alternative
• Axis II outside of MR
• Studies that could not be easily performed in
  schools
• Qualitative analyses or published in non-peer
  reviewed journals

29
Scientific Merit Rating Scale

• Rated from 0-5
• Research design
• Measurement of the DV
• Measurement of the IV
• Participant selection
• Generalization

30
Coding

• Beneficial treatment effects reported
• Single-case designs
• Strong, moderate weak
• Functional relationship has been established is
  replicated
• Pont of comparison across conditions exists
• Magnitude of change is consistent
• Percentage eon non-overlapping points is
  impressive
• No treatment effect reported
• Adverse treatment effects reported

31
National Standards Project

• http//www.nationalautismcenter.org/about/national
  .php

32
References

• APA Presidential Task Force on Evidence-Based
  Practice. (2006). Evidence-based practice in
  psychology. American Psychologist, 61, 271-285.
• Chambless, D.L., Baker, M., Baucom, D.H.,
  Beutler, L.E., Calhoun, K.S., Crits-Christoph,P.,
  et al. (1998). Update on empirically validated
  therapies, II. The Clinical Psychologist, 51,
  3-16.
• Ghezzi, P.M., Williams, W.L., Carr J.E.
  (1999). Autism Behavior analytic perspectives.
  Reno, NV Context Press.
• Green, G. (1996). Evaluating claims about
  treatments for autism. In C. Maurice, G. Green,
  S.C. Luce (Eds.), Behavioral intervention for
  young children with autism. (pp. 15-28). Austin,
  TX Pro-ed.
• Metz, B., Mulick, J.A., Butter, E.M. (2005).
  Autism A lat-20th-century fad magnet. In J.W.
  Jacobson, R.M. Foxx, J.A. Mulick (Eds.),
  Controversial therapies for developmental
  disabilities. (pp. 237-263). Mahwah, NJ
  Lawrence Erlbaum.
• ODonohue, W., Ferguson, K.E. (2006).
  Evidence-based practice in psychology and
  behavior analysis. The Behavior Analyst Today,
  7, 335-350.
• Vyse, S. (2005). Where do fads come from? In
  J.W. Jacobson, R.M. Foxx, J.A. Mulick (Eds.),
  Controversial thearpies for developmental
  disabilities. (pp. 3-17). Mahwah, NJ Lawrence
  Erlbaum.