Evidence Based Practice

1
Evidence Based Practice

• Steven R. Pruett, Ph.D, CRC
• SERNRA Conference
• May 18, 2005

2
Evidence-Based Practice

• Since the passage of the HMO Act of 1973, there
  have been many changes in the American healthcare
  system.
• Consumer-driven
• Market-based
• Customer choice
• Customer satisfaction
• Health outcomes

3
Evidence-Based Practice

• Rehabilitation Healthcare systems, researchers,
  practitioners and other stakeholders need to
  provide evidence to support the effectiveness of
  services.
• Particularly relevant in rehabilitation has has
  given rise to the emphasis on consumerism,
  consumer involvement, and program evaluation.

4
Evidence-Based Practice

• One of the ways healthcare providers have
  responded to these managed care system demands is
  through evidence-based practice.

5
Definitions

• Rosenberg and Donald (1996) defined
  evidence-based medicine as a process of turning
  clinical problems into questions and then
  systematically locating, appraising, and using
  contemporaneous research findings as the basis
  for clinical decisions.
• Sackett, Rosenberg, Gray, Haynes, and Richardson
  (1996) described evidence-based practice as the
  conscientious, explicit, and judicious use of
  current best evidence in making decisions about
  the care of individual patients.

6
Definitions

• DePalma (2000) further refined the definition of
  evidence-based practice as a complete process
  beginning with knowing what clinical questions to
  ask, how to find the best practice, and how to
  critically appraise the evidence for validity and
  applicability to the particular care situation.
  The best evidence then must be applied by a
  clinician with expertise in considering the
  patient's unique values and needs. The final
  aspect of the process is evaluation of the
  effectiveness of care and the continual
  improvement of the process.
• Ottenbacher and Mass (1998) indicated that the
  best evidence used to support evidence-based
  practice is derived from a series of research
  studies results in an empirical consensus
  regarding the effectiveness of a treatment
  approach.

7
Evidence-Based Practice Rehab Case Management

• Limited empirical support for rehab case mgt
  interventions.
• Central focus of this service is to evaluate,
  managed coordinate services for PWDs
  (Chronister, da Silva Cardoso, Lee, Chan
  Leahy, 2005)
• Medical Case Mgt assesses, plans, implements
  coordinates, monitors evaluates options and
  services to meet individuals health needs
  through communication and available resources to
  promote quality cost-effective outcomes.
  (Mullahy, 1995)
• Voc case mgt is the movement of a client through
  the rehab process includes the mgt and
  coordination of all services needed to achieve
  successfully the rehab goal.
• (Cox, Connolly Flynn, 1981)

8
Evidence-Based Practice Rehab Case Management

• Socase management emphasizes
• Evaluation
• Outcomes
• Quality assurance
• Thereforecase managers are already working, in
  part, as evidence-based practitioners without
  explicitly recognizing it.

9
Rehab and Evidence-Based Practice

• Quality Assurance standards of practice
• Case Management Society of America
• Individual Case Management Assoc Aetna Health
  Plan developed Case Management Practice
  Guidelines.
• CRCC code of professional ethics.
• Decision-making process
• Evaluation needs to be based on credible
  scientific evidence versus subjective experience.

10
Rehab and Evidence-Based Practice

• Rehabilitation researchers must conduct
  systematic research to demonstrate the
  effectiveness of rehabilitation counseling/case
  management interventions.
• RCs/case managers must be able to assist clients
  in selecting the most appropriate medical,
  psychological, educational, social and vocational
  interventions for a clients particular
  situation.

11
Rehab and Evidence-Based Practice

• To better practice evidence-based rehabilitation
  RC/case managers must have the following
  knowledge
• Research design
• Statistics
• How to access pertinent research

12
Evidence-Based PracticeGeneral Concepts

• Gold standard of best practice in medicine
• In medicine, with its positivist scientific
  methods tradition, the gold standard for
  scientific evidence is still randomized clinical
  trials and the method of choice for determining
  the cumulative evidence of the effectiveness of a
  treatment is meta-analysis.
• Randomized clinical trials
• Meta-analysis

13
Evidence Based Practice

• Steps for the evidence-based practice of medicine
  by practitioners
• Formulate a clear clinical question from a
  patient's problem.
• Search the literature for relevant clinical
  articles.
• Evaluate (critically appraise) the evidence for
  its validity and usefulness.
• Implement useful findings in clinical practice.
• (Rosenberg Donald, 1996)

14
Evidence-based PracticeSome Rehab Questions

• What processes/techniques make a specific
  rehabilitation intervention work?
• For whom is the intervention most effective?
• Are certain interventions/programs better for
  certain populations?
• Who should receive a specific intervention or
  program? When? And for how long?

15
A Hierarchy of Levels of Best Evidence

• Level 1 evidence is defined as strong evidence
  from at least one systematic review of multiple
  well-designed randomized controlled trials.
• Level 2 evidence is defined as strong evidence
  from at least one properly designed randomized
  controlled trial of appropriate size.
• Level 3 evidence is defined as evidence from
  well-designed trials without randomization,
  single group pre-post, cohort, time series, or
  matched case-controlled studies.

16
A Hierarchy of Levels of Best Evidence

• Level 4 evidence is defined as evidence from
  well-designed non-experimental studies from more
  than one center or research group.
• Level 5 evidence is defined as opinions of
  respected authorities, based on clinical
  evidence, descriptive studies, or reports of
  expert committees.

17
Related Concepts Empirically Supported Treatment

• APA Division 12 defined empirically supported
  treatment (EST) as clearly identified
  psychological treatments shown to be efficacious
  in controlled research studies with a delineated
  population.
• EST should be evaluated in terms of efficacy
  (statistical and clinical significance),
  effectiveness (clinical utility), and efficiency
  (cost-effectiveness).

18
Related Concepts Empirically Supported Treatment

• Patterned after the FDA guidelines for approval
  of new drugs, the APA Div 12 Task Force on the
  Promotion and Dissemination of Psychological
  Procedures established two criteria for
  establishing the empirical validity of a
  psychotherapeutic approach
• The approach is superior to a placebo or other
  treatment or
• The approach is equal to an established
  treatment, in at least two studies established by
  different investigators.

19
Related Concepts Meta-Analysis

• Meta-analysis is a method used to review research
  literature based on statistical integration and
  analysis of research findings.
• In treatment effectiveness meta-analysis, the
  dependent variable is the effect size (i.e., the
  outcomes or results of each study selected for
  review transformed into a common metric across
  studies) and the independent variables are study
  characteristics (i.e., participants,
  interventions, and outcome measures).

20
Related Concepts Meta-Analysis

• Meta-analysis is an effort to review the results
  of a research domain in quantitative terms. The
  intent is to identify what significant
  relationships exist between study features
  (independent variables) and effect sizes
  (dependent variable).
• Benefits The benefits of meta-analysis include
  its ability to
• Synthesize the results from many studies
  succinctly and intuitively for nonscientific
  communities,
• Illustrate the amount and relative impact of
  different programs on different criteria for
  policy decision-making purposes, and
• Identify the most effective programs and
  highlight gaps or limitations in the literature
  to suggest directions for future research

21
Related Concepts Meta-Analysis

• A common index of the size of the effect produced
  by each study is the effect size index g, which
  is the standardized difference between the sample
  mean of the treatment group and the sample mean
  of the control group (Wampold, 2001). A positive
  score indicates that the treatment group
  outperformed the control group, and a negative
  score has the reverse meaning. However, the
  effect size index g is a sample statistics. As
  such, it is a biased estimator of the true (i.e.,
  population) effect size.

22
Related Concepts Meta-Analysis

• Hedges and Olkin (1985) provided the effect size
  index d as a good approximation of the unbiased
  estimator and the index d for aggregating the
  effect sizes across studies as estimate of
  population effect size. The unbiased effect size
  indexes d and d are commonly reported in
  meta-analytic studies.
• Standardized Mean Difference Effect Size
• large effect d .80
• medium effect d .50
• small effect d .20

23
Evidence-Based Practice Implications for Rehab
Counseling

• Implications for counseling research The need
  for level 1 evidence (empirical supported
  treatment and meta-analysis) related to the
  effectiveness of rehabilitation interventions
  (individual ingredients/components of RC as well
  as RC as interventions)
• Implications for practitioners For
  practitioners, evidence-based practice is a
  research utilization issue (ability to judge the
  quality of an individual research study and a
  collection of studies, the ability to select the
  best interventions on an individualized basis,
  and the ability to search for research
  information using the Internet and other library
  tools)

24
Resources for Evidence-Based Practice

• Agency for Healthcare Policy and Research (AHRQ)
• Title IX of Public Health Services Act
• Healthcare Research and Quality Act of 1999
• Primary agenda is to research health outcomes,
  develop effective outcome measures evaluate
  overall quality of care.
• Patient Outcomes Research Teams (PORTs)
• Clinical Practice Guidelines
• Clinical performance measures
• Report Cards

25
PORTs

• Designed to determine the most effective
  treatment and pattern of care for a specified
  clinical area through lit reviews
  meta-analysis.
• Partnership with National Institute of Health.
• Some PORTs that are relevant to rehab
• Effectiveness of alternative treatments for type
  II diabetes mental health problems in the
  Mexican-American population (funded in part by UT
  Health Science Center)
• Secondary prevention of stroke, hip fx repair,
  osteoarthritis, TKR, back pain tx and assessment.

26
Clinical Practice Guidelines

• Frequently developed from PORT findings
• Offer healthcare providers well-founded,
  cost-effective treatment methods for various
  clinical conditions.
• PORT study on Stroke Rehabilitation led to the
  clinical practice guideline Post-stroke
  Rehabilitation (AHRQ, 2000).
• Development use of clinical practice guidelines
  has become so well accepted that AHRQ no longer
  sponsors development since many public private
  entities are doing this independently.
• National Guideline Clearinghouse
  (www.guideline.gov)

27
Clinical Performance Measures

• Used to assess services of a healthcare
  professional by reviewing
• appropriateness of service
• timeliness of service, and
• access to service
• Measures are detailed and condition specific
• Attempts to measure quality of care.
• Currently there are approximately 1,200
  performance measures
• Are grouped by
• Disease/condition
• Treatment/intervention
• Domain
• Organization
• Available at the National Quality Measures
  Clearinghouse (www.qualitymeasures.ahrq.gov)

28
Clinical Performance Measures

• Measures developed by
• Joint Commission on Accreditation of Healthcare
  Organizations
• Health Plan Employers Data Information Set
• Healthcare Cost Utilization Project
• Outcome Assessment Information Set
• The University of Wisconsin Nursing Home quality
  indicators.
• VA external review program

29
Report Cards

• Information obtained through these Clinical
  Performance Measures are frequently used for
  marketing and other communication purpose to
  consumers and other purchasers through Report
  Cards
• Report Cards are a response to the
  consumer-driven healthcare system and provides
  consumers with information concerning the care
  recommended by clinical practice guidelines,
  outcomes expected under specific situations and
  the wide range of performance measures used to
  evaluate the quality different aspects of care
  offered in a user-friendly format.

30
Other resources

• American Congress of Rehabilitation Medicine
• www.acrm.org/Resources/Evidence-BasedResources.htm
  

31
Research Issues

• Rehabilitation research has identified important
  rehab counseling/case mgt functions and knowledge
  domains, BUT
• As of this time, no empirical evidence has been
  generated that supports the effectiveness of
  these functions or knowledge domains on rehab
  outcomes.

32
Research Issues

• It is possible to infer some evidence using
  research from allied disciplines (e.g.,
  counseling and clinical psychology)
• For example the process variable of working
  alliance has gained overwhelmingly strong
  empirical evidence as a primary influence on
  counseling outcomes (Wampold, 2001).

33
Research Issues

• There is some evidence for effectiveness of
  counseling factors in rehab counseling
• Bolton and Akridges (1995) meta-analysis of 15
  experimental evaluations of 10 skills training
  interventions (e.g., social skills, stress mgt.,
  problem-solving skills, career decision-making
  skils) across some 61 outcome measures.
• Estimated true effect size of .93 indicated
  skills training services substantially benefit
  the typical VR client.

34
Practice Issues

• Rehab Counselors/case managers need to aware of
  contemporary research in rehab as well as in
  allied fields.
• This will promote evidence-based practice and
  insure people with disabilities and chronic
  illnesses receive effective services.

35
Practice Issues

• RC Masters education curricula has only one
  class in research methods
• Probably the most unpopular course and
  unfortunately translates to negative attitudes
  towards using research.
• Other problems to research utilization
• Practicing counselors generally dont believe
  research results can translate into their work
• Lack of time on the job
• High cost of continuing education
• Weak research analysis skills

36
Practice Issues

• How to correct these problems
• Rethink curricula regarding research
• Need for more creative activities that promote
  working knowledge of manual and computer searches
• Solid understanding of different research designs
  and issues related to power analysis, effect size
  and meta-analysi.
• Regular in-service training within rehab work
  environments

37
Summary

• Rehab is one of many health-related disciplines
  facing demands of a managed-care system.
• Professionals in the social and behavioral
  sciences are having to rely on credible evidence
  to justify their interventions and maintain their
  identity in a climate of healthcare outcomes.

38
Summary

• Evidence-based practice and empirically validated
  interventions dominate the healthcare research
  and are most likely here to stay.
• How well rehabilitation researchers and
  practitioners work together to respond to the
  demands of this evidence-based climate is crucial
  for the professions viability.

39
Summary

• Rehabilitation researchers need to conduct more
  experimental studies to validate the
  effectiveness of specific rehab counseling/case
  management interventions as well as to determine
  the overall effectiveness of rehab counseling and
  case management.
• Rehabilitation educators need to focus on
  improving the quality and effectiveness of
  courses in research methods to facilitate
  evidence-based practitioners.

40
Summary

• Rehabilitation students need to be able to
  understand a wide array of research designs and
  methodologies as well as access and critically
  evaluate research from rehab and related
  literature.
• Work settings need to provide pre-service and
  in-service training regarding evidence-based
  practice and incorporate this type of service
  delivery into their job descriptions.

41
Summary

• Evidence-based practice is the standard of
  practice in healthcare.
• The extent to which rehabilitation
  counselors/case managers, researchers and
  educators prepare for and accept EBP will dictate
  the degree to which rehabilitation can be
  performed correctly and adequately by the
  professionals providing the service.
• (Chronister et al., 2005)