Using systematic reviews to inform practice

1
Using systematic reviews to inform practice

• Statewide School-based
• OT/PT Conference
• October 28, 2005
• Steven M. Cope, Sc.D., OT

2

• Evidence-based medicine is the integration of
  best research evidence with clinical expertise
  and patient values
• Sackett et al. (2000)

3

• An evidence-based occupational therapy practice
  uses research evidence together with clinical
  knowledge and reasoning to make decisions about
  interventions that are effective for a specific
  client(s)
• Law Baum (1998)

4

• Although there is an undeniable art to pediatric
  physical therapy, the heart of our practice
  should be the scientific basis of our
  interventions. The challenge is to integrate art
  and science in making clinical decisions that
  allow us to provide our patients and families
  with optimal care.
• M.J. Barry

5
Evidence-based practice is

• based on a single patient
• based on clinical judgement and patient values
  (client-centered)
• guided by research rather than dictated by it

6
Being an evidence-based practitioner means

• You value the importance of scientific literature
  as a foundation for clinical decision making
• You frequently ask yourself, what evidence
  exists in the literature to support or refute the
  intervention I am about to provide?
• You believe your interventions will be more
  effective when research evidence is integrated

7
Assumptions of EBP

• Scientific research provides an important basis
  for verifying the effectiveness of our
  interventions
• The effectiveness of interventions are improved
  under an evidence-based approach
• Life-long learning is important to effective
  clinical practice

8

• Diagnostic and technical skills increase with
  experience however, clinical effectiveness
  deteriorates with time unless current knowledge
  is used to modify practice patterns

9
Barriers to EBP

• Time for individual study and group discussion
  (lack of)
• Access to scientific literature (lack of)
• Interpretation of published findings (difficult
  to understand)
• Attitudes towards EBP may be negative

10
Evidence-based practice Getting started

• Pose a researchable question
• Search literature for best evidence
• Do critical appraisal of studys validity
• Integrate evidence into clinical decision
• Evaluate clinical effectiveness

11
Pose a Researchable Question

• Identify clinical problem
• Example Children with cerebral palsy have
  spasticity and underlying muscle weakness leading
  to functional movement difficulties
• Identify intervention(s)
• -Strength training
• Relative outcomes
• -strength, spasticity, and functional movement
• Patient characteristics
• -children with spastic CP

12
The Research Question

• Is strength training intervention effective at
  increasing strength and functional movement
  relevant outcomes in children with spastic CP
  patient characteristics?

13
Evidence-based practice Getting started

• Pose a researchable question
• Search literature for best evidence
• Do critical appraisal of studys validity
• Integrate evidence into clinical decision
• Evaluate clinical effectiveness

14
Search Literature for Best EvidenceWhat does
this mean?

• Research on subjects whose characteristics are
  similar to your patients
• Research on interventions that match the one you
  want to provide
• Research on outcomes that are of interest and
  apply to both you and the patient
• Research that is credible and believable

15
Evidence-based practice Getting started

• Pose a researchable question
• Search literature for best evidence
• Do critical appraisal of studys validity
• Integrate evidence into clinical decision
• Evaluate clinical effectiveness

16
Critical Appraisal

• Design (I-V)
• Sample Size (A-C)
• Internal Validity (1-3)
• External Validity (a-c)

17
Design
18
Sample Size

• A n 20 per group
• B n lt 20 per group

19
Internal Validity

• 1 High internal validity
• No alternate explanation for outcome
• 2 Moderate internal validity
• Attempt to control for lack of randomization
  biases
• 3 Low internal validity
• Two or more serious alternative explanations for
  outcome

20
Threats to Internal Validity

• Hawthorne effect
• Maturation
• Testing effect
• Experimenter bias
• Co-intervention effects
• Errors with data
• No Randomization
• Attrition

21
External Validity

• a High external validity
• Participants represent population, and treatments
  represent current practice
• b Moderate external validity
• Between high and low
• c Low external validity
• Heterogeneous sample without being able to
  understand whether effects were similar for all
  diagnoses or treatments do not represent current
  practice

22
Critical Appraisal

• Strongest evidence
• IA1a
• Weakest evidence
• VB3b

23
Evidence-based practice Getting started

• Pose a researchable question
• Search literature for best evidence
• Do critical appraisal of studys validity
• Integrate evidence into clinical decision
• Evaluate clinical effectiveness

24
Integrate Evidence into Clinical Decision

• The evidence reviewed should help you decide 1)
  do I start an intervention Im not currently
  using 2) do I stop doing an intervention I am
  currently doing? 3) continue?
• Was the evidence you found best evidence?
• Does the evidence support or refute the
  intervention you want to provide? For which
  outcomes? Which patients responded well?
• How much evidence exists?
• How much evidence is needed to make this
  decision?

25
Evidence-based practice Getting started

• Pose a researchable question
• Search literature for best evidence
• Do critical appraisal of studys validity
• Integrate evidence into clinical decision
• Evaluate clinical effectiveness

26
Evaluate Clinical Effectiveness

• Reflection/assessment after performing the
  intervention
• Did your patient experience a positive change?

27
Systematic Review What is it?

• A summary of several research articles on one
  topic presented in table and narrative formats
• Qualitative rather than quantitative analysis
• Subjective interpretation
• Time consuming process
• Strongest form of evidence

28
Systematic Review 1Research Question

• What is the effectiveness of sensori-motor and
  motor learning handwriting interventions on
  improving writing legibility and speed in
  children with handwriting difficulties?

29
Search strategies

• Key terms (handwriting, writing, written
  communication, treatment, OT, intervention, )
• Data bases (PubMed)
• Entrance criteria (level of evidence, diagnosis,
  age, years since publication)

30
Critical Appraisal

• Two or more people read articles independently
  and reported relevant information on a form for
  discussion
• Group discussion to develop consensus on key
  decisions (journal club)
• Use AACPDM methodology to present results (table
  and narrative format)

31
Results

• Sixteen studies located
• Eight eliminated from review because they didnt
  meet entrance criteria
• Eight studies selected for review

32
Systematic Review of Handwriting Interventions
33

34
Table 2
35
Systematic Review Discussion

• 4 studies used a combination of sensorimotor and
  motor learning-based intervention however, in
  these studies, emphasis was on sensorimotor-based
  intervention
• 1 study used only sensorimotor intervention
• 2 studies used only motor learning (practice)
• 2 studies compared sensorimotor with motor
  learning (practice)

36
Discussion continued

• Children in all the studies ranged from 5-11
  years of age and were WNL for cognitive function
• All the children receiving intervention were
  identified as having handwriting difficulties
• Interventions ranged from 3 hours to 30 hours and
  from 6 days to 7 months. Most sessions were 30
  minutes long

37
Discussion continued

• 3 articles were randomized controlled trials and
  represented strong evidence the remaining
  articles were either non-randomized or lacked
  control groups
• 4 of 9 of the studies showed significant
  improvement for handwriting and an additional 3
  demonstrated trends toward improvement
• Studies that looked at quality of handwriting
  legibility showed that children achieved modest
  benefits
• Studies that looked at handwriting speed showed
  no change in this variable
• Long term effects were not studied
• Outcomes measured primarily focused on activity
  level (handwriting), but all levels were
  represented

38
Clinical Bottom Line

• We do know that handwriting is an area that can
  improve with intervention
• The evidence to date shows some benefit of
  intervention, however, the studies have not
  determined what the benefits from these
  interventions are long-term. Future studies
  should examine the relative effectiveness of
  sensorimotor and motor learning approaches and
  the long-term effects of each intervention

39
Clinical Bottom Line

• This systematic review of handwriting
  interventions reveals a relatively small number
  of studies, only a few with rigorous designs
• After reviewing the scientific evidence we are
  still not sure of the best method for improving
  handwriting. More studies need to be done that
  isolate sensorimotor and motor learning
  interventions to determine if one approach is
  more effective than the other.

40
Credits

• This review was completed in May 2004 by Rula
  LaLicata, Stephanie Beilke, Mary Lassanske, Lisa
  Villardita, Nicole Rosalez, and Steve Cope it
  was updated in May 2005 by Cathy Payne and Steve
  Cope

41
Systematic Review 2Research Question

• What do we know about the effectiveness of
  strengthening exercises for children with spastic
  cerebral palsy?

42
Focused Research Questions

• What is the effect of strengthening exercises on
  force production in children with spastic CP?
• Are there adverse effects to strengthening
  spastic muscles?
• What is the effect of strengthening exercises on
  functional outcome in children with spastic CP?

43
Method of Review

• Literature search was performed using on-line
  databases Pubmed, PEDro, CINAHL
• Each study selected for review was read and
  critically appraised by two people
• Reviewers collaborated on classification of the
  article in terms of threats to internal validity,
  level of evidence rating, statistical
  significance, and clinical importance

44
Method of Review cont.

• Information and findings from all articles were
  summarized in table and narrative format
  according to AACPDM methodology

45
Results

• The literature search yielded 15 articles
  specifically addressing strengthening exercise
  and spastic CP
• 6 of the 15 articles were eliminated because they
  did not meet entrance criteria
• 9 articles were reviewed

46
Summary of Studies
Table 3
Table 3
47
Abbreviations for Table

• EEI Energy Expenditure Index
• LSUT Lateral Step-Up Test
• MAS-STS Motor Assessment Scale Sit to Stand
• GMFM Gross Motor Function Measure
  
• SPPA Self-Perception Profile for Adolescent
  
• 1 Fowler, et.al. (2001)
• 2 Damiano, et.al. (1995)
• 3 Eagleton, et.al. (2004)
• 4 Blundell, et.al. (2003)
• 5 MacPhail Kramer, (1995)
• 6 Damiano Abel, (1998) Colored no clinical
  importance
• 7 OConnell Barnhart, (1995) Italics small
  clinical importance
• 8 Dodd, et.al. (2003) Regular font
  clinically important
• 9 Darrah, et.al. (1999)

48
Abbreviations cont.

• Level of Evidence
• I Randomized controlled trial
• III One group pre-post test
• Ratings of Internal Validity
• S Strong, M Moderate, W Weak

49
What is the effect of strengthening exercises on
force production in children with spastic
cerebral palsy?

• Evidence from the studies reviewed showed that
  for the majority of studies, children, and
  muscles studied, children with CP did improve
  their strength, primarily in their lower
  extremities

50
Are there adverse effects to strengthening
spastic muscles?

• Only one study specifically examined the effects
  of muscle strengthening on spasticity, and it
  provided evidence that spasticity was not
  increased

51
What is the effect of strengthening exercises on
functional outcome in children with spastic
cerebral palsy?

• There is evidence to suggest that function
  improves after a strengthening program
• Function was operationally defined in most
  studies as a performance on the GMFM and various
  parameters derived from gait analysis

52
Additional outcome measures

• Other impairment level outcomes were measured
  such as ROM, Heart Rate (HR), and energy
  expenditure
• Results indicate that strengthening did have a
  positive effect on energy expenditure but no
  effect on ROM and HR

53
Suggestions for Future Research

• A study design with other types of CP other than
  spastic (i.e. dystonic and athetoid)
• It remains unknown if strengthening programs can
  be used (and if they are safe) for very young
  children and toddlers with spastic CP
• The best type of strengthening exercise and the
  most effective had high intensity and duration
• The effects on upper extremities are not known
• The long-term effects of a strengthening program
  on strength and function are not known

54
Clinical Bottom Line

• This review revealed a reasonable number of
  moderate quality studies
• There is clear evidence that strengthening
  programs are effective for increasing strength in
  children with spastic CP
• There is also evidence to suggest that kids not
  only get stronger, but also have an increase in
  functional skills
• There has only been one randomized study done,
  and there are still many unanswered questions
  which suggests that this area of research is in
  its early phases
• More research with rigorous designs and larger
  sample sizes is needed

55
Credits

• This systematic review was completed in May, 2005
  by Christina Keller, DPT and Steve Cope, Sc.D.,
  OT

56
Systematic Review 3

• How effective are sensory-based interventions
  that are used in school settings for improving
  time on task in children with ADHD?

57
Purpose

• The purpose of this project was to systematically
  review the scientific literature investigating
  the effectiveness of treatments used in the
  classroom for improving attention and behavior in
  children with ADHD and PDD (including autism).
• Research Question What is the effectiveness of
  interventions using sensory-based approaches in
  improving attention span and behavior in children
  with ADHD or PDD in the classroom setting?

58
Inclusion Criteria

• Population studied was ADHD or PDD
• Interventions that affect classroom function and
  likely to be provided by an occupational
  therapist
• Level III of evidence or higher
• Peer-review journal only
• 1992-present

59
Procedures

• Terms used for research ADHD, treatment, OT
  treatment, time on task, school-based therapy,
  therapy, rehabilitation, weighted vests, OT, PT,
  PDD, autism, sensory integration, sensory
  modulation, attention and time
• Data bases used Ebscohost, OT Search, OCLC First
  Search, Proquest-Cinahl, OT Seeker, PsychInfo,
  ERIC, and PubMed
• Articles were read individually and then reviewed
  and discussed by all group members to identify
  internal and external validity, level of
  evidence, and clinical and statistical
  significance

60
Procedures cont.

• Information and findings from each article were
  summarized in table and narrative format
  according to the AACPDM methodology.

61
Results

• 8 articles addressing interventions that affect
  classroom function in children with ADHD or PDD
  from 1992 to present were located
• - 2 randomized control trials (level I)
• - 1 nonrandomized control trial (level III)
• - 4 single subject design (level IV)
• - 1 case study (level V)

62
Treatments

• Therapy Balls as seats in classroom
• Interactive Metronome/Video Game
• Weighted Vests
• Hug Machine
• Chewy
• Sensory Integration

63
Clinical Populations

• Diagnoses
• - 63 ADHD
• - 7 PDD
• - 17 Autism
• - 3 Autistic-like tendencies

• Age Range
• 2 to 13 years

64
Outcomes Measured

• Behavior Issues
• Social Skills
• Attention
• Sensory Motor
• Academic Cognitive Skills
• Hyperactivity
• Peer/Adult Interaction
• Mastery of Play
• Approach to New Activities
• Touch
• Non-engaged Behavior

65
Table 1 Study Characteristics
66
Results Table I
67
Results Table II
A Schilling et al (2003), B Shaffer et al
(2001), C Scheerer (1992), D Linderman
Stewart (1999), E Edelson et al. (1999), F
VandenBerg (2001), G Case-Smith Bryan (1999),
H Fertel-Daly (2001) Boldnot clinically
important
68
Conclusion

• There are two well design research studies and a
  limited number of articles found on this topic
• Some findings were positive although many were
  judges as having limited clinical importance
• Statistical significance was difficult to
  determine due to lack of recorded data
• Majority of outcomes were categorized as
  participation restrictions with some activity
  limitations.

69
Conclusion

• The interventions varied greatly in terms of
  type, intensity, and duration but all focused on
  improving the attention through the sensory
  systems
• While these types of treatments are commonly
  used, there is limited evidence that supports
  their effectiveness of improving classroom
  functioning
• Long term effects of treatment were not reported.

70
Clinical Bottom Line

• There is a small number of clinical trials
  investigating the effectiveness of sensory-based
  interventions for attention many are poorly
  designed with inconclusive results and
  questionable clinical importance
• More well designed studies are needed to confirm
  the effectiveness of treatments used to improve
  classroom functioning in children with ADHD or
  PDD.

71
Credits

• This review was completed in May of 2004 by
  Juliet Bartels, Michelle DoBas, Tammy Juern, and
  Steve Cope