Adjusting our FOCUS! Measuring Meaningful Clinical Outcomes

1
Adjusting our FOCUS!Measuring Meaningful
Clinical Outcomes
F Focus on the O Outcomes of C
Communication U Under S Six
2
Team FOCUS

• Prof. Nancy Thomas-Stonell, PI
• Dr. Bruce Oddson, Co-PI
• Dr. Peter Rosenbaum, Co-PI
• Dr. Karla Washington, Post-Doctoral Fellow
• Ms. Bernadette Robertson, Research Coordinator
• Ms. Joan Walker, Research Assistant
• Funding
• SickKids Foundation
• Canadian Institutes of Health Research (CIHR)
• Bloorview Childrens Hospital Foundation

3
Eleven Research Partners in Five Provinces Across
Canada

• Eastern Healthcare, St. Johns, NL
• Nova Scotia Hearing and Speech Centres, NS
• Beyond Words, Preschool Speech and Language
  Program, York Region, ON
• Waterloo Preschool Speech and Language Program,
  ON
• Holland Bloorview Kids Rehabilitation Hospital,
  ON
• Hamilton Preschool Speech and Language Service,
  ON
• Technology Access Clinic, ON
• Wellington-Dufferin Guelph Region Preschool Sp
  Lang Service, ON
• ErinoakKids Centre for Treatment and Development
  Halton-Peel Preschool Speech and Language
  Program, ON
• Calgary Health Region, AB
• BC Centre for Ability, BC

4
A Typical Day in the Life of a SLP!

• Example prepared with thanks to Laurie Graham

5
Preschool Speech-Language Pathologist

• 4 year old boy, A.B., presents for service
• Formal assessment results (currently available
  standardized tests) indicate a moderate speech
  and language disorder

6
Assessment reveals

• Has a lot of trouble with sounds
• i.e. ish instead of fish
• og instead of frog
• Has trouble with pronouns
• i.e. often says he instead of she

7
Parents reports the boy is

• extremely frustrated, has tantrums
• teased at school
• kids and teacher have trouble understanding him
• shy, evidence of low self-esteem

8
Clinical Goals for 8 week treatment block

• A.B. will produce /f/ in word initial position in
  structured settings 80 of the time.
• A.B. will produce she appropriately in phrases
  in structured settings 80 of the time.
• Speech-Language Pathologist documents parent
  comments in client file.

9
Eight weeks later

• During the last session, the Speech-Language
  Pathologist decides her goals have not been met
  as the child is only performing tasks at a 50
  success rate.
• The parents state that tantrums have decreased,
  A.B. is no longer being teased at school, and
  seems more confident in communicating.

10
The importance of parent comments

• The Speech-Language Pathologist has chosen to
  include many parent comments in her client notes
  because they give her an indication of the
  childs interactions with others.

11
The importance of parent comments

• Interaction is fundamental to the development of
  communication - the more you interact, the more
  you practice communication skills.
• The ability to communicate with peers and
  teachers is fundamental to academic and social
  success (i.e. group work).

12
Parent comments continued..

• Including parent comments in client notes,
  although recommended by regulatory bodies of the
  profession, is not required.
• There are no valid and reliable measures (i.e.
  tests/questionnaires) that capture the real-world
  changes observed by parents and clinicians!

13
Outcome measurement for A.B.

• The Speech-Language Pathologist wishes she could
  more thoroughly document the behavioural,
  interactive and social changes seen by A.B.s
  parent.
• She suspects that A.B.s gains are associated
  with therapy (not just normal development) but
  there is no way to prove her hypothesis.

14
Decision time

• There are other children on the waitlist.
• Given the limited funding available, A.B. is not
  offered a second block of therapy.

15
Increasing concern

• Clinicians, researchers and disability advocates
  are concerned that changes which may be important
  results of therapy are overlooked.

16
What are clinically meaningful outcomes?
17
World Health Organization (WHO) Health Frameworks

• In 1980 WHO (1980) came out with their first
  health framework the International Classification
  of Impairments, Disabilities and Handicaps
  (ICIDH)
• Impairment (whats broken)
• Disability (what cant you do)
• Handicap (limitations in the real world)

Impairment
Disability
Handicap
18
International Classification of Functioning,
Disability and Health (ICF-2003)Children Youth
Version (ICF-CY - 2007)
Health Condition
Body Functions Structures
Activities
Participation
Environmental Factors
Personal Factors
19
ICF ICF-CY Domains

• Body Functions Physiological
• (e.g., voice, oral motor, speech production)
• Body Structures anatomical
• (e.g., hearing loss CL/P)

20
ICF ICF-CY Domains

• Activities
• Tasks and actions by an individual.
• Capacity - performance of a task in a standard
  environment.
• Participation
• Involvement in a life situation.
• Performance performance of tasks in a in the
  current environment.

21
Contextual Factors

• Environmental Factors
• external influences on functioning and disability
  related to physical, social and attitudinal
  world. (stairs, culture, support system)
• Personal Factors
• internal influences on functioning and disability
  (personality influences on coping style)

22
ICF Health Framework

• Has positive and negative components.
• Uses a bi-directional model.
• Doesnt take developmental stages into account.
• e.g. temper tantrums/frustrations for
• 2 year olds

23
ICF-CY Health Framework

• New codes to capture the functional
  characteristics of a developing child. Expanded
  codes include
• Learning new skills
• Play
• Adaptability
• Persistence
• Exploration

24
Why use ICF-CY model?

• Outcomes need to be evaluated across ICF-CY
  domains.
• Several studies (Sarno, 1969, Aten, 1986) have
  noted poor correlation between body
  structure/function outcomes and activity and
  participation outcomes
• depends not only on skill levels, but also
  personality, coping skills, social support
  systems...

25
How do we measure these outcomes?
26
We need treatment outcome measures!

• We need outcomes measures to evaluate the impact
  of treatment on childrens lives.

27
Outcomes vs. Outcome Measures

• Any consequence of healthcare is an outcome!
• Outcome environment treatment client
  severity

28
There are many types of outcomes.

• Avoiding adverse affects of care (nobody dies)
• Improved physiologic status ( phonation time)
• Reduction in symptoms ( dysfluencies)
• Improved functional status (use telephone)
• Minimizing costs
• Minimizing length of care

29
Outcome Measure

• A treatment outcome measure is a validated test
  designed to measure change in function.
• It measures, in quantitative terms, the impact of
  routinely delivered care on clients lives.

30
Treatment Outcome Measures need to be proven to
work!

• Garbage in garbage out!
• Information generated by outcome studies is only
  useful if the measure is clinically useful and
  scientifically sound (van der Putten et al.,
  1999).

31
Why cant we use our standardized tests?
32
Standardized tests...

• Determine the presence or absence of a
  communication disorder. They do not change.
• They provide too little information (insufficient
  number and variety of items) to monitor progress.
• Huang, Hopkins Nippold (1997). Satisfaction
  with Standardized Language Testing A survey of
  Speech-Language Pathologists. Language Speech
  Hearing Services in Schools 28, 12-29.

33
Treatment Outcome measures

• Outcomes measures at a minimum need to be proven
  to reliably distinguish between children who
  improve from therapy and those who do not
  improve.

34
Why use treatment outcome measures?

• To improve treatment services in an
  evidence-based manner.
• To measure clinically important change.
• To determine optimal length for treatment.
• To select the best treatment approach for each
  child.

35
CASLPA Position Statement on Outcome Measures

• CASLPA encourages and supports the use and
  development of outcome measures by
  speech-language pathologists and audiologists
• Outcome measures should be used to improve
  practice in an evidence-based manner in the best
  interests of clients.
• CASLPA Position Statement on Outcome Measures
  May, 2010 (www.caslpa.ca)

35
ASHA Convention
November 18-20 2010
36
The FOCUS journey began in 1998

• Holland Bloorview Kids Rehabilitation Hospital
  wanted an outcome measure for speech-language
  therapy that could be used across programs.
• Diverse population
• CP/CLP/ABI

37
Began our search for a treatment outcome measure.

• TOMS and AusTOMS are very broad measures of
  change. Scale has many descriptors. Hard to know
  what changed.
• GAS (individualized and time consuming).
• ASHA NOMS had no proven reliability or validity.
  We completed a two-year study to evaluate the
  NOMS. Results indicated poor sensitivity to
  change.

38
What do we do now?

• with no existing valid, reliable and responsive
  communication outcome measure for preschool
  children available?

39
Development of the FOCUS

• F ocus on the
• O utcomes of
• C ommunication
• U nder
• S ix
• Thomas-Stonell, N., Oddson, B., Robertson, B.
  Rosenbaum, P. Development of the FOCUS (Focus on
  the Outcomes of Communication Under Six), a
  communication outcome measure for preschool
  children. Developmental Medicine and Child
  Neurology 2010, 5247-53.

40
Our Goal

• To develop a valid, reliable, responsive
  treatment outcome measure that captures real
  world changes following speech and language
  treatment.

41
Developing the FOCUS

• In the previous outcome study, we collected data
  from parents of 210 preschool children receiving
  speech-language treatment and their clinicians.
• (Thomas-Stonell, Oddson, Robertson
  Rosenbaum, Predicted and Observed Outcomes in
  Preschool Children Following Speech and Language
  Treatment Parent and Clinician Perspectives. JCD
  42 (2009) 29-42.)

42
Developing the FOCUS

• They were asked to describe the changes they
  observed in their child during/following therapy.
• My child is now able to
• What other changes did you see?
• Why is that important?

43
Method 6 Linked-Steps

• Content analysis of our descriptive data to
  create FOCUS items.
• 2. Test the measure with clinicians and families.
  
• 3. Revise the measure using the parent and
  clinician feedback.
• 4. Test the revised measure again.
• 5. Revise measure a second time.
• 6. Test measure a third time to obtain
  preliminary reliability and validity data.

44
Content Analysis

• Content analysis is the systematic, objective
  analysis of message characteristics to make
  valid inferences from text. (Neuendorf, 2002)

45
Content Analysis

• Identify recurring categories of change and
  calculate percentages of occurrence for each
  category.
• The recurring categories reflected the ICF-CY
  framework.

46
Coding Comments

• Working Slides

47
Developing the FOCUS

• FOCUS is driven by DATA,
• no preconceived ideas
• FOCUS items were developed from categories cited
  by gt10 of parents clinicians.
• Resulted in 200 items, reduced to 103.
• Items used parents own wording.

48
Sample Body Functions Item Development
49
Item Development Body Functions

• Parent Comment
• Pronounces words much more clearly
  (specifically F sounds, L sounds when prompted -
  he still has work to do with L's)
• Category/ICF-CY coding Body Functions
• Articulation Functions b320
• FOCUS Item
• My childs speech is clear.

50
Sample Activities/CapacityItem Development
51
Sample Item Development Activities

• Parent Comment
• Says more words. Put more words together.
• Category/ICF-CY coding Activities/Capacity
• Communicating producing
• Speaking d330
• FOCUS Item
• My child can string words together.

52
Sample ParticipationItem Development
53
Sample Item Development Participation

• Parent Comment
• His play with peers has improved in terms of
  sharing, turn-taking, following conversations,
  acting less aggressively.
• Category/ICF-CY coding Activities
  Participation
• Complex Interpersonal Interactions d720
• FOCUS Item
• My child plays well with other children.

54
Sample Personal FactorsItem Development
55
Sample Item Development Personal Factors

• Parent Comment
• More confident in playing with peers or
  entering a new group.
• Category/ICF-CY coding Personal Factors
• Coping Style/Behavior Pattern
• FOCUS Item
• My child makes friends easily.

56
Initial FOCUS 103 items

• Body Functions 9
• Activity/Capacity 28
• Participation/Performance 54
• Personal Factors 20
• Environmental Factors 3
• Percentages exceed 100 as some items had 2
  codes.

57
Criterion-referenced

• Performance is judged according to pre-stated
  criterion.
• Take a verbal snapshot of childs skills at
  Time 1 and Time 2 and use the changes in the
  scores to measure change.
• Developed a parent and a clinician version.

58
Response Categories 1
My child talks a lot.
Not at all like my child A little like my child Somewhat like my child A fair bit like my child Quite a bit like my child Very much like my child Exactly like my child
59
Response Categories 2
My child plays well with other children.
Can not do at all Can do with a great deal of help Can do with a lot of help Can do with a bit of help Some-times does without help Often does without help Can always do without help
60
Sample FOCUS Form
61
Sample FOCUS Items

• 1. My clients speech is clear.
• 2. My client speaks slowly when not understood.
• 3. My client can string words together.
• 4. My client speaks in complete sentences.
• 5. My client uses correct grammar when speaking.

62
Sample FOCUS Items

• 6. My client talks a lot.
• 7. My client is confident communicating with
  adults who know my client well.
• 8. My client uses language to communicate new
  ideas.

63
FOCUS Instructions

• FOCUS designed for children from birth to 6
  years.
• If children are too young to complete some of the
  items, parents and clinicians need to score the
  items as
• Not at all like my child.
• This allows these emerging skills to be measured.

64
Instruction Example

• A typical child of 15 months is probably only
  speaking in one-word phrases, so the response to
  the item
• My child uses correct grammar when speaking
• would be Not at all like my child.

65
FOCUS Definitions

• Talking, tell, speaks, speech
• and words refers to verbal speech.
• For example,
• My child talks a lot. refers to verbal
  communication.

66
FOCUS Definitions

• Communicating, conversations, participates
  and asking can be any form of communication.
• (e.g. pecs, AAC, sign)
• For example
• My child will ask for help.

67
FOCUS Definitions

• Some children using AAC began to verbalize
  during their speech therapy.
• This is a very important functional change.
• We needed to ensure that the FOCUS could capture
  this change.

68
FOCUS Phase 1 Testing (N 74)

• FOCUS revised using measurement science.
• Data driven!
• Items were deleted if
• Poor distribution of scores
• Poor completion rate
• Not sensitive to change
• Redundant
• Items not clear

69
Clinician and Parent Feedback

• Difficulty completing the school items in Nova
  Scotia, (more rural setting).
• Both parents and clinicians requested more
  questions for younger children.
• They suggested items such as babbling, imitation

70
Revisions

• Broadened definition of school.
• Added 5 new items for younger children.
• My child is reluctant to talk.
• My child takes turns.
• My child does not interact with others.
• My child is independent.
• My child uses immature language.
• My child uses words to request items.

71
Second FOCUS Testing (N 65)

• FOCUS reduced to 77 items, including 5 new items
  for young children.
• Tested again with different parents.
• High internal consistency indicated that the
  FOCUS items had some redundancy.
• Parents ? .98
• Clinicians ? .83
• FOCUS revised and reduced to 50 items.
• Thomas-Stonell et al., 2010

72
Results Item Distribution

• Initial FOCUS
• Body Functions 9
• Activities/Capacity 28
• Participation/Perf. 54
• Personal Factors 20
• Environ. Factors 3

• Final FOCUS
• Body Functions 2
• Activities/Capacity 34
• Participation/Perf. 56
• Personal Factors 10
• Environ. Factors 0

• Increased Activities and Participation items.
• One Body Function item remains (Speech Rate).

73
Results Item Distribution

• Initial FOCUS
• Body Functions 9
• Activities/Capacity 28
• Participation/Perf. 54
• Personal Factors 20
• Environ. Factors 3

• Final FOCUS
• Body Functions 2
• Activities/Capacity 34
• Participation/Perf. 56
• Personal Factors 10
• Environ. Factors 0

• Remaining FOCUS items demonstrated the most
  sensitivity to change.

74
Phase 3 Testing

• Factor analysis indicates
• one construct!
• FOCUS has 50 items.
• FOCUS takes 10 minutes to complete.

75
Communicative Participation

• Communication in life situations where
  knowledge, information, ideas or feelings are
  exchanged. (Eadie et al, 2006)
• Life situation communication within a social
  context.
• Exchange reciprocal nature of communication.

76
Communicative Participation

• The fundamental feature of communicative
  participation is the complex interaction between
  the speaker and the social context.

77
Where are we now?
Reliability and Validity Study CIHR 2009 - 2011
78
FOCUS Journey

• Evaluating other outcomes measures 1998
• Coding collected comments
  2002
• Seek development funds 2003
• FOCUS development study 2005
• FOCUS validation study
  2009

79
Reliability

• Parents completed the FOCUS twice, 7 days apart.
• Parent test-retest reliability was high!
• The same clinician scored the FOCUS twice during
  a 30 day no treatment interval (N 19).
• Clinician test-retest reliability was high.

80
Clinician Inter-Rater Reliability

• Two different clinicians administered the FOCUS
  on the same child twice within a 30 day
  no-treatment interval.
• Clinicians inter-rater reliability was also very
  high.

81
Validity Testing
82
Construct Validity

• Construct validity is the extent to which a
  measure correlates with the construct is was
  designed to measure. (Streiner Norman, 1995)
• Generally, a number of independent studies are
  required to establish the credibility of a
  measure.

83
Preliminary Validity Testing PEDS-QL

• Parents of 22 children completed the FOCUS and
  the Pediatric Quality of Life Inventory (PedsQL)
  at the start and end of a treatment block.
• Higher FOCUS scores at the end of treatment
  correlated with higher PedsQL total scores (r
  .466, p .029).

84
Preliminary Validity Testing PEDS-QL

• Higher FOCUS scores were specifically correlated
  with higher scores in the psychosocial domain -
  emotional, social and school functioning (r
  .518, p .013).
• Positive correlations between FOCUS scores and
  the PedsQL indicate that the FOCUS has construct
  validity.

85
Preliminary Validity TestingConstruct Hypothesis

• The FOCUS will measure more change during a
  Treatment Interval than during the Wait List
  Interval.
• (assuming treatment works!)

86
Demographics

• 43 preschool children with communication
  impairments from
• Holland Bloorview Kids Rehabilitation Hospital,
• KidsAbility
• Alberta Health Services.
• Mean age 2.7 yrs. (age range 1.25 4.8 yrs)
• 63 of participants were boys.

87
Communication Function Classification System
(CFCS) (Hidecker, 2008)

• Level I Effective Sender and Receiver with
  unfamiliar and familiar partners
• Level II Effective but slower paced Sender
  and/or Receiver with unfamiliar and familiar
  partners
  Level III Effective Sender and Receiver with
  familiar partners
• Level IV Sometimes Effective Sender and/or
  Receiver with familiar partners
• Level V Seldom Effective Sender and Receiver
  even with familiar partners

88
Severity

• Children ranged in CFCS from 1 (mild) to 5
  (severe).
• The majority of the children (70) were
  classified in Level IV Level V.
• 51 of the children also had a diagnosis of
  developmental delay.

89
Methods

• Parents and clinicians completed the FOCUS at
  assessment, start and completion of a treatment
  block.
• On average, there were 36 days between assessment
  and start of treatment.
• On average, there were 107 days between the start
  and end of treatment.
• Ave amount of treatment provided was 9.7 hours.

90
Preliminary FOCUS Results

• Significant change was noted by both parents and
  clinicians after treatment. No change was noted
  during the waiting list period.
• Parents and clinicians score identical amounts of
  change from T2 T3.
• FOCUS demonstrates both stability and the ability
  to measure change.

91
Validity Testing VABS II(Washington, 2011)

• Progress measured by the FOCUS was compared to
  progress measured by the Vineland Adaptive
  Behavior Scales (VABS-II)
• VABS II selected as it assesses communication
  skills as well as broader participation (i.e.,
  Socialization) skills.

92
Method

• Sixty-seven parents of preschool children ages 3
  to 6 years old with communication disorders
  participated.
• Parents recruited from one of three agencies
• Holland Bloorview Kids Rehabilitation Hospital
  (Integrated Education and Therapy Program)
• Toronto Preschool Speech and Language Services
  West Quadrant
• University of Western Ontario, tykeTALK

93
Preschoolers Group Description

• Group 1 Communication Disorder only and
  receiving intervention
• Group 2 Communication Disorder and a
  developmental mobility impairment and receiving
  intervention
• Group 3 - Control participants, on waitlist for
  intervention

94
Method

• Fifty-two children received direct group or
  individual intervention with an SLP
• Fifteen children acted as waiting list controls.
  
• A different SLP completed VABS-II and FOCUS by
  telephone with the parent following treatment.

95
VABS-II
Measure Purpose
Vineland Adaptive Behavior Scales II (VABS-II Sparrow, Cicchetti, Balla, 2005) Assessment of everyday adaptations for four major domains, including socialization. Raw scores used to establish participation skills

• Interpersonal Relationships
• Play Leisure Time
• Coping Skills

96
VABS-II

• Interpersonal Relationships
• Demonstrates friendship seeking behaviors with
  others the same age (e.g., Do you want to
  play?)
• Play Leisure Time
• Plays simple make-believe activities with others
  (e.g., plays dress-up, pretends to be
  superheroes)
• Coping Skills
• Ends conversation appropriately (e.g., says
  Good-bye).

97
VABS-II Response Options
Response Option Description
2 Usually
1 Sometimes or partially
0 Never
DK Dont know
98
Results

• Changes on the FOCUS and VABS-II Socialization
  domain are significantly correlated.
• Participants receiving intervention experienced
  significantly greater gains compared to controls

99
Parent Comments - Intervention

• Re-check 1
• Sometimes will wait his turn, will share with
  others and can follow nonverbals

• Assessment
• Will play at cousin's house, can be sociable
  and hands on, initiates and makes requests

Parent Comments - Intervention

• Re-check 2
• More confidence, more likely to initiate,
  sometimes asks to play with others

100
Discussion

• Correlations between the FOCUS and the VABS-II
  Socialization domain demonstrates construct
  validity
• The FOCUS is another measure of Participation,
  although somewhat different from the VABS-II
• SLP administered/supervised
• Shorter administration time
• Sensitive to changes in communication-level
  participation

101
Participation Predictors

• Wanted to know which factors predicted the
  Participation changes measured by the FOCUS
• Multiple regression analyses were preformed on
  the results of the 52 children who received
  therapy.

102
Predictors
Measure/Procedure Predictor Variable
Demographic Information Age, sex
The matrices component of the Kaufman Brief Intelligence Test II (K-BIT2 Kaufman Kaufman, 2004) Nonverbal IQ
Communication Domain of the Vineland Adaptive Behavior Scales (VABS-II Sparrow et al., 2005) Pre-Tx Communication (parent)
Communication Function Classification System (CFCS Hidecker et al., 2008) Pre-Tx Communication (SLP)
Socialization Domain of the VABS-II Pre-Tx Participation Skills
Physician/SLP Report Presence of a Physical Disability
SLP/Parent Report English as a Second Language
SLP Report Amount of Direct Intervention
103
General Results

• SLP treatment has a positive effect on childrens
  ability to participate in their world!
• Specific factors unique to children predicted
  improved Participation skills
• Factors unique to childrens environment may be
  predictive as well

104
Case Study 1Pretreatment Participation Skills

• Parent Description of Participation Skills
• Sometimes goes to parties on weekends. Will go
  to grandmother's house. Does play well with other
  kids. Can take turns.

105
Case Studies

• Three Preschool Children Attending
  Speech-Language Therapy
• (Washington, 2010)

106
Case Study 1 Child with Communication
Disorder and Mobility Impairment

• 5 yrs 3 month old boy with Pierre Robin Syndrome.
• Mild physical impairment (GMFCS Level 1) due to
  club foot.
• Some fine motor difficulties (OT)
• Communication disorder secondary to cleft lip and
  palate.

107
Case Study 1Pretreatment Communication Skills

• Describe your childs communication abilities.
  (e.g., listening and talking skills)

Parent
SLP
108
Case Study 1Pretreatment Communication Skills

• CFCS level 3 (Hidecker, 2008)
• Effective sender and receiver with familiar
  partners
• Difficulties with speech sounds and resonance.
  Mild expressive language difficulties.
• Parent Description of Communication Skills
• Okay communication. Pronunciation is hard for
  strangers to understand. Better with repetition.
  Makes it hard for others to understand him, but
  great personality.

109
Case Study 1Pretreatment Participation Skills

• Describe your childs participation skills.
• (e.g., Does your child play at other childrens
  homes or go to birthday parties or other social
  events?)

SLP Parent
110
Case Study 1Pretreatment Participation Skills

• Parent Description of Participation Skills
• Sometimes goes to parties on weekends. Will go
  to grandmother's house. Does play well with other
  kids. Can take turns.

111
Case Study 1 Treatment

• 15.5 hours of group treatment
• Total duration 29 weeks.
• Treatment Goals
• Mark final consonants in words with hard contact,
  
• /t,d,f,s/-all word positions,
• Improve consonant blends,
• Reduce nasal turbulence on fricatives.

112
Case Study 1 Pre - Post Treatment Scores

• Parent FOCUS Change
• Pre 280
• Post 336 56 points
• VABS Communication
• Pre 120
• Post 148 28 points
• VABS Socialization
• Pre 98
• Post 146 48 points

113
Case Study 1Post Treatment Communication Skills

• Describe your childs communication abilities
  since the last interview.

Parent
SLP
SLP
Parent
Orange pretreatment Green
Post Treatment
114
Case Study 1Post Treatment Communication Skills

• Parent Description of Communication Skills
• /l / /s/ have improved. He is better. Clearer
  to others, especially non-family members. Now he
  is using more and longer sentences.
• This is very important because other people can
  understand him better now.

115
Case Study 1 Post Treatment Participation
Skills

• Describe your childs participation abilities
  since the last interview.

SLP Parent
SLP Parent
Orange pretreatment Green
Post Treatment
116
Case Study 1Post treatment Participation Skills

• Parent Description of Participation Skills
• He takes turns and listens better. He responds
  to questions better.
• This is important because he can be with other
  people better and not be sad.
• Other Observations
• He has become better overall. He is talking and
  playing more.

117
Case Study 1High Change FOCUS Items gt 3

• My childs speech is clearer. 5
• My child can string words together.
• My child speaks in complete sentences.
• My child uses correct grammar when speaking.
• My child can communicate independently with
  adults who do not know my child well.
  

118
Case Study 1High Change FOCUS Items

• My childs communication skills get in the way of
  learning.
• My child will try to carry on a conversation with
  adults who do not know my child well.
• My child can communicate effectively with adults
  who do not know my child well.

119
Case Study 1High Change FOCUS Items

• Many of the play and peer items were scored at
  level 6 (Often does without help) at the start of
  treatment.
• Most of these items also improved, but they could
  only improve by 1 point.
• Therefore they were not included in the high
  change items described above.

120
Case Study 2
121
Case Study 2 Child with Communication
Disorder and Mobility Impairment

• 4 yrs 1 month old boy with Cerebral Palsy
  (Spastic Quad).
• GMFM 4
• Uses a wheelchair most of the time Also has a
  walker.
• CFCS 1
• An effective communicator in most situations

122
Case Study 2Pretreatment Communication Skills

• Describe your childs communication abilities.
  (e.g., listening and talking skills)

SLP
Parent
123
Case Study 2Pretreatment Communication Skills

• Parent Description of Communication Skills
• Still developing vocabulary. Using 4-5 word
  sentences. Learning new words and word
  approximations, but I dont always know what he
  wants which leads to frustration on his part.

124
Case Study 2Pretreatment Participation Skills

• Describe your childs participation skills.
• (e.g., Does your child play at other childrens
  homes or go to birthday parties or other social
  events?)

SLP
Parent
125
Case Study 2Pretreatment Participation Skills

• Parent Description of Participation Skills
• Not always sociable. No mobility issues affect
  this. He does not imitate and changing activities
  is difficult.

126
Case Study 2 Treatment

• 41 hours of group treatment
• Total duration 29 weeks.
• Treatment Goals
• Increase vocabulary.
• Improve understanding and use of concepts.
• Improve understanding and use of action words.
• Appropriate responses to questions.
• Expand sentence length.

127
Case Study 2 Pre - Post Treatment Scores

• Parent FOCUS Change
• Pre 270
• Post 246 - 24 points
• Follow-UP 309 63 points ( 39 points)
• VABS Total Score
• Pre 89
• Post 135 46 points
• Follow-UP 136 1 point ( 47 points)

128
Case Study 2Post Treatment Communication Skills

• Describe your childs communication abilities
  since the last interview.

SLP Parent
SLP Parent
Orange pretreatment Green
Post Treatment
129
Case Study 2Post Treatment Communication Skills

• Parent Description of Communication Skills
• He is talking a lot more now and answers
  questions appropriately.
• This is very important to us because now we are
  sure about what he wants/needs. We feel better
  about addressing his needs. We feel like better
  parents.

130
Case Study 2Post Treatment Participation Skills

• Describe your childs participation skills.
• (e.g., Does your child play at other childrens
  homes or go to birthday parties or other social
  events?)

SLP
Parent
SLP
Parent
Orange pretreatment Green
Post Treatment
131
Case Study 2Post treatment Participation Skills

• Parent Description of Participation Skills
• Increased initiation noted. Increased attention
  during circle time.
• This is important because he can interact with
  others.
• Other Observations
• He is more aware that his actions lead to
  results. Have an impact on others and his
  environment.

132
Case Study 2Follow-Up Communication Skills

• Describe your childs communication abilities
  since the last interview.

SLP Parent
SLP
Parent
SLP Parent
Orange pretreatment Green
Post Treatment Blue Follow-UP
133
Case Study 2Follow-Up Communication Skills

• Parent Description of Communication Skills
• May not say much, but vocabulary has definitely
  improved. Increased grammar (possessive form).
• This is important because it helps him
  communicate with his peers and allows him to find
  new ways of expressing himself.

134
Case Study 2Follow-Up Participation Skills

• Describe your childs participation skills.

SLP Parent
SLP
Parent
SLP
Parent
Orange pretreatment Green
Post Treatment Blue Follow-Up
135
Case Study 2Follow-Up Participation Skills

• Parent Description of Participation Skills
• Great. He is highly engaged in circle time. He
  still needs help physically but is willing to
  participate. This is great because he has
  initiative!
• Other Observations
• He has better memory and is more curious about
  world. He is showing likes/dislikes more and
  starting to assert himself beyond food preference
  (e.g. Dressing).

136
Case Study 3
137
Case Study 3 Child with Communication Disorder

• 3 yrs 6 month old boy.
• Severe speech and language disorder.
• Difficulties with both receptive and expressive
  language
• CFCS Level 4
• Inconsistent Sender and/or Receiver with
  familiar partners

138
Case Study 3Pretreatment Communication Skills

• Describe your childs communication abilities.
  (e.g., listening and talking skills).

Parent
SLP
139
Case Study 3Pretreatment Communication Skills

• Parent Description of Communication Skills
• Poor clarity of speech. Delayed grammar.
• Late talker.
• People dont understand him. He is limited in
  expressing himself to others because they dont
  understand.

140
Case Study 3Pretreatment Participation Skills

• Describe your childs participation skills.
• (e.g., Does your child play at other childrens
  homes or go to birthday parties or other social
  events?)

SLP Parent
141
Case Study 3Pretreatment Participation Skills

• Parent Description of Participation Skills
• He is very sociable and entertaining.
• He makes friends easily.

142
Case Study 3 Treatment

• 8 hours of group treatment
• Total duration 5 weeks
• Treatment Goals
• Improve expressive language.
• Increase MLU / expand phrases.
• Teach vocabulary using themes.

143
Case Study 3 Pre - Post Treatment Scores

• Parent FOCUS Change
• Pre 246
• Post 296 50 points
• VABS Communication
• Pre 94
• Post 113 19 points
• VABS Socialization
• Pre 117
• Post 134 17 points

144
Case Study 3Post Treatment Communication Skills

• Describe your childs communication abilities.
  (e.g., listening and talking skills).

Parent
SLP
Orange pretreatment Green
Post Treatment
145
Case Study 3 Post Treatment Communication

• Parent Description of Communication Skills
• His speech still not clear.
• He is trying to make sentences.
• His vocabulary has improved but he still has
  difficulty with concepts such as
  first/middle/last and with following
  instructions.

146
Case Study 3 Post Treatment Participation

• Describe your childs participation skills.
• (e.g., Does your child play at other childrens
  homes or go to birthday parties or other social
  events?)

SLP Parent
Parent
Orange pretreatment Green
Post Treatment
147
Case Study 3 Post Treatment Participation
Skills

• Parent Description of Participation Skills
• He is talking more. He finds games to play. He
  initiates more.
• He is more likeable and has more friends now.
• Other Observations
• He is more confident now.

148
Case Study 3High Change FOCUS Items gt3

• My childs communication skills get in the way of
  learning. 5
• My childs communication skills limit his
  independence. 5
• My child waits for her/his turn to talk. 4

149
Case Study 3High Change FOCUS Items 3

• My childs is confident communicating with adults
  who do not know my child well.
• My child is understood the first time when s/he
  is talking with other children.
• My child takes turns.
• My child can tell stories that make sense.

150
Case Study 3High Change FOCUS Items

• Even though the parent did not rate the
  communication skills as improved, she felt that
  participation skills had improved a lot!
• She was no longer concerned that communication
  skills were interfering with independence and
  learning.

151
FOCUS

• Limitations
• The 7 point rating scale may not have been
  sensitive enough to capture communication changes
  (change from low ability to average ability).

152
Discussion

• Improvement was measured by the FOCUS, VABS
  communication and VABS socialization domain
  scores.
• In Case 1 and 2, the parent noted improved
  communication skills. In Case 3, the parent did
  not rate communication skills as improved.
• For all children, there were improvements in
  participation scores!

153
FOCUS

• An outcome measure that only measured changes in
  specific communication skills (e.g., MLU,
  expressive grammar, articulation) would have
  missed many of the positive changes associated
  with treatment.
• Even when there were no identified concerns with
  participation pretreatment, improvements were
  noted after treatment.

154
Summary
155
Summary

• The use of a newly developed measure of
  paediatric participation outcomes, the FOCUS,
  has provided evidentiary support that speech and
  language intervention can have a broad and
  positive effect on progress in participation
  skills!

155
ASHA Convention
November 18-20 2010
156
Examples of Real Life Impact of Speech-Language
Therapy

• More sociable.
• Understood better by others.
• Improved attention and listening skills.
• Improved play with peers.
• Increased communication with others.
• Less frustration/improved confidence.
• Fewer negative behaviors/temper tantrums.

157
Summary

• An outcome measure that focuses solely on speech
  and language skills (i.e. impairments) would miss
  the large changes associated with communicative
  participation.

158
Summary

• Preliminary results suggest that
• The FOCUS is successfully measuring the real
  world communication outcomes corresponding at
  the ICF level of participation.

159
FOCUS Journey continues

• Evaluating other outcomes measures 1998
• Coding collected comments
  2002
• Seek development funds 2003
• FOCUS development study 2005
• FOCUS validation study
  2009
• Dissemination of the FOCUS 2011

160
Final Thoughts

• The evaluation of outcomes in the field of
  speech-language pathology would benefit from the
  development and implementation of additional
  measures of communicative participation.

161
Acknowledgements

• A special thank you to all of the families and
  clinicians who participated in these studies.

162
nthomasstonell_at_hollandbloorview.ca