Documenting Family Outcomes:

1
Documenting Family Outcomes

• NECTAC National TA Meeting
• Measuring Child and Family Outcomes
• April 25, 2006
• Albuquerque, NM
• Don Bailey

2
Goals for Today

• Describe the Part C and Preschool family
  indicators
• Summarize what states are doing now
• Think about issues and next steps

3
Three important questions about services for
families

• What services/supports were provided to families?

4
Three important questions about services for
families

• What services/supports were provided to families?
• Were these services/supports satisfactory to
  families?

5
Three important questions about services for
families

• What services/supports were provided to families?
• Were these services/supports satisfactory to
  families?
• Were family outcomes actually achieved?

6
What is a Family Outcome?

• A benefit experienced by families as a result of
  services and supports received
• An outcome is not the receipt of services or
  satisfaction with services
• An outcome is what happens as a result of
  services provided to families

7
ECO Recommended Family Outcomes
ECO

• Understand their childs strengths, abilities,
  and special needs
• Know their rights and advocate effectively for
  their children
• Help their children develop and learn
• Have support systems
• Access desired services, programs, activities in
  their community

8
ECO Family Outcomes and OSEP APR Requirements
Part C
ECO
OSEP

• Understand their childs strengths, abilities,
  and special needs
• Know their rights and advocate effectively for
  their children
• Help their children develop and learn
• Have support systems
• Access desired services, programs, activities in
  their community

• Percent of families participating in Part C who
  report that EI services have helped the family
• Know their rights
• Effectively communicate their childrens needs
• Help their children develop and learn

9
Four important questions about services for
families

• What services/supports were provided to families?
• Were these services/supports satisfactory to
  families?
• Were the services/supports helpful to families in
  achieving family outcomes?
• Were family outcomes actually achieved?

10
Examples

• My family was given information about our rights
• The information we received about family rights
  was understandable
• The information we received from EIP helped us
  understand our rights
• We know our rights and what to do if we are not
  satisfied.

11
What are states doing right now?

• Analysis based on December reports
• Probably out of date already lots of changes
  are happening!
• Cross-state consistency will enhance national
  reporting

12
SPP Results- Data Sources

• Family surveys- 54 of 56 states
• Family interviews- 6 states
• Family focus groups- 4 states
• Record review- 1 state

13
Family Surveys

• State developed- 21 states
• Based on ECO survey- 5 states
• Based on NCSEAM survey- 4 states
• NCSEAM survey- 19 states
• Reviewing NCSEAM survey- 12
• ECO survey- 10 states
• Reviewing ECO survey- 11
• Other surveys- 2 states

14
When (and how often) will family data be
collected?

• Annually (in general)- 11 states
• Annually in certain month- 14
• Annually at IFSP reviews- 4 states
• At/near exit- 8 states
• Entry and exit- 3 states
• Undecided/not reported- 12 states

15
Which families will be included?

• Sampling 14 states
• All families- 37 states
• Who have participated for at least 6 months- 16
  states
• Collect on all families but report to OSEP those
  who have participated at least 6 months- 2 states
• Undecided/ not reported- 5 states

16
Data collection strategies

• Personally hand survey to families, with return
  envelope-10 states
• Mail to distribute and return- 5
• Family organizations, e.g. PTIs or volunteers
  available to help families respond
• Telephone options for families who prefer
  responding verbally
• Translators available for multiple languages

17
Observations on the data

• Different response options were used
• Yes, no, dont know
• Strongly disagree to strongly agree
• Different criteria for positive responses
• Strongly agree and agree
• Only strongly agree

18
Very preliminary preview of targets

• Only 2 states provided targets
• One based targets on increasing the percent of
  families reporting agree or strongly agree
• One based targets on increasing the number of
  families who strongly agreed

19
Some quick observations

• States are trying to figure this out
• There are lots of variables that will likely
  affect the data states collect and report
• Right now it would be very hard to say that we
  are heading towards cross-state comparability in
  data

20
Why did the ECO Center decide to develop an
assessment tool?

• We wanted a tool with a direct relationship to
  the 5 proposed outcomes
• We wanted to try an alternative format to
  existing instruments
• We wanted to give families, programs, and states
  an option to consider during this era of rapid
  changes and challenges
• We needed to give states a way to collect APR
  data
• Ultimately wanted a tool to which families would
  respond positively (face validity, social
  validity)

21
ECO ScaleFamily Outcomes Survey

• Three items for each of the five proposed family
  outcomes
• Each item rated on a scale from 1-7
• Descriptive statements for ratings of 1, 3, 5, 7
• Blanks left for in between ratings of 2, 4, 6
• Three additional items for states to use in
  responding to APR requirements (two versions, one
  for Part C, one for Part B

22
Helping your child develop and learn 7. All
parents help their children develop and learn,
but sometimes it can be hard to know what to do.
How would you describe your ability to help your
child develop and learn?
8. All parents help their children behave the way
they would like, but sometimes it can be hard to
know what to do. How would you describe your
ability to help your child learn to behave the
way you would like?
9. Your family has worked with professionals to
develop a plan to help your child learn new
skills and behaviors. How much are you able to
help your child learn or practice these new
skills at home or in your community?
23
Your feelings about early intervention. 16. To
what extent has early intervention helped your
family know and understand your rights?
17. To what extent has early intervention helped
your family effectively communicate your childs
needs?
18. To what extent has early intervention helped
your family be able to help your child develop
and learn?
24
ECO Family Outcomes and OSEP APR Preschool
Requirements
ECO
OSEP

• Understand their childs strengths, abilities,
  and special needs
• Know their rights and advocate effectively for
  their children
• Help their children develop and learn
• Have support systems
• Access desired services, programs, activities in
  their community

• Percent of parents with a child receiving special
  education services who report that schools
  facilitated parent involvement as a means of
  improving services and results for children with
  disabilities

25
No Child Left Behind Definition of Parent
Involvement

• The participation of parents in regular, 2-way
  and meaningful communication involving student
  academic learning and other school activities,
  including/ensuring that
• Parents play an integral role in assisting their
  childs learning
• Parents are encouraged to be actively involved in
  their childs education at school
• Parents are full partners in their childs
  education and are included, as appropriate, in
  decision-making and on advisory committees to
  assist in the education of their child

26
Your feelings about your childs school. 16. To
what extent has your childs preschool program
helped you know how to help your child learn?
17. To what extent has your childs preschool
encouraged you to be actively involved in your
childs education at school?
18. To what extent has your childs preschool
included you as a full partner in making
decisions about your childs education?
27
What have we learned so far?

• Parents have been very positive about the scale
• It takes 8-15 minutes to complete
• They understand and like the format and content
• They feel the wording is not critical or
  evaluative, but more descriptive of where they
  are
• They would be more honest in completing the
  instrument if they knew that the professionals
  working with their child did not see their
  individual responses
• State and local programs also are very positive
  about its potential usefulness

28
Understanding Your Childs Strengths, Abilities
Special Needs
29
Understanding Your Childs Strengths, Abilities
Special Needs
30
Understanding Your Childs Strengths, Abilities
Special Needs
31
Knowing Your Rights AdvocatingFor Your Child
32
Knowing Your Rights AdvocatingFor Your Child
33
Knowing Your Rights AdvocatingFor Your Child
34
Helping Your Child Develop Learn
35
Helping Your Child Develop Learn
36
Helping Your Child Develop Learn
37
Having Support Systems
38
Having Support Systems
39
Having Support Systems
40
Accessing Your Community
41
Accessing Your Community
42
Accessing Your Community
43
Knowing and Understanding Rights
44
Knowing and Understanding Rights
45
Effectively Communicate Your Childs Needs
46
Helping your child develop and learn
47
Helping your child develop and learn
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Next Steps

• Conduct initial pilot test with 1000 families in
  IL 1000 in TX (Spring 06)
• Examine patterns of responses, internal
  consistency (Summer 06)
• Revise instrument based on pilot test (Summer and
  Fall 06)
• Conduct larger field-test with 5000 families in
  IL 5000 in TX (Fall and Spring 06)
• Develop web-based alternative for families to
  respond (Fall 06)

52
Other hopeful activities

• Develop laptop version for use with low-literacy
  families and/or multiple languages
• Create a mechanism whereby we can support states
  in data collection and analysis
• Create a national database to examine family
  outcomes over time and as a function of program
  practices
• Develop ways that programs can use data at level
  of state, local, and individual family to improve
  practice

53
Concluding Statements

• Family outcomes constitute an important indicator
  of program effectiveness
• Measuring family outcomes needs to be done in a
  way that makes sense to families, is easily
  summarized, and is useful for program improvement
• We look forward to working with you on this
  important effort