Early Childhood and Accountability

1
Early Childhood and Accountability

• OSEPs Project Directors Meeting
• August 2006

2

• Jennifer Tschantz, OSEP
• Lou Danielson, OSEP
• Kathleen Hebbeler, SRI International
• Corry Robinson, University of CO
• Mary McLean, University of WI-Milwaukee
• Beth Rous, University of Kentucky
• Pat Trohanis, University of NC, NECTAC

3
Objectives

• Provide background to OSEPs early childhood
  outcomes work
• Hear diverse perspectives on critical issues
  related to early childhood assessment and
  accountability
• Facilitate a dialogue on these critical issues

4
Why the focus on early childhood outcomes?

• Improve results for young children with
  disabilities and their families
• Meeting a need in the field
• Development of outcomes for general early
  childhood programs
• Address GPRA, PART, and IDEA 2004

5
Approach

• Short-term obtain data from the States
  regarding child and family outcome GPRA
  indicators used to assess program performance at
  the Federal Level
• Long-term support the development of State
  early childhood outcome data collection used for
  program improvement by States, local programs and
  service providers
• On-going stakeholder involvement

6
OSEPs Key Investments

• Early Childhood Outcomes (ECO) Center
• Funded Fall 2003
• Provides leadership and technical guidance
• General Supervision Enhancement Grants (GSEGs)
• 2004 18 awards focused on ec outcomes
• 2006 9 award focused on ec outcomes
• National Early Childhood Technical Assistance
  Center (NECTAC)
• Provision of TA and 2 national conferences

7
State Performance Plan (SPP) and Annual
Performance Report(APR)

• Each state required to submit SPP in December
  2005 and an APR beginning Feb. 2007
• Part C SPP has 14 indicators
• Part B SPP (includes preschool) has 20 indicators
• Early childhood outcomes are part of the SPP/APR

8
Critical Events

• Spring 2005 Public input on what should be
  collected with regard to child and family
  outcomes
• Summer 2005 OSEP released the reporting
  requirements
• December 2005 States submitted their plans for
  outcome data collection in their State
  Performance Plan
• Spring 2006 States begin collecting data
• February 2007 Status and entry data due
• February 2008 first progress data due

9
OSEP Reporting Requirements Part C and Preschool
Child Outcomes

• Percent of children who demonstrate improved
• Positive social emotional skills (including
  positive social relationships)
• Acquisition and use of knowledge and skills
  (including early language/communication and
  early literacy)
• Use of appropriate behaviors to meet their needs

10
Reporting Categories

• of children who reach or maintain functioning
  at a level comparable to same-age peers
• of children who improve functioning but do not
  achieve functioning comparable to same age peers
• c. of children who do not improve functioning

3 outcomes x 3 percentages 9 numbers
11
Proposed Change to Reporting Categories
CURRENT
a. of children who reach or maintain functioning at a level comparable to same-aged peers

b. of children who improve functioning (not in a)
c. of children who did not improve functioning
PROPOSED CHANGE
a. of children who maintain functioning at a level comparable to same-aged peers
b. of children who improved functioning to reach a level comparable to same-aged peers
c. of children who improve functioning but did not reach a level comparable to same-aged peers
d. of children who did not improve functioning
12
Additional information about child indicators

• Child must be in program at least 6 months
• Outcomes are broad and functional
• All outcome areas apply to all children
  regardless of area receiving services
• These progress indicators compare entry to exit
  data for each child, requiring a minimum of two
  data points
• Not mandating assessment tools, States have
  flexibility to choose own tool or tools.

13
Analysis of State Performance Plans

• ECO summarized State Plans for collecting child
  outcome data
• Based on SPPs submitted in December 2005
• Many States doing more than what they sent in and
  many state plans have evolved since December

14
SPP AnalysisPart C Outcomes Data Sources
15
SPP AnalysisPreschool Outcomes Data Sources
16
Commonly Reported Assessment Instruments Part C

• Of 28 states who listed specific assessment
  instruments
• HELP- 15 states
• BDI/BDI-2- 13 states
• AEPS- 11 states
• Creative Curriculum- 6 states
• ELAP- 6 states
• Not yet determined- 23 states

17
Commonly Reported Assessment Instruments
Preschool

• Of 31 states who listed specific assessment
  instruments
• BDI/BDI-2- 9 states
• Creative Curriculum- 8 states
• Brigance- 7 states
• High Scope COR- 6 states
• AEPS- 5 states
• State developed assessments- 7 states
• Not yet determined or not reported- 27 states

18
State approaches to assessments

• One assessment selected by state
• List of assessments developed by state programs
  pick
• Programs can use whatever they have been using

19
ECO Child Outcomes Summary Form (COSF)

• States need to be able to aggregate data across
  tools need a common metric to which data from
  different assessments can be converted
• COSF
• Provides scores directly on each of the 3
  outcomes
• Allows different assessment data to be
  transformed to a common metric (1 to 7 scale)
• Allows for increments of significant progress to
  be tracked over time
• Allows for any degree of progress to be tracked
  over time

20
(No Transcript)
21
Use of the form

• ECO envisioned the form as the final step in a
  team process where the childs functioning was
  discussed and consensus was reached
• Alternatives One professional completing, team
  members completing the form individually
• Behind the Scenes Alternative Converting
  online assessment data directly to the 7-point
  scale

22
Current Activities

• States currently focused on training
• Development of guidance materials on the COSF
• Reliability studies on the COSF
• Encouraging states to start thinking about how
  they will use these data

23
Summary

• All States are required to submit data on 3
  outcomes for all children participating in Part C
  and Part B Preschool programs.
• States implementing a variety of approaches to
  produce these data
• Information from assessments are critical
• Single assessment statewide
• Small set of approved assessments
• Any assessment in use

24
Summary

• States are making critical decisions related to
  assessment right now.

25

• Questions for the Panel

26
Assessment Tools / Use of Data

• Is it professionally acceptable (and will the
  data be valid) to use instruments designed for
  screening and eligibility, progress monitoring,
  or other purposes for accountability?

27
Assessment Tools / Use of Data

• Can and should the same assessment data be used
  for accountability, program improvement and
  individual progress monitoring purposes?
• Can data providers are collecting for other
  purposes (i.e., progress monitoring) be credible
  for accountability?

28
Training

• What is your sense of the current level of
  expertise of the field with regard to early
  childhood assessment?
• What has to be done to increase the overall level
  of knowledge of providers?

29
Families

• What is the role of the family in early childhood
  assessment for accountability purposes?
• How do we include families in a meaningful way?

30
Assessment Practices

• How can we ensure that accountability has a
  positive impact on assessment practices in early
  childhood and promotes authentic assessment?