Improving Data Quality and Validity

1
Improving Data Quality and Validity

• ASR Winter Meeting
• January 29, 2009
• Tempe, AZ

2
Todays Focus

• New race/ethnicity codes
• Requirements for collecting, maintaining and
  reporting
• Potential impact on accountability
• Graduation rate in new regulations
• Calculation of rate
• Data necessary to calculate
• Statistical Arguments for new workbook
• Next steps for subgroup

3
New Race/Ethnicity Codes

• Became final in October 2007
• Addresses 3 issues
• How data will be collected from students and
  staff
• How data will be aggregated when reporting
• How data on multiple races will be reported and
  aggregated for ESEA (NCLB)
• Includes timeline for implementation

4
New Race/Ethnicity Codes

• Data Collection
• Two part response required
• Is student or staff Hispanic/Latino or of Spanish
  origin?
• Select from one or more of 5 racial groups
• American Indian or Alaskan Native
• Asian
• Black or African American
• Native Hawaiian or Other Pacific Islander
• White

5
New Race/Ethnicity Codes

• Definitions
• Hispanic/Latino
• means a person of Cuban, Mexican, Puerto Rican,
  South or Central American, or other Spanish
  culture or origin, regardless of race
• American Indian or Alaskan Native
• means a person having origins in any of the
  original peoples of North and South America,
  including Central America, who maintains a tribal
  affiliation or community attachment

6
New Race/Ethnicity Codes

• Definitions (cont)
• Asian
• means a person having origins in any of the
  original peoples of the Far East, Southeast Asia,
  or the Indian subcontinent including, for
  example, Cambodia, China, India, Japan, Korea,
  Malaysia, Pakistan, the Philippine Islands,
  Thailand and Vietnam
• Black or African American
• means a person having origins in any of the Black
  racial groups of Africa

7
New Race/Ethnicity Codes

• Definitions (cont)
• Native Hawaiian or Other Pacific Islander
• means a person having origins in any of the
  original peoples of Hawaii, Guam, Samoa, or other
  Pacific Islands
• White
• means a person having origins in any of the
  original peoples of Europe, the Middle East, or
  North Africa

8
New Race/Ethnicity Codes

• Reporting in 7 Categories
• Hispanic/Latino by each race and Not
  Hispanic/Latino by each race
• American Indian or Alaska Native
• Asian
• Black or African American
• Native Hawaiian or Other Pacific Islander
• White
• Two or more races

9
New Race/Ethnicity Codes

• Potential impact on ESEA (NCLB)
• Regs say
• states will continue to have discretion in
  determining which racial group are major
• Using data collected at the school level,
  states will continue to be able to count
  individual students as a part of the same major
  racial groups for AYP purposes in the same manner
  that they do currently.

10
New Race/Ethnicity Codes

• So, what does that really mean?
• Dont have to change
• But, remember the way in which the data is
  collected will be different
• Currently, the sum of the percent of students in
  each racial group 100
• Wont necessarily be the case unless add another
  group 2 or More
• May impact subgroup sizes

11
New Race/Ethnicity Codes

• Timeline for Implementation
• By the fall of 2010 in order to report data for
  the 2010-2011 school year
• Still required to compare new data to old data
  trends
• Cant wait until there are 2 years of new data

12
New Race/Ethnicity Codes

• Suggestions
• Create a plan to migrate to the new collection
  process sooner rather than later
• Run some simulations to look at potential impact
  what ifs??
• Review potential impact on accountability
  subgroups and minimum n
• Create a process to minimize potential threats to
  data quality

13
New Race/Ethnicity Codes

• Your thoughts???

14
New Graduation Rate

• Non-regulatory Guidance issued in December 2008
  by USED
• Intended to establish a uniform and more accurate
  measure of grad rate
• Four year Adjusted Cohort Graduation Rate
  however, can request to use Extended year
  Adjusted Cohort Graduation Rate

15
New Graduation Rate

• Four Year Adjusted Cohort Grad Rate
• Calculated by dividing the number of students who
  graduate in four years or less with a regular
  high school diploma by the number of students who
  form the adjusted cohort for that graduating
  class

16
New Graduation Rate

• Formula for Calculating the Four Year Adjusted
  Cohort Grad Rate for Class of 2012 (9th graders
  in 2008)
• Number of cohort members who earned a regular
  high school diploma by the end of the 2011- 2012
  school year
• __________________________________________________
  ______
• Number of first-time 9th graders in fall 2008
  (starting cohort) plus students who transfer in,
  minus students who transfer out, emigrate, or die
  during school years 2008-2009, 2009-2010,
  2010-2011, and 2011-2012

17
New Graduation Rate

• Differs from NGA rate
• NCLB rate does not allow cohort reassignment
• NGA rate allows students with disabilities and
  recent immigrant LEP students to be re-assigned
  to a different cohort to allow more time to
  complete school
• NCLB rate includes only those students who
  graduate with Regular Diploma
• NGA rate permits students who graduate with
  modified high school diplomas to count as
  graduates

18
New Graduation Rate

• Students who graduate in less than 4 years are
  counted in the cohort in which he or she started
  9th grade
• Students who graduate in the summer following 4th
  year can be included as graduating on-time
• Can produce lagging graduation rates for AYP
  purposes

19
New Graduation Rate

• Some interesting observations that will impact
  data collection processes and procedures
• Students who drop out before being enrolled in
  9th grade are not included in the cohort
• Student must be enrolled as a first time 9th
  grader to be in cohort

20
New Graduation Rate

• 2. Student who changes subgroup
• membership during the 4 years
• State must develop business rules that clearly
  define how this will be handled for graduation
  rate (state discretion but must be applied
  consistently across grade spans)

21
New Graduation Rate

• 3. Student who graduates in more than 4 years
  (including those students with disabilities with
  an IEP that provides more time to graduate)
• Counted as a non-graduate in the 4th year in
  school
• In denominator of 4 year rate and never included
  as a graduate even though he/she graduated

22
New Graduation Rate

• Can use Extended Year Rate
• State can request to use an extended-year
  adjusted cohort graduation rate for AYP
  determinations
• Can then include students who graduate in more
  than 4 years
• Subject to peer review and approval

23
New Graduation Rate

• Formula for Calculating the Extended Year (5
  Year) Adjusted Cohort Grad Rate for Class of 2012
  (9th graders in 2007)
• Numerator in the four-year adjusted cohort
  graduation rate plus the number of students from
  the cohort who earned a regular high school
  diploma by the end of the 2011-2012 school year
• __________________________________________________
  ______
• Denominator in the four-year adjusted cohort
  graduation rate plus students who transferred in
  during the 2011-2012 school year minus students
  who transferred out, emigrated, or died during
  the 2011-2012 school year

24
New Graduation Rate

• Timeline for Implementation
• Must be reported in report card that gives
  results for the 2010-2011 assessments
• Must be used for accountability in 2012
• Includes school, district and state levels
  aggregated and disaggregated

25
New Graduation Rate

• Bottom Line Now is the time!
• States will need accurate individual student
  level data ASAP
• Determine data fields necessary
• Think long term and set up a data system that can
  be used by schools and districts for data-driven
  decision making
• Collect data at the lowest granular level
  possible
• Can always aggregate up
• Run simulations to see impact

26
New Graduation Rate

• Thoughts???

27
New Workbook Submission

• Big Questions
• Will the regulations be changed with a new
  administration?
• What is likely to change?
• Not the topic for today though

28
New Workbook Submission

• Fall Meeting Subgroup Focus
• How to use data to substantiate a states
  argument for
• minimum n
• use of confidence intervals
• number of ways to make AYP

29
New Workbook Submission

• List of potential questions emerged
• What evidence and what does it show?
• What is the impact of the request?
• What can be concluded?
• How consistent are the decisions being made and
  is there variability within?
• Is there historical evidence that is useful?

30
New Workbook Submission

• Data and data related issues emerged
• What data elements are necessary?
• Are they available?
• How accurate is the data?
• How current is the data?
• How is every student accounted for?

31
New Workbook Submission

• Short Term (completed)
• Compile arguments/data states used for 1st peer
  review process (current AYP)
• Longer Term
• Produce short document with questions and some
  possible examples of what/how
• Minimum n
• Confidence Interval
• Number of ways to make AYP

32
New Workbook Submission

• Begin work on Longer Term project
• Draft by June meeting
• Talk more about this at lunch with subgroup
  meeting
• Who can help?

33
Questions Suggestions Comments ?????????
34
Thank youthank you