Purpose of the study

1
(No Transcript)
2
Purpose of the study
This study examines the NCLB accountability
systems for 28 states. We took 36 real schools
from around the nation (18 elementary, 18 middle)
and asked - How many would make AYP in each
state? - What explains the variation?
3
AYP provisions examined

• Proficiency cut scores in math and reading for
  2006
• Annual performance targets (AMOs) for 2008
• Minimum subgroup sizes (n sizes) for 2008
• Margins of error (confidence intervals) for
  2008

4
Limitations of the study

• We were not able to consider
• - NCLBs safe harbor provision
• - Growth models
• - Nuances related to the exclusion of special
  education students and English Language Learners

5
Finding 1 Huge variation in AYP ratings for our
elementary schools

• Almost all our sampled schools failed to make AYP
  in some states, and nearly all of these same
  schools made AYP in others.
• In Massachusetts, only 1 of 18 elementary schools
  made AYP.
• In Wisconsin, 17 of these same 18 schools made
  AYP.
• Same kids, same academic performance, same
  schoolsand very different results.

6
Number of sampled elementary schools that made
AYP in 2008, by state
7
Finding 2 There was less variation for our
middle schools because most did poorly everywhere

• In 21 states, two or fewer middle schools made
  AYP.
• In no state did even half of the 18 middle
  schools make AYP.
• This is mostly because of the larger size of
  middle schools which means they are held
  accountable for the performance of more
  subgroups.

8
Finding 3 Cut scores are important, but so
are annual targets

• Some states (like California) have high cut
  scores but still permit nearly all schools,
  including poor performers, to make AYP because of
  low annual targets.
• But other states that have been criticized for
  their low NCLB cut scores (e.g., Colorado), have
  tough annual targets that seem reasonable
  relative to their tests.

9
Overall proficiency rates of the elementary
school sample in math
10
Math proficiency rates of the elementary school
sample relative to each states 2008 targets
11
Finding 4 State determinations around subgroup
size are critical

• Generally
• the lower the states subgroup size (n size),
• the more subgroups for which the typical school
  is accountable,
• and the more separate targets that school must
  hit.

12
Case in point High-performing Chaucer Middle
School
13
Elementary subgroup performance of sample schools
under the 2008 Ohio AYP rules
14
When the special ed or ELL subgroups count, the
school almost always fails
15
Implications

• For an accountability system to be effective,
  educators must believe that it is fair,
  consistent, and understandable.
• Unfortunately, the way NCLB rates schools appears
  to be idiosyncraticeven randomand opaque.
• The success or failure of a given school under
  NCLB is driven as much by the way the law is
  implemented by its home state as its student
  performance.

16
This is the Accountability Illusion.
17
(No Transcript)