States Requests to Amend Their Educational Accountability PlansAn Overview

1
States Requests to Amend Their Educational
Accountability PlansAn Overview
William J. Erpenbach, WJE Consulting, Ltd Ellen
Forte Fast, edCount, LLC and Lori Cavell,
CCSSO CCSSO State Collaborative on Assessment
and Student StandardsAccountability Systems and
Reporting September 30, 2004Portland, Oregon
2
Overview

• Background
• Follow up paper on State accountability plan
  amendments and approvals.
• Information Presented Today
• Organizing categories for the amendment requests.
• Decisions of the U. S. Department of Education
  (ED).
• Conclusions including unanticipated or surprise
  approvals.
• Caveats
• Summary has not been merged with information
  about original plans does not reflect a
  comprehensive picture of what states are doing.
• Not all states requested amendments and we cannot
  infer why.
• Some states have submitted amendments two or more
  times since 2003 some two or more times this
  year alone.
• ED does not necessarily respond quickly, even to
  seemingly simple requests.
• ED letters do not always address all amendment
  requests or provide decision rationales.
• States named only if ED has provided a public,
  written response.

3
Organizing Categories

• Standards and Assessments
• Adequate Yearly Progress (AYP) Model
• Inclusion
• AYP Consequences and Reporting
• Conclusions

4
General Impressions

• Many States seized on EDs unanticipated plan
  approvals we reported in June 2003.
• Dont ask if its likely you will get an answer
  youll not like.
• Most frequent plan amendments requested
• Participation Rate flexibility.
• LEP student flexibility.
• SWDs flexibility.
• Larger minimum n for LEP student and SWDs
  subgroups.
• Confidence intervals.
• OAIsfrom absolutes to exceed or make progress
  toward and averaging over two or three years.

5
Creativity, Surprises, and Reaches

• Arkansas, Delaware, Iowa, Tennessee, and
  othersnew model for AYP for LEAs.
• North CarolinaSchool AYP based on students
  eligible or served under Title I.
• Use of CIs with Safe Harbor determinations
  limited to 75.
• Excluding opted out students in Participation
  Rate determinationsjury still out.
• Washington, Arkansas, and othersuse of a
  progressive minimum n (results in higher
  minimum n for districts in some cases).
• Not rolling up data to meet minimum n
  requirements for AYP determinations.
• Including only students enrolled FAY and tested
  in denominator for calculating Percent
  Proficient.
• Several Statesadded rounding up rules for AYP
  calculations.
• Longer than calendar year to define AYP for some
  SWDs.

6
Consistent Nos

• Same cell two consecutive years.
• Retroactive applications.
• Out-of-level testing other than within 1 cap.
• Blanket exemptions for unexpected medical
  emergencies.
• Apportioning subgroup membership.

7
Requests Awaiting Public Response

• Use of results for a sub-set of grades until
  2005-06 in states already administering
  assessments in grades 3 8.
• Use of results of re-tests after official
  administrations in AYP determinations.
• Inclusion of students in only one program
  subgroup (SWDs, LEP, Economically Disadvantaged).
• Use of growth models in place of Percent
  Proficient or Safe Harbor.
• Exclusion from Participation Rate calculations
  for students opted out by parents in States where
  this right is protected by law.

8
Standards and Assessments

• Changes to a Test or Grade Level Used for AYP
• Banking of Test Results
• Use of Out-of-Level Assessments or Other
  Alternate Assessments
• Adjustments of Scores Designating Proficiency,
  i.e., Cut Scores

9
Standards and Assessments(Changes to a Test or
Grade Level Used for AYP)

• Back off from use of existing 3 through 8 test
  results to only 3 5 and 6 9 grade spans until
  2005-06 (plus 10 12 grade span).
• Back away from implementation of new instruments
  or policies prior to 2005-06.

10
Standards and Assessments(Banking of Test
Results)

• Problematic during original reviewsnow,
  generally okay to bank results from
  administrations prior to the official one.
• Oregon now allowed to use results that are
  accumulated across a school year for each
  student.
• Allow districts to choose administration date.
• Allow use of Performance (not Participation)
  results from re-testing that occurs after the
  official administration.
• Use of scores from all grade levels at which
  students can take a test (Percent Proficient
  based on number tested).

11
Standards and Assessments(Use of Out-of-Level
Tests or Other Alternate Assessments)

• Several States have requested approval to use
  out-of-level assessmentsnow effectively
  defined by ED as a test based on alternate
  standards for severely cognitively disabled
  students subject to the 1 cap.
• Alabama, Indiana, and Nebraska approved to use
  results of alternate assessments for severely
  cognitively disabled students in AYP
  calculations.
• West Virginia approved to develop an equivalent
  form of its State assessment for students with
  cognitive impairments that would not be subject
  to the 1 cap.
• Three other States have asked to continue using
  out-of-level assessments for AYP without regard
  to the 1 cap until they undergo the next round
  of Peer Reviews on standards and assessments.
  They argue that more time is needed to develop
  alternate assessments for SWDs.

12
Standards and Assessments(Adjustment of Scores
Designating Proficiency, i.e., Cut Scores)

• No State has yet requested a general change or
  notified ED of a comprehensive revisiting of
  achievement standards.
• One change in cut score necessary due to an error
  during the original process.
• Two States set student academic achievement
  standards for new tests and requested amendments
  to reflect these new standards.

13
Adequate Yearly Progress (AYP) Model

• AYP Application
• Rules for Targeted Assistance Schools
• Triggers for Identification
• Rules for Identifying Districts for Improvement
• Retroactive Application
• AYP Indicators
• Percent Proficient
• Participation Rate
• Other Academic Indicators
• Annual Measurable Objectives, Intermediate Goals,
  and Trajectories
• Strategies Intended to Enhance Reliability
• Changes to Minimum n
• Consideration of ErrorConfidence Intervals and
  SEM
• Safe Harbor

14
Adequate Yearly Progress (AYP) Model (AYP
Application Rules for Targeted Assistance
Schools)

• North Carolina will permit districts to calculate
  AYP for schools that operate Targeted Assistance
  Programs based on only those students served or
  eligible to be served through Title I.
• Another State has a similar request pending.

15
Adequate Yearly Progress (AYP) Model (AYP
Application Triggers for Identification)

• Changes to conditions for overall ID
• Arkansas will compare current year to average of
  most recent three years and use most favorable in
  AYP decisions and Washington will average data
  across grades and over two years.
• West Virginia, Oregon, Pennsylvania, and others
  denied use same subject, same group for two
  consecutive years (ED provided extensive
  rationale statement to West Virginia)
• North Carolina and Illinois approved to limit
  identification for improvement to same subject
  AYP misses while Oregon approved for same OAI
  (both policy shifts by ED)
• LEAsby grade span and subject area okay
• Student groups
• A few requests for specific combinations of group
  performance

16
Adequate Yearly Progress (AYP) Model (AYP
Application Rules for Identifying Districts for
Improvement)

• At least eight States approved to modify the
  method by which school districts are identified
  for improvement.
• States include Arkansas, Delaware, Iowa, Maine,
  North Carolina, Ohio, Tennessee, and West
  Virginia.
• Districts permitted to determine AYP by grade
  spans such as elementary, middle, and high
  school.
• AYP determined on basis of grade span performance
  in academic subject areas only.
• Districts identified for improvement only if all
  grade spans miss AYP target in same subject area
  two consecutive years.

17
Adequate Yearly Progress (AYP) Model (AYP
Application Retroactive Application)

• ED consistently denied such requests from States
  (department included its rationale in letter to
  North Carolina).
• Kentucky made AYP determinations (retroactively)
  based on amendments submitted to ED but not yet
  approved. ED accepted but warned State about
  future withholding of Federal funds if repeated.
• Unclear whether some States may continue to push
  EDs policy decision on this subject.

18
Adequate Yearly Progress (AYP) Model (AYP
Indicators Percent Proficient and Participation
Rate)

• Percent Proficientmostly FAY
• Oregon and Kentucky will adopt new FAY
  definitions and Iowa received approval for
  defining FAY for some SWDs as being more than one
  year.
• Wyoming received approval to use up to two years
  of data when making AYP determinations.
• Another State asked to use growth indices in lieu
  of Percent Proficient.
• Some States will use only students enrolled FAY
  and tested in determining Percent Proficient
  (approved for Maryland and Georgia in 2003
  reviews).
• Participation Ratemostly new flexibility
• West Virginia not allowed to provide blanket
  exemptions for medical emergencies but okay on
  case-by-case basis.
• Other States would like to grant/ignore PR if
  fewer than 5 students not tested.
• Several States will increase minimum n for PR
  and apply statistical tests to calculations.
• One State seeking to include opted-out students
  if PR at least 90 but gt95, count these students
  as not proficient if their inclusion would reach
  at least 95 and recalculate PP.

19
Adequate Yearly Progress (AYP) Model (AYP
Indicators Other Academic Indicators)

• Most common change involved shifting from meeting
  a target to making progress toward a target.
• Elementary and Middle School
• Kentucky allowed to lag OAI one year.
• Others will change OAI to another indicator and
  some lowered target.
• Graduation Rate
• Mainecount only one dropout eventstudent is
  unit, not event.
• Othersapply 95 CI, use probability model
  (undefined), include GED recipients.
• Kansas, Kentucky, Tennessee, and others allowed
  to count as graduates SWDs who take more than 4
  years consistent with IEP.
• Changes to target or to use of combination of
  status and progress targets.
• Only used for schools with graduation as mission.

20
Adequate Yearly Progress (AYP) Model(Annual
Measurable Objectives, Intermediate Goals, and
Trajectories)

• AMOs and Trajectories
• Few requests for changes in these areas.
• One State would like to round its Percent
  Proficient and AMOs to whole numbers.
• Arkansas approved to use a confidence interval to
  make AYP determinations involving AMOs and IGs.
• Kentucky will set these based on grade span
  configurations to cover differing LEA
  configurations.

21
Adequate Yearly Progress (AYP) Model(Strategies
Intended to Enhance Reliability Changes to
Minimum n)

• Arkansas, Kentucky, North Carolina, and others
  increased their minimum ns.
• Several will combine across years while some have
  decided the opposite.
• Allow choice for small schools/groups.
• Many approvals to use larger minimum n for SWDs
  and/or LEP students including Kansas, Missouri,
  New Jersey, and South Carolina.
• One request to maintain low minimum n but only
  analyze if the group gt 15 of student population.
• Several asked to increase minimum n for LEAs
  (denied) but those requesting use of a
  progressive minimum n approved (including
  Arkansas and Washington) which tends to have the
  same result.

22
Adequate Yearly Progress (AYP) Model(Strategies
Intended to Enhance Reliability Consideration of
Error)

• Alabama and two others99 for Percent
  Proficient.
• North Carolina, Pennsylvania other others95.
• Six States requested use of CI for Safe
  HarborApproved but limited to 75 without stated
  rational.
• South Carolina received approval to use a SEM in
  calculating Percent Proficient.

23
Adequate Yearly Progress (AYP) Model(Safe
Harbor)

• Use of 75 confidence interval in safe harbor
  reviews approved for Alaska, Kansas, Maine,
  Pennsylvania, and othersmost had requested at
  least 95.
• Tennessee allowed to use value-added model in
  safe harbor.
• Kansas permitted to use current years results
  compared with previous years to make safe
  harbor reviews for small schools even though
  minimum n not met in either case.

24
Inclusion

• Students with Disabilities
• Students with Limited English Proficiency

25
Inclusion

• Generally
• Many States approved to apply new flexibility.
• Many will use higher minimum n for these
  subgroups.
• Broader use of laws to keep students no longer
  directly served in the subgroups for AYP
  determinations.
• Students with Disabilities
• Some also asked to allow some kind of waiver to
  1 capKansas approved for a reallocation
  process.
• ED denied Tennessees request to include scores
  of gifted students in SWDs category consistent
  with State law.
• Count as graduates students taking longer than
  four years in high school consistent with IEPs.
• Students with Limited English Proficiency
• Alaska will not be allowed exclude results for
  Heritage language students from AYP until grade
  6.

26
AYP Consequences and Reporting

• Timing and Degree of Consequences
• Rewards and Recognition
• Change to Program or Demographic Variables
• Reporting Timeline

27
AYP Consequences and Reporting

• Timing and Degree of Consequences
• ED denied West Virginias request to switch order
  of school choice and provision of supplemental
  educational services but indicated that the State
  could offer both in the first year of
  identification for improvement.
• Another State has asked to lag its accountability
  cycle by one year and another has requested
  approval to tailor the degree of sanction to a
  schools level of need.
• Rewards and Recognition
• Alaska, Delaware, and North Carolina will add new
  levels of recognition.
• New Jersey will set priorities for instructional
  intervention efforts based on extent to which AYP
  targets were missed.

28
AYP Consequences and Reporting

• Changes to Program or Demographic Variables
• Illinois and Maine will add a multi-ethnic
  category to their AYP and reporting subgroups.
• Maine will eliminate its Caucasian group since it
  does not differ significantly from the all
  students group.
• Another State that does not have a statewide
  student-level database or student-level variable
  for determining income wants to designate every
  student who attends a school offering Title I
  services as being in a low socio-economic status.
• Reporting Timeline
• California and South Carolina will combine
  previously separate State and Federal report
  cards.
• Texas will align its State and Federal reporting
  timelines.

29
States Requests to Amend their Educational
Accountability PlansAn Overview
finis