UNDERSTANDING ADEQUATE YEARLY PROGRESS 2006

1
UNDERSTANDING ADEQUATE YEARLY PROGRESS 2006

• Pennsylvania Department of Education
• Bureau of Assessment and Accountability
• May 2006

2
No Child Left Behind Act of 2001

• Federal Act that requires all students to be
  Proficient in reading/language arts and
  mathematics by 2014

3
No Child Left Behind Act of 2001

• Federal mandate that directs states to implement
  within the parameters of the act.
• Each state required to produce an Accountability
  Workbook
• Respond to 10 principles
• Each year states may petition USDE to amend their
  accountability workbook

4
Pennsylvania Accountability Workbook

• 2004
• Confidence Interval
• Graduation and Attendance Thresholds
• 2005
• PPI
• Grade Spans at District Level

5
USDE Flexibility

• 2004
• 3 Year Average for Participation Rate
• LEP Students in 1st Year of Enrollment in US
  School
• 2005
• Proxy
• 2 Year LEP Exited

6
USDE Flexibility

• 2006
• 2 Year IEP Exited
• Proxy
• Students Displaced by Katrina (?)

7
PennsylvaniaConsolidated State
ApplicationAccountability Workbook  May 1,
2003Revised May 30, 2003Revised May 8,
2004Revised June 15, 2005(Revised sections
1.1, 1.2, 1.6, 3.2, 3.2c, 4.1, 5.4, 5.5, 5.6,
7.1, 7.2, 9.1, 9.2, and 10.1) for State Grants
under Title IX, Part C, Section 9302 of the
Elementary and Secondary Education Act (Public
Law 107-110)
8
TARGETS

• Performance on the state assessment
• Participation in the state assessment
• Other academic indicator
• Attendance
• Graduation

9
OTHER ACADEMIC INDICATOR

• District
• Attendance and graduation
• Therefore, districts have four targets
• School
• Attendance or graduation
• Therefore, schools have three targets

10
DISTRICTS OTHER ACADEMIC INDICATORS

• Attendance
• Based on the attendance rate of all the schools
  in the district that dont have a graduating
  class
• Graduation
• Based on the graduation rate of all the schools
  in the district that have a graduating class

11
DISTRICTS OTHER ACADEMIC INDICATORS - Graduation

• 2004-05 Graduation rate is used for 2005-06 AYP
• Threshold to make the Graduation target is 80 OR
  growth from 2003-04.

12
Graduation Rate

• Number of students who graduated in 2005 ?
• Number of students who graduated in 2005
• Number of seniors who dropped out in 2004-05
• Number of juniors who dropped out in 2003-04
• Number of sophomores who dropped out in 02-03
• Number of freshmen who dropped out in 2001-02

13
Graduation Rate

• students who graduated
• in 2005 ?
• students who graduated
• in 2005
• senior d.o. in 04-05
• junior d.o. in 03-04
• soph. d.o. in 02-03
• frosh d.o. in 01-02

• 145 graduates ?
• 145 graduates
• 3 drop outs
• 4 drop outs
• 5 drop outs
• 2 drop outs
• 159 145?15991

14
DISTRICTS OTHER ACADEMIC INDICATORS - Attendance

• 2004-05 Attendance rate is used for 2005-06 AYP
• Threshold to make Attendance rate is 90 OR
  growth from 2003-04
• BUT - - -

15
DISTRICTS OTHER ACADEMIC INDICATORS - Attendance

• If you didnt have 90 attendance rate in
  2004-05, AND
• You dont show growth from 2003-04, AND
• Your attendance rate for 2005-06 is either 90 OR
  shows growth from 2004-05,
• You may enter your 2005-06 attendance data
• May 25 to June 9

16
SCHOOLS OTHER ACADEMIC INDICATORS - Attendance

• Attendance
• Used if the school does not have a graduating
  class
• Ex. K-3, K-5, K-8, 6-8,
• Based on the attendance rate of all the grades in
  a school, not just the tested grades
• 2004-05 attendance rate used for 2005-06 AYP

17
SCHOOLS OTHER ACADEMIC INDICATORS - Attendance

• If you didnt have 90 attendance rate in
  2004-05, AND
• You dont show growth from 2003-04, AND
• Your attendance rate for 2005-06 is either 90 OR
  shows growth from 2004-05,
• You may enter your 2005-06 attendance data
• May 25 to June 9

18
SCHOOLS OTHER ACADEMIC INDICATORS - Attendance

• If youre a new school in 2005-06 and dont have
• attendance data for 2004-05, use the May 25 June
• 9 window to
• cross-walk 04-05 data from the old school to the
  new school
• OR
• submit your 2005-06 attendance data
• IF YOU HAVE NO ATTENDANCE DATA ON
• THE AYP DATA FILE, YOU WILL BE
• IDENTIFIED AS NOT MEETING AYP!

19
Attendance

• Average Daily Attendance (ADA) ?
• Average Daily Membership (ADM)

20
Attendance
21
SCHOOLS OTHER ACADEMIC INDICATORS - Graduation

• 2004-05 Graduation rate is used for 2005-06 AYP
• Threshold to make the Graduation target is 80 OR
  growth from 2003-04.

22
OTHER ACADEMIC INDICATOR SUMMARY

• District
• Attendance
• 90 or growth
• AND
• Graduation
• 80 or growth
• 2004-05 data
• Rates determined by totals in district

• School
• Attendance
• 90 or growth
• OR
• Graduation
• 80 or growth
• 2004-05 data
• Rates determined by totals in school

23
OTHER ACADEMIC INDICATOR SUMMARY

• Attendance
• Can enter 2005-06 data if its needed to meet the
  AYP threshold

• Graduation
• Can not enter 2005-06 data

24
Other Academic IndicatorsAttendance Rate and
Graduation Rate

• QUESTIONS?

25
AYP Targets

• PARTICIPATION
• in the state assessment
• Grades 3, 5, 8, and 11
• Math and reading
• All students enrolled as of the last day of
  testing window

• PERFORMANCE
• on the state assessment
• Grades 3, 5, 8, and 11
• Math and reading
• Only students enrolled for a full academic year

26
PARTICIPATION

• 95 Threshold
• Reported for both math and reading
• Composite of all students is always reported
• Subsets of the composite (subgroups) reported
  only when there are at least 40 students
• Nine subgroups

27
Subgroups

• American Indian or Alaskan Native
• Asian or Pacific Islander
• Black/African American non-Hispanic
• Latino/Hispanic
• White non-Hispanic
• Multi-Racial/Ethnic
• IEP
• LEP
• Economically Disadvantaged

28
PARTICIPATION

• Math
• All students
• American Indian or Alaskan Native
• Asian or Pacific Islander
• Black/African American non-Hispanic
• Latino/Hispanic
• White non-Hispanic
• Multi-Racial/Ethnic
• IEP
• LEP
• Economically Disadvantaged

• Reading
• All students
• American Indian or Alaskan Native
• Asian or Pacific Islander
• Black/African American non-Hispanic
• Latino/Hispanic
• White non-Hispanic
• Multi-Racial/Ethnic
• IEP
• LEP
• Economically Disadvantaged

29
PARTICIPATION

• Rate calculation
• Number of students who received a
  score/performance level ? Number of students
  enrolled as of the last day of the testing window
• Example 114 students received a
  score/performance level ? 120 students enrolled
• 114 ? 120 95

30
PARTICIPATION

• Three ways to calculate
• Current year
• Average of current year previous year
• Average of current year previous two years (3
  year average)
• Report the highest of the three!

31
PARTICIPATION

• 2006
• 120 enrolled
• 110 scored
• 2005
• 118 enrolled
• 114 scored
• 2004
• 122 enrolled
• 117 scored

• 2006
• 110120 91.6
• 2006 2005
• 224238 94.1
• 2006 2005 2004
• 341 360 94.7
• 94.7 rounds up to 95

32
PARTICIPATION

• IEP
• Current year
• Current year 2 year exited current year
• Average of current year previous year
• Average of current year 2 year exited current
  year previous year
• Average of current year previous 2 years
• Average of current year 2 year exited current
  year previous 2 years

33
PARTICIPATION

• LEP
• Current year
• Current year 2 year exited current year
• Average of current year previous year
• Average of current year 2 year exited current
  year previous year
• Average of current year 2 year exited current
  year previous year 2 year exited previous
  year
• Average of current year previous year 2 year
  exited previous year
• Average of current year previous 2 years
• Average of current year 2 year exited current
  year previous 2 years
• Average of current year 2 year exited current
  year previous year 2 year exited previous
  year 3 years ago
• Average of current year previous year 2 year
  exited previous year 3 years ago

34
PARTICIPATION

• How is it possible to have less than 100
  participation?
• Student had an extended absence for the entire
  testing window
• Student had a parental request for exclusion from
  the assessment
• Other
• Student was absent without make-up for one or
  more sections of math or reading
• Failure to meet the attempt logic

35
PARTICIPATION

• These do not affect participation rate
• Medical emergency
• PASA
• LEP in 1st year of enrollment
• Court/Agency placed (no IEP)
• Katrina (?)

36
Participation Rate

• QUESTIONS?

37
AYP Targets

• PARTICIPATION
• in the state assessment
• Grades 3, 5, 8, and 11
• Math and reading
• All students enrolled as of the last day of
  testing

• PERFORMANCE
• on the state assessment
• Grades 3, 5, 8, and 11
• Math and reading
• Only students enrolled for a full academic year

38
PERFORMANCE

• Math
• 45 threshold
• Composite of all students is always reported
• Subsets of the composite (subgroups) reported
  only when there are at least 40 students
• Nine subgroups

• Reading
• 54 threshold
• Composite of all students is always reported
• Subsets of the composite (subgroups) reported
  only when there are at least 40 students
• Nine subgroups

39
PERFORMANCE

• Calculations for
• (Only performed for FAY students who received a
  score)
• All students
• Racial/Ethnic groups
• Economically disadvantaged

40
PERFORMANCE

• Current year
• Current year confidence interval
• Average of current year previous year
• Average of current year previous year
  confidence interval
• Safe Harbor
• Safe Harbor confidence interval
• Pennsylvania Performance Index (PPI)

41
PERFORMANCE

• Calculations for
• (Only performed for FAY students who received a
  score)
• LEP
• IEP

42
PERFORMANCE

• Current year
• Current year two year exited
• Current year confidence interval
• Current year two year exited confidence
  interval
• Average of current year previous year
• Average of current year previous year
  confidence interval
• Average of current year two year exited
  previous year
• Average of current year two year exited
  previous year confidence interval
• Safe Harbor
• Safe Harbor confidence interval
• Pennsylvania Performance Index (PPI)

43
Confidence Interval

• z p - ? ? ?( ?(1-?)/n)
• p the proportion Proficient in the
  district/school/subgroup
• ? the statewide target for Proficient
• n number of students

44
CONFIDENCE INTERVAL

• Z p - ?
• ? (? (1-?) n)

• p (Proportion Proficient in the
  district/school/subgroup)
• p .40
• ? (Statewide Target for Proficient)
• ? .45
• n (number of students)
• n 42

45
CONFIDENCE INTERVAL

• The value of z is then compared with the critical
  value of z to determine if the observed
  difference is statistically significant. Testing
  this difference at the .05 probability, the value
  of z is compared with zcrit of -1.645 and if the
  observed z is greater than or equal to -1.645, we
  can conclude that the observed proportion has
  achieved the AYP threshold.

46
SAFE HARBOR

• Reduce the percentage of students below
  Proficient
• in 2004-05 by at least 10 in 2005-06.
• 2005
• 20 Proficient
• 80 below Proficient
• Reduce 80 by 10
• 8
• 2006
• 29 Proficient
• 71 below Proficient
• Reduced percentage by 9 (8 needed)

47
Safe HarborConfidence Interval

• For safe harbor we use the standard error of the
• difference in proportions. The controlled
• error rate is .75. The approach for applying
• confidence intervals to safe harbor is as
  follows.
• P1 is the first observation  P2 is the target
  given
• the first observation (P2 .9P1 10), i.e.,
  the
• reduction of the non-proficient students by
• 10.  The formula for calculating the standard
• error then is SESQRT (P1(100-P1)/n
• P2(100-P2)/n).

48
Safe HarborConfidence Interval

• For example, school X enrolled 550 students in
  Year 1 and had
• 20 of its students performing in Reading at
  Proficient and
• Advanced and in Year 2 enrolled 580 students and
  had 23 of its
• Students performing at Proficient and Advanced in
  Reading. To
• calculate the standard error of this difference,
  we take the square
• root of ((2080)/550) ((28 72)/580). This
  equals the square root
• of (2.91 3.05), or 2.44. We then must adjust
  the error rate by the
• Appropriate z-score or .68. We take 2.44 and
  multiply it by .68
• (1.66). We add 1.66 to 23 (24.66) and compare
  this value to the
• target (P228). Since 24.66 is less than the
  target (28), we can say
• with confidence that the AYP relative growth
  criterion has not
• been met by school X.

49
PENNSYLVANIA PERFORMANCE INDEX

• Advanced
• Proficient
• Basic
• High Basic
• Low Basic
• Below Basic
• High below Basic
• Low Below Basic

50
PPI

• Advanced X 1
• Proficient X 1
• High Basic X .8
• Low Basic X .6
• High Below Basic X .4
• Low Below Basic X .2
• Untested X 0

51
PPI

• 10 Advanced
• 20 Proficient
• 20 High Basic
• 20 Low Basic
• 15 High Below Basic
• 10 Low Below Basic
• 5 Untested

• 10 X 1 10
• 20 X 1 20
• 20 X .8 16
• 20 X .6 12
• 15 X .4 6
• 10 X .2 2
• X 0 0
• 66

52
PPI

• Goal is to reach 100 by
• 2014
• 100 66 34
• 12 years
• 34 12 2.83
• 2003 target is
• 66 2.83 68.83

53
PPI
54
PPI
55
School vs. District

• Participation and Performance

56
Participation and Performance

• District
• Grade spans
• 3-5
• 6-8
• 9-12

• School
• Based on tested grades in the school
• K 8
• 3, 5, 8
• K 5
• 3 5
• 9 12
• 11

57
District
58
Participationand Performance

• Questions?

59
Attribution

• Where do those results go?

60
Attribution
61
Alternate Assessment

• Grade 3 8 or 11
• Significant cognitive disabilities
• Requires intensive instruction to learn
• Requires adaptation and support
• Requires substantial modifications
• Participation in the general education curriculum
  differs greatly

62
Alternate Assessment

• No more than 1 of your Advanced and Proficient
  scores can come from the alternate assessment.

63
Alternate Assessment

• PSSA
• 90 students
• Enrolled full academic year
• 40 Advanced and Proficient

• PASA
• 10 students
• Enrolled full academic year
• 10 Advanced and Proficient

64
Alternate Assessment

• 90 PSSA
• 10 PASA
• 40 Advanced and Proficient on PSSA
• 10 Advanced and Proficient on PASA

• 90 10 100
• 1 of 100 1
• 40 Advanced and Proficient on PSSA 1 Advanced
  or Proficient on PASA 41
• 41/100 41

65
Attribution
66
Attribution

• Attribution Map can be found on PDE Web site
• www.pde.state.pa.us

67
CORRECT YOUR DATA

• May 24 June 7
• DRC Corrections System Phase 1
• www.drc-web.com/onlinecorrections
• Grades 4 8 and 11
• No 3rd grade students
• No PASA students
• Concentrate on Grades 5, 8, 11 for AYP
• All fields can be corrected

68
The PSSA Data Correction Window for CTEPhase I
May 24-June 9

• Student coding can be corrected at this time.
  Please be sure to pay attention
• to these items
•  CTE
• All Career tech students with occupational
  declaration forms on file should be coded as
  Career Tech or CTE
•  
• Tech Prep
• If students are coded Tech Prep, be sure that
• there is an articulation agreement with a post
  secondary institution for those students
• the students are ALSO coded as career tech
•  
• Cluster
• The career tech teachers can provide a list of
  areas that belong to each cluster

69
DRC Corrections System

• QUESTIONS?

70
July 17th What Happens?

• Preliminary AYP Data File
• www.drc-web.com/reportdelivery
• DRC Corrections System Phase II
• (July 17 21)
• www.drc-web.com/onlinecorrections
• AYP Appeals (July 17 28)
• www.pde.state.pa.us

71
July 2006
www.drc-web.com/reportdelivery
72
July 2006
www.drc-web.com/onlinecorrections
DRC Corrections System Phase II
73
July 2006
www.pde.state.pa.us
2006
AYP Appeals Window
for Schools and Districts
74
2006 AYP Appeals Procedure

• Cover Page (required with package)
• Routing Page (required with package)
• Four Appeal Forms
• District Appeal Form
• School Appeal Form
• Charter School Appeal Form
• Full-time CTC/AVTS Appeal Form

75
2006 AYP Appeals Procedure

• If filing an appeal for more than one school in a
  district, you must use a separate form for each
  school.

76
2006 AYP Appeals Procedure

• Submit appeals packet to
• Dana Klouser
• Administrative Officer
• Pennsylvania Department of Education
• Bureau of Assessment Accountability
• 333 Market Street, 9th Floor
• Harrisburg, PA 17126-0333
• Postmark by July 28, 2006
• (Appeals postmarked after July 28, 2006 cannot
  be accepted!)

77
2006 AYP Appeals Procedure

• In an effort to protect each LEAs right to
  due process, the Department will review each
  issue that an LEA raises in an AYP appeal and
  determine how to resolve the issue.

78
2006 AYP Appeals Procedure

• The Bureau of Assessment Accountability
• will determine if the issues in the appeal are
  within its area of expertise and authority to
  remedy, if it is determined to be a valid appeal.
• OR
• will determine that the issues raised in the
  appeal are outside its expertise or authority to
  remedy.

79
2006 AYP Appeals Procedure

• Appeals that raise issues that are outside the
  expertise or authority of the Bureau of
  Assessment and Accountability will be
• Transferred to the Division of Federal Programs
  for Complaint Resolution

80
2006 AYP Appeals Procedure

• Appeals that are within the expertise and
  authority of the Bureau of Assessment and
  Accountability will be
• Reviewed
• Determinations Made by August 11, 2006

81
AYP Appeals

• QUESTIONS?

82
Understanding AYP 2006

• THANK
• YOU!