AZ LEARNS

1
AZ LEARNS AchievementProfiles
Arizona Department of Education September 2002
2
What is Purposeful School Accountability?
Fair and accurate measurement of school
performance designed to publicly identify
and improve schools
3
Goals

• Understand the statistical foundation of the
  achievement profiles
• Understand the growth cut points and resulting
  outcomes
• Provide the configurations of subject/grade
  values that will be used to assign school
  classifications

4
Advantages of AZ LEARNS

• Recognizes growth across studentgroups
• Includes longitudinal growth indicator
• Includes added evidence
• Evaluates according to compensatorymodel

Note None of these advantages allowable under
NCLB draft regulations
5
Visual Model of Achievement Profile
6
Elementary Model

• AIMS
• Are schools improving at teaching in key grades,
  regardless of student group?
• MAP
• Are schools achieving One Years Growth with
  students the school has had an opportunity to
  educate?

7
High School Model

• AIMS
• Graduation Rate
• Dropout Rate
• 9th Grade Indicator

8
What are the indicators ofschool performance?
Baseline
Maintaining Adequate Performance/Underperforming
Excelling/Improving

-
Under-Performing
Improving
-

Growth
9
From Concepts to Calculations
M or E
Baseline
Maintaining Line
Growth Points
10
Setting the Maintaining Line AIMS scores
1. Rank all schools by percentage
proficient 2. Count up to reach 20 of total
enrollment 3. The percentage of students
proficient in that school is the starting
point 4. Calculate a Maintaining Line by
subject and grade level
11
Maintaining Line andBaseline Group Separation
Points
Sample Maintaining Points
12
Growth Across Student Groups
AZ LEARNS provides incentive for the advancement
of students across performance levels . . .
Falls Far
Approaches
Meets
Exceeds
Below
As opposed to focusing on only one group of
students
13
Growth Points Awarded for
1. Negative change in the Falls Far Below
category and positive change in the Meets or
Exceeds Category on AIMS
2. Percentage of students making One-Years Gain
on MAP
3. Percentage of students with extended writing
trait score of 24 or higher
14
3-Year Rolling Average

• Minimizes group differences
• Increases reliability
• Identifies trends in school performance instead
  of bumps

01 02 03
00 01 02
/3
/3
15
Calculating Growth Points Example Arizona School
(Math Grade 3)
16
Calculating Growth Points Example Arizona
School (Math Grade 8)
17
Benchmarking Growth PointExpectations

• Use state performance over last three years as
  a benchmark

• Appropriate benchmark, because does not outpace
  or lag behind school performance

• Develop growth point expectations based on
  distance from state mean

18
Calculating Statewide Growth Cut Points

• Calculate growth points for every
  school/subject/grade combination in the state
• Calculate the statewide mean growth point value
• Calculate the standard deviation of the statewide
  distribution of growth point values
• Use the standard deviation as a common metric to
  establish the growth point cutoffs

19
Example Math 3rd Grade
M or E
Baseline
Maintaining Line
_X
-.5 SD
1.0 SD
-1.0 SD
.5 SD
-6.7
.8
8.3
15.8
23.3
Arizona School
Growth Points
20
Example Math 8th Grade
M or E
Baseline
Maintaining Line
_X
-.5 SD
1.0 SD
-1.0 SD
.5 SD
-13.4
-6.3
0.9
8.1
15.2
Arizona School
Growth Points
21
MAP and Extended Writing Added Evidence
M or E
Baseline
Maintaining Line
_X
-.5 SD
1.0 SD
-1.0 SD
.5 SD
Growth Points
22
Added Evidence Values for MAP
Example Math Grade 3
Standard deviation 15.02
Arizona School
23
Added Evidence Values for EWS
24
Example Math 3rd Grade
M or E
Baseline
Maintaining Line
_X
-.5 SD
1.0 SD
-1.0 SD
.5 SD
-6.7
.8
8.3
15.8
23.3
Growth Points
Arizona School 15.9 Growth Points
25
Added Evidence Values for MAP
Example Math Grade 8
Standard deviation 14.33
Arizona School
26
Example Math 8th Grade
M or E
Baseline
Maintaining Line
_X
-.5 SD
1.0 SD
-1.0 SD
.5 SD
-13.4
-6.3
0.9
8.1
15.2
Growth Points
Arizona School 8.7 Growth Points
27
Visual Model of Achievement Profile
28
Combining into Final Classification -Arizona
Elementary School
Example 3, 5, 8 (Nine Subject/Grade Values)
Achievement Profile Improving
29
Combining into Final Classification -High School
Example High School (Four Subject/Grade Values)
Achievement Profile Maintaining Adequate
Performance
30
Targets for Dropout andGraduation Rates
Dropout
Graduation
31
Points for Dropout andGraduation Rates
32
Combining Other Subject/grade Configurations

• Two Subject/Grade Values

• Three Subject/Grade Values

33
Combining Other Subject/grade Configurations

• Four Subject/Grade Values

• Five Subject/Grade Values

34
Combining Other Subject/grade Configurations

• Six Subject/Grade Values

• Seven Subject/Grade Values

35
Combining Other Subject/grade Configurations

• Eight Subject/Grade Values

• Nine Subject/Grade Values

36
Small SchoolsDefined as a Student Count of 100
or Less
PROBLEM One or very few students can have a
major impact on the school
results
SOLUTION Remove low-performing outlier students
and conduct the conventional
analysis. Award school
highest classification.
(Small School Adjustment or SSA)
INTERPRETATION Eliminates the impact of
low-performing outlier
students
Note Some schools are too small to be
classified using conventional statistics
37
Small Schools
Estimated percentage of schools needing the Small
Schools Adjustment 10
Estimated percentage of schools that are too
small to be classified 15
Note Preliminary Outcome Data DOES NOT include
Small Schools Adjustment. Statewide
results will improve.
38
Questions?
If you have questions please contactResearch
PolicyArizona Department of Education1535 West
Jefferson Street, Bin 6Phoenix, Arizona
85007Tel [phone]Fax [phone]