PVAAS

1
PVAAS Pennsylvania Value-Added Assessment
Systemhttp//www.portal.state.pa.us/portal/serve
r.pt/community/pa_value-added_assessment_system_(p
vaas)/8751
2
The TWO PVAAS Methodologies
Looking Forward/Planning PVAAS Projection
Reports For Individual Students and Cohorts of
Students
Looking Back/Evaluation Value-added Growth
Reports For Cohorts of Students
Today
NO individual student growth measure
3
PVAAS IS

• A statistical analysis of existing assessment
  data (PSSA).
• A realistic measurement of a schools
  progress/growth over time.
• Progress data to add to achievement data as
  schools are analyzing their work with students
  and making improvement plans.

4
PVAAS IS NOT

• Another test.
• A method for providing a measure of growth for an
  individual student.

5
Current Data Collection Approaches
High Achievement
Average Achievement
Low Achievement
Achievement is based on each students ending
point.
6
PVAAS Adds Another Variable To AchievementProgres
s
Below Growth Standard
At or Above Growth Standard
Progress is based on each students starting
point.
7
Viewing Achievement and Progress Together

• Scatter Plots

8
Scatter Plot Quadrants and Their Meanings
Excelling
Achieving but Slipping
Underperforming but Improving
Underperforming and Slipping
9
Please review this document in your folder
10
Example Regional Scatter Plot
11
Scatter Plot Comparison

• State vs. IU 12 vs. YCSD
• Math and Reading
• Grades 4 8

12
Statewide to IU Region to DistrictGrade 4 Math
Statewide Grade 4 Math IU 12 Grade 4 Math York City SD Grade 4 Math

13
Statewide to IU Region to DistrictGrade 4 Reading
Statewide Grade 4 Reading IU 12 Grade 4 Reading York City SD Grade 4 Reading

14
Statewide to IU Region to DistrictGrade 5 Math
Statewide Grade 5 Math IU 12 Grade 5 Math York City SD Grade 5 Math

15
Statewide to IU Region to DistrictGrade 5 Reading
Statewide Grade 5 Reading IU 12 Grade 5 Reading York City SD Grade 5 Reading

16
Statewide to IU Region to DistrictGrade 6 Math
Statewide Grade 6 Math IU 12 Grade 6 Math York City SD Grade 6 Math

17
Statewide to IU Region to DistrictGrade 6 Reading
Statewide Grade 6 Reading IU 12 Grade 6 Reading York City SD Grade 6 Reading

18
Statewide to IU Region to DistrictGrade 7 Math
Statewide Grade 7 Math IU 12 Grade 7 Math York City SD Grade 7 Math

19
Statewide to IU Region to DistrictGrade 7 Reading
Statewide Grade 7 Reading IU 12 Grade 7 Reading York City SD Grade 7 Reading

20
Statewide to IU Region to DistrictGrade 8 Math
Statewide Grade 8 Math IU 12 Grade 8 Math York City SD Grade 8 Math

21
Statewide to IU Region to DistrictGrade 8 Reading
Statewide Grade 8 Reading IU 12 Grade 8 Reading York City SD Grade 8 Reading

22
11th Grade Scatterplots

• The PSSA-PVAAS Scatter Plots visually display the
  following data from the statewide distribution of
  PSSA results for 2009-10
• Achievement
• The mean predicted performance of the cohort.
• The actual mean performance of all students in
  the state is displayed for reference
• Growth
• School Effect Observed Mean Predicted Mean
• Standardized School Effect School Effect
  relative to its error displays degree of
  confidence
• The following two slides explain the meaning of
  each of the regions of an 11th grade scatter
  plot.

23
The Scatter Plot Coordinate System for Grade 11
Vertical Axis Mean Predicted Scaled Score
Growth Descriptor Boundaries (2SEs)
Horizontal Axis Growth Measure in Standard Errors
2010 Mean Scaled Score
24
Scatter Plot Regions and Their Meanings for
Grade 11
Higher Performing Met Expectations
Higher Performing Below Expectations
Higher Performing Above Expectations
Lower Performing Above Expectations
Lower Performing Below Expectations
Lower Performing Met Expectations
25
Statewide to IU Region to DistrictGrade 11 Math
Statewide Grade 11 Math IU 12 Grade 11 Math York City SD Grade 11 Math

26
Statewide to IU Region to DistrictGrade 11
Reading
Statewide Grade 11 Reading IU 12 Grade 11 Reading York City SD Grade 11 Reading

27
Overview of PVAAS Reporting
28
PVAAS Reporting

• District Level Reporting
• School Level Reporting
• Grade Level Reporting
• Reading (Grades 4-8, 11)
• Math (Grades 4-8, 11)
• Science (Grades 4, 8, 11)
• Writing (Grades 5, 8, 11)
• Subgroups Reporting (ex. IEP, ED, ELL)
• Performance Levels (Advanced, Proficient, Basic,
  Below Basic)
• Student Level Reporting
• Reading, Math, Science, Writing (grades vary
  based on content area)

29
Have all of our groups of students experienced
one year of academic growth during the last
school year?
Value-added Growth Reports
30
PVAAS Value-added CohortGrowth Descriptors
Grade 4 8 Math and Reading
Favorable Indicator Estimated gain at or above growth standard. Students in this cohort have made at least one year growth. All schools can achieve this rating.
Caution Indicator Estimated gain below growth standard but by less than one standard error. Students in this cohort have grown less than the standard.
Stronger Caution Estimated gain below growth standard by more than one but less than two standard errors. Students in this cohort have fallen behind their peers.
Strongest Warning Estimated gain well below growth standard by more than two standard errors. Students in this cohort have made little progress.
31
District/School Report Example
32
PVAAS Predictive School Report
33
How are our best students, our average students,
our struggling students growing?

• Performance Diagnostic Report

34
Growth Standard MethodologyWhat the whiskers
tell us
Exceeded Growth Standard More than One Years
Growth Green
Met the Growth Standard Made One Years Growth
Yellow
Growth Standard (One Years Growth)
Below Growth Standard Less than One Years
Growth Rose
35
Predictive MethodologyWhat the whiskers tell us
ABOVE Expectations -- Observed Score Was Greater
Than Predicted Score
NDD No Detectable Difference Between Observed
Score and Predicted Score
Observed Predicted (One Years Growth)
BELOW Expectations -- Observed Score Was Less
Than Predicted Score
36
Performance Diagnostic Report
37
Performance Diagnostic Report
38
What is the likelihood that a student will be
proficient on a future PSSA exam?

• PVAAS Projections

39
Student Projection Methodology

• A separate longitudinal modeling process focused
  on the performance scores of individual students.
• The model uses all of the available data (both
  math and reading) to project future PSSA
  performances in Math, Reading, Science and
  Writing.
• Projections can be calculated for an individual
  student or groups of students.

40
PVAAS Student Projection Report
41
Identified Students Meeting Search Requirements
42
District (Single Grade)Projection Summary Report
43
PVAAS Fall 2009 Reporting Projections to Basic,
Proficient, Advanced
Math and Reading 3 ? 4 or 5 4 ? 5 or 6 5 ? 6 or 7 6 ? 7 or 8 7 ? 8 8 ? 11 Writing 3 ? 5 4 ? 5 5 ? 8 6 ? 8 7 ? 8 8 ? 11 Science 3 ? 4 4 ? NA 5 ? 8 6 ? 8 7 ? 8 8 ? 11
Grade PSSA Last Taken
Grade PSSA Last Taken
Grade PSSA Last Taken
44
Approved PAs Proposal for Growth Model as
Means to Make AYP