Where is Student Academic Growth

1
Where is Student Academic Growth?

• Scaling for Simplification
• (New insights from Floridas experiences)

2
Calling all VS Geeks1!

• First of many sessions on vertical scaling,
  growth, and other related topics
• Eleven sessions a session at almost all of the
  time slots
• You can sleep in on Wednesday morning

1 AKA psychometricians, state assessment
directors, data analysts, data wonks, university
professors, etc.
3
Participants

• Gene Hoffman, HumRRO
• Donna Wolak, Reading TD, Florida DOE
• Vince Verges, Math TD, Florida DOE
• Robert Lissitz, University of Maryland
• Cornelia Orr, Florida DOE

4
Content Standards and Tests

• Sunshine State Standards in1995-96
• Grade Spans K-2, 3-5, 6-8, and 9-12
• Original Tests 4/5, 8, 10 in Reading, Writing,
  and Mathematics
• Grade Level Expectations in 2000
• Grade Level Tests 3-10 1999-2001
• New Test Development 1999-2001
• Field Test 2000 Reading and Math only
• 1st Scores Reported 2001
• Vertical Scale
• Established in 2001,
• Reported first in 2002

5
Vertical Scale Implementation

• 2001 First Data Collection - March
• Derive Scales (0-3000) August
• Reading 86-3008
• Mathematics 375-2709
• Use of Scale Vertically Articulated Standards
• 2002 Replication (more items 16 up down)
• Results Approximately the same
• Decision No changes to scale
• 2005 Replication (same as 2002)
• Results Similar results (interesting insights)
• Other Consideration Content Standards Under
  Revision
• Decision No changes to scale

6
Gene Hoffman, HumRRO

• Vertical Scaling Contractor

7
Test Development and Content Concerns

• Donna Wolak, Reading
• Vince Verges, Mathematics

8
Lessons Learned
9
Lessons Learned or Stuff That Matters

• It matters which items you choose.
• It matters that your content standards are
  vertically aligned.
• It matters how on grade level performance is
  defined.
• It matters if your test has ceiling/floor
  effects.
• It matters how sensitive the test is to
  instruction.

10
Lessons TO BE Learned

• How stable is the scale (sensitive to
  instruction)?
• When should the scale be changed (drift or no
  drift)?
• What are the impacts on accountability and AYP
  when the scale is changed?
• If the scale is changed will there be more or
  less growth or just different numbers?

11
Robert Lissitz, U of Maryland

• Discussant

12
Monitoring the Performance of the Vertical Scale

• Implementation/Replication
• Progress of Grade Level Cohort
• Analysis of Vertical Scale
• Analysis of Proficiency Results
• Analysis of Growth for Matched Cohorts
• Year-to-year comparisons
• Growth differences by score level

13
Mathematics Cohort Performance On the Vertical
Scale
14
Mathematics Cohort Performance Percent
Proficient Results
15
Reading Cohort Performance Vertical Scale
Results
2001 Grade Level
8th
7th
6th
5th
4th
3rd
16
Reading Cohort Performance Percent
Proficient Results
17
Mathematics Growth for Two Matched Cohorts
18
Reading Growth for Two Matched Cohorts
19
Mathematics Growth by Proficiency Level
Note These graphs contain discrete data. The
line graph is used to facilitate the
interpretation of the growth across two adjacent
grades.
20
Reading Growth by Proficiency Level
Note These graphs contain discrete data. The
line graph is used to facilitate the
interpretation of the growth across two adjacent
grades.
21
Contact Information

• Dr. Cornelia S. Orr, Administrator
• Assessment and School Performance
• cornelia.orr_at_fldoe.org
• [phone]