Using Growth Models in Longitudinal Data Systems

1
Using Growth Models in Longitudinal Data Systems

• May 2, 2006

2
Overview

• What are some of the growth models?
• What data are necessary?
• What are some of the policy implications?

3
What are some of the Growth Models?

• CCSSO book Policymakers Guide to Growth Models
  for School Accountability How Do Accountability
  Models Differ?
• National Center for Assessments presentation at
  AERA

4
What are some of the Growth Models?

• Definition of Growth Model for Accountability
• Measures progress of same students from one year
  to the next
• Answers the question How much, on average, did
  students performance change?
• May be benchmarked against a target

5
What are some of the Growth Models?

• Regression Models
• Value Tables

6
What are some of the Growth Models?

• Regression Models
• Value-Added Models
• Questions to be answered On average, did the
  students change in performance meet the growth
  expectation? By how much did the average change
  in student performance miss or exceed the growth
  expectation?
• Use background characteristics or prior
  achievement as controls to isolate the effects of
  a school, program or teacher on student academic
  progress
• Tennessee Value Added Assessment System (TVAAS)

7
What are some of the Growth Models?

• Regression Models (continued)
• Hierarchical Modeling Framework (CRESST model)
• Looks at the relationship of where a student
  starts and how much they grow
• Complex model with complex calculations
• Hybrid Success Model (Kingsbury)
• Similar to HM model
• Looks at individual student growth at at least
  two points in time using a common scale across
  all grades
• Based on Scale Scores

8
What are some of the Growth Models?

• REACH model (Doran)
• Uses value added results to serve as the basis
  for measuring progress towards a goal
• Dependent on scale scores

9
What are some of the Growth Models?

• Issues inherent in Regression Models
• Very complex with complex calculations
• Rely on scale scores and typically a vertical
  scale for the assessment system
• Could be difficult to explain

10
What are some of the Growth Models?

• Value Tables
• Measures individual student progress from one
  year to the next using performance levels and
  assigns a value to that growth
• Points depend on the values or outcomes that
  are desired
• Calculate the average growth for a school or
  subgroup

11
Sample Value Table
12
What are some of the Growth Models?
13
What data are necessary?

• Most of the systems either require
• Annual assessment systems that test at
  consecutive grades
• Assessments that measure at least 2 points in
  time
• Data systems that track students through the use
  of a unique student identifier
• Data systems to maintain multiple years of
  assessment data
• Some require vertical scales
• Some require articulated standards and
  performance levels that have consistent meaning
  across grades

14
What are some of the policy implications?

• Huge data systems requirement
• Infrastructure
• Data Collection
• Student Demographics
• Multiple years of assessment scores tied to
  unique ID
• Data Reporting

15
What are some of the policy implications?

• Measuring growth helps determine what model to
  choose
• What does the state value?
• How should growth be measured based on values?

16
What are some of the policy implications?

• Cost
• Multiple costs
• Data infrastructure and systems
• Assessments
• Are additional assessments necessary
• Are assessments vertically scaled or are
  performance levels articulated across grades
• Are content standards vertically aligned
• Resources to calculate
• Communication of growth model and results

17
What are some of the policy implications?

• NCLB Requirements
• Is model compensatory
• Not an option for NCLB use at this time
• Are all students proficient by 2014 or several
  years beyond if entered close to 2013
• Is the data infrastructure there to support the
  model
• Are the core principles of NCLB maintained
• Are the requirements for Standards and
  Assessments in place (approved through peer
  review process)

18
What are some of the policy implications?

• Many more policy implications and more issues
  some known and some being discovered as states
  design growth models for accountability use

19
Contact Information

• Cathy Wagner
• Minnesota Department of Education
• cathy.wagner_at_state.mn.us
• Phone number
• Robin Taylor
• Delaware Department of Education
• rtaylor_at_doe.k12.de.us
• 302-735-4080