Vertical Scale and Texas Projection Measure

1
Vertical Scale and Texas Projection Measure

• Dr. Stacy Daugherty Edvance Research
• Laurie ODonnell Dickinson ISD

2
Scaling Test Performance

• A raw score is often used as an indicator of a
  students performance on a particular test.
• Example 41 out of 50 (or 82)
• Raw scores may have different meaning between two
  tests if the tests are not of equal difficulty.
• Does 41/50 (82) indicate the same level of
  performance on an easy test as it does a hard
  test?

3
Raw Scores Subtraction Test 4th Grade Math

• Suppose Student A gets 4
• correct out of the following 6
• problems
• 1. 13 2
• 2. 276 35
• 3. 45 23
• 4. 894 94
• 5. 637 16
• 6. 3,962 2,241

• Suppose Student B gets 4
• correct out of the following 6
• problems
• 13 8
• 2. 276 39
• 3. 45 7
• 4. 894 55
• 5. 637 140
• 6. 3,962 2,974

Question If Student A and Student B both get 67
correct (raw score), have they demonstrated the
same level of learning?
3
4
Scale Score

• A scale score is a conversion of the raw score
  onto a scale that is common to all test forms
  for the assessment.
• The scale score takes into account the difficulty
  level of the specific set of questions.
• 36/50 3750 41/50
• Harder Test Form Easier Test Form

5
TAKS Horizontal Scale

• For TAKS, the scale scores were defined such that
  the Met Standard and Commended Performance cuts
  were always to same scale values across forms and
  grades
• Met Standard 2100
• Commended 2400
• This is known as a horizontal scale

6
Horizontal Scales

• Limitations of horizontal scales
• Scale scores across test forms can be compared
  within a grade, but not across grades for a
  subject.
• Standards (Met Standard and Commended) cannot be
  numerically compared across grade levels
• Example 2100 in Grade 5 ? 2100 in Grade 6
• It is difficult to evaluate individual student
  progress from grade to grade.

7
Vertical Scale

• A vertical scale score
• Shows how well a student did on the TAKS -
  calculated from the number of questions a student
  answered correctly
• Can be numerically compared across different
  grades for the same subject
• Changes in a students vertical scale scores show
  academic progress over time

8
Vertical Scale Score

• Potential score range from approximately 0 to
  1000
• Vertical scale scores in grades 3-6 will be
  preceded by an E or an S to indicate language
• Grade 4 mathematics examples
• E 562
• S - 585

9
Performance Standards on Vertical Scale

• SB 1031 required a vertical scale for grades 3-8
  in reading and mathematics
• Performance standards were reviewed and some will
  be increased between 2009 and 2010 to reflect an
  appropriate increase in expectations across
  grades
• Vertical scale scores in 2009 reflect the current
  cut points
• Vertical scale scores in 2010 will reflect
  revised (higher) cut points

10
Vertical Scale Required Adjustments to
Performance Standards

• Aligning the current horizontal scale scores
  (2100 for Met Standard and 2400 for Commended) to
  vertical scale scores will require the
  cutpoints for Met Standard and Commended on
  some assessments to be increased

Grade Subject TAKS English TAKS English TAKS Spanish TAKS Spanish
Grade Subject Met Std Commended Met Std Commended
3 Reading - - - -
3 Math - - Increase -
4 Reading - - - -
4 Math - - Increase Increase
5 Reading - - - -
5 Math - Increase - -
6 Reading Increase Increase Increase Increase
6 Math - Increase Increase Increase
7 Reading - -
7 Math - -
8 Reading Increase Increase
8 Math - Increase

• Will be applied beginning in Spring 2010
• Will be reported beginning in Spring 2009

11
Grade 8 Reading Example

• Spring 2009
• Met Standard 2100
• Met Standard 670 (previous cut score)
• Spring 2010
• Met Standard 700 (revised cut score)

Additional 4 questions needed to pass in 2009
12
Grade 6th Reading Example

• Spring 2009
• Met Standard 2100
• Met Standard 614 (previous cut score)
• Spring 2010
• Met Standard 644 (revised cut score)

Additional 3 questions needed to pass in 2009
13
Vertical Scale Cutpoints (English) Met Standard
14
Vertical Scale Cutpoints (English) - Commended
15
Vertical Scale Cutpoints (Spanish) Met Standard
16
Vertical Scale Cutpoints (Spanish) - Commended
17
Vertical Scale Score Math Example
Description Grade 5 2008 Grade 6 2009 Growth

Student Score 580 650 70
MET STANDARD 603 637 34
Difference -23 13 36
Passed No Yes Excellent
Uses 2009 standards
18
Exercise 1 VS Score - Reading
Description Grade 6 2008 Grade 7 2009 Growth

Student Score 601 657
MET STANDARD 614 670
Difference
Passed?
Hint Use 2009 standards this year to determine
Met Standard.
19
Spring 2009 Reporting Changes

• Same basic reporting information as past years
• Old scale with 2100 as Met and 2400 as Commended
  Performance
• Same graph on Confidential Student Report (CSR)
• New Information
• Vertical scores added near Lexile/Quantile scores
• Data file
• Vertical scale scores for 2009
• Vertical scale scores for 2008 (converted)
• Parent brochure explaining new information
• Blackline master information

20
Vertical Scale(Interpreting the Increases in
Performance Standard)
Grade Level
Vertical Scale Score (Note In 2010, a
Vertical Scale Score of 670 For 8th Grade Reading
Does NOT Meet Std)
Horizontal Scale Score 2100 Met Standard
21
Vertical Scale Parent Script

• Your child received a vertical scale score for
  both reading and math. This score is based on
  the number of questions your child answered
  correctly.
• You can use your childs vertical scale score
  to compare performance from one year to the next
  in the same subject area. This score will help
  you keep track of your childs progress in
  reading and math.
• This is how we track her progress
• 1. Compare this years score to last years
  to measure growth.
• 2. Compare her growth to the expected
  growth.
• 3. Compare her score to the performance
  standards.
• Additional information is available on the
  TEA website.
• Computers are available on campus for parents
  at .

22
Which TAKS Will NOT Have a Vertical Scale?

• Writing
• Science
• Social Studies
• High School Assessments
• End-of-Course Assessments

23
What About Other Assessments?

• TAKS (Accommodated) will be reported on the same
  vertical scale as TAKS
• TELPAS will be reported on a vertical scale with
  score range approx. 0 1000
• TEA is considering a vertical scale for TAKS-M
• There are no plans currently for a vertical scale
  for TAKS-Alt or EOC

24
Vertical Scales

• Expected Growth Amounts

Grade Reading Reading Reading   Math Math Math
Grade Prior Yr Current Yr Expected Gain   Prior Yr Current Yr Expected Gain
3rd to 4th 483 554 71   500 554 54
4th to 5th 554 620 66   554 603 49
5th to 6th 620 644 24   603 637 34
6th to 7th 644 670 26   637 670 33
7th to 8th 670 700 30   670 700 30
25
Texas Projection Measure

• (TPM)

26
Why a Projection Model?

• To offer alternative approaches to demonstrating
  student achievement that meet both state and NCLB
  goals

27
How the Projection Model Works

• TPM estimates whether a student is likely to pass
  TAKS assessments at a future grade 5th, 7th
  (writing only), 8th, or 11th
• TAKS
• TAKS (Accommodated)
• LAT
• TPM is based on
• Students current performance on TAKS
• TAKS scores of all the students on the campus
  that the student attends

28
TPM Content Areas

• TPM will be reported on
• Mathematics
• Reading/ELA
• Science
• Social Studies
• Writing

29
TPM Projection Subjects and Grades
TAKS Subject Grades Projected From Grade Projected To
Reading/ELA 3 and 4 5
Reading/ELA 5, 6, and 7 8
Reading/ELA 8, 9, and 10 11
Reading (Spanish) 3 and 4 5 (in Eng. and Span.)
Reading (Spanish) 5 and 6 8 (English)
Mathematics 3 and 4 5
Mathematics 5, 6, and 7 8
Mathematics 8, 9, and 10 11
Mathematics (Spanish) 3 and 4 5 (in Eng. and Span.)
Mathematics (Spanish) 5 and 6 8 (English)
Writing 4 7
Writing (Spanish) 4 7 (English)
Science 5 8
Science 10 11
Science (Spanish) 5 8 (English)
Social Studies 8 and 10 11
30
TPM Predictors

• Uses three predictors for reading and mathematics
  projections
• Students current year scale score in reading
• Students current year scale score in mathematics
• Average campus scale score in the projection
  subject (i.e. reading for reading, math for math)
• For science, social studies, and writing, add the
  students scale score for that subject for a
  total of four predictors

31
Predictors Used in TPM
Projection Subject Predictors Used
Reading 1. Student Reading Scale Score (current year)
Reading 2. Student Math Scale Score (current year)
Reading 3. Mean Campus Scale Score in Reading (prior year)
Math 1. Student Math Scale Score (current year)
Math 2. Student Reading Scale Score (current year)
Math 3. Mean Campus Scale Score in Math (prior year)
Writing 1. Student Writing Scale Score (current year)
Writing 2. Student Reading Scale Score (current year)
Writing 3. Student Math Scale Score (current year)
Writing 4. Mean Campus Scale Score in Writing (prior year)
Science 1. Student Science Scale Score (current year)
Science 2. Student Reading Scale Score (current year)
Science 3. Student Math Scale Score (current year)
Science 4. Mean Campus Scale Score in Science (prior year)
Social Studies 1. Student Social Studies Scale Score (current year)
Social Studies 2. Student Reading Scale Score (current year)
Social Studies 3. Student Math Scale Score (current year)
Social Studies 4. Mean Campus Scale Score in Social Studies (prior year)
32
Equation

• Reading Prediction Equation for next high-stakes
  grade (6th grade projecting to 8th grade)

ReadingScore_2010 Constant
(Rd-Co Rd SS) (Math-Co Math SS)
(C-Mean_Co C-Rd-SS)
Constant a value determined when the
coefficients are developed annually Rd-Co
coefficient calculated from prior years grade 6
reading scores Rd SS students current year
scale score in reading Math-Co coefficient
calculated from prior years grade 6 math
scores Math SS students current year scale
score in math C-Mean_Co coefficient calculated
from prior years campus averages in grade 6
reading C-Rd-SS prior years campus average in
grade 6 reading
33
Equation 6th Reading Projected to 8th Reading

• Projected Reading Score2010
• Constant
• (Rd-Co Rd-SS)
• (Math-Co Math SS)
• (C-Mean-Co C-Rd-SS)

34
Equation 6th Reading Projected to 8th Reading
Equation

• Projected Reading Score2010
• 225.28
• (0.4770 Rd-SS)
• (0.2511 Math SS)
• Texas Projection Measure information and
  resources See this site for TAKS Mean Scores
  for campuses and districts
• http//www.tea.state.tx.us/index3.aspx?id3688me
  nu_id3793

35
Equation 6th Reading Projected to 8th Reading

• Projected Reading Score2010
• 225.28
• (0.4770 650)
• (0.2511 550)
• (0.0784 650)

36
Equation 6th Reading Projected to 8th Reading

• Projected Reading Score2010
• 225.28
• 310.05
• 138.105
• 50.96
• 724.395

If the projected score is equal to or greater
than the Met Standard cut score, then the student
IS predicted to meet the standard at the next
high-stakes grade level.
37
Grade 6 to Grade 8 Reading
Student Scores Student Scores Student Scores
0.4770 0.2511 0.0784
  CONSTANT Reading Vertical SS Math Vertical SS Campus Vert. SS Reading PROJECTION Met TPM?
1 225.28 640 640 640 741 Y
2 225.28 775 775 775 850 Y
3 225.28 600 600 600 709 Y
4 225.28 650 550 650 724 Y
5 225.28 650 550 550 717 Y
6 225.28 550 550 550 669 N
7 225.28 550 640 550 691 N
8 225.28 550 550 700 681 N
Note. Grade 8 Met Standard (Reading)700,
Commended Performance (Reading) 850
Grade 6 Met Standard (Reading)644, Commended
Performance (Reading) 797, Met Standard
(Math)637, Commended Performance (Math)783
38
Equation5th Science Projected to 8th Science

• Projected Science Score2010
• Constant
• (Rd-Co Rd-SS)
• (Math-Co Math-SS)
• (Science-Co Science-SS)
• (C-Mean-Co C-Science-SS)

39
The Equation Section is Finished
Work it out yourself or use the TPM calculator
link below
http//forwardfocus.pearson.com/tpmcalculator/
40
Distinguishing Vertical Scales from TPM

• Vertical Scales
• Vertical Scales will reflect individual student
  growth from year-to-year
• Students previous year performance is
  significant in fully utilizing Vertical Scales
• Primary use instruction
• What is working?
• What is not working?

41
Distinguishing Vertical Scales from TPM

• Texas Projection Measure
• TPM is a projection of a future TAKS score based
  on a students current TAKS scores
• Similar to projecting how tall a 10-year old
  child will be when he/she reaches 18
• Uses current student performance data
• Spring 2009 test results
• Students previous year performance is irrelevant
  to TPM
• TPM formula does NOT include an analysis of
  whether an individual student made progress from
  the previous year to the current year
• Primary use accountability
• Students counted as successful even though they
  Did Not Meet Standard on the test

42
(No Transcript)
43
Which students will have predicted performances?

• Students must have a current, valid score in the
  TPM projection content area Reading/ELA and
  Math.
• Students must have taken TAKS, TAKS
  (Accommodated), or LAT for all content-area
  predictors.
• Students must have taken the same language
  version for all content area predictors.
• To be decided and announced this fall
• A TPM methodology to allow two different
  languages in 2010)
• TPM for students who take TAKS-M in grades 4, 7,
  and 10

44
Which students will have predicted performances?
Current year Reading/ELA score for TAKS Current
year Math score for TAKS
YES!
Current year Reading score for TAKS-Acc Current
year Math score for TAKS-M
NO!
Current year Reading score for Spanish TAKS
Current year Math score for TAKS-Acc
NO!
Current year Reading score for Spanish TAKS
Current year Math score for Spanish TAKS
YES!
Note using current information on 9/18/09
44
45
How will this affect state and federal
accountability?

• All students who meet proficiency OR who are
  predicted to meet proficiency in the next
  high-stakes grade level will be counted in the
  Met Standard group.

46
Campus A
Spring 2009
Students who Met Standard in 2009
Yes
Yes
Yes
Yes
Yes
No
No
No
No
No
Met Standard based on assessment results 50
Students who are Predicted to Meet Standard based
on 2009 scores
Yes
Yes
Yes
Yes
No
No
No
Yes
Yes
Yes
5 students Met Standard
2 students who Did Not Meet Standard (but are
predicted to)
7 students count as Met Standard
Met Standard 70
46
47
Reporting 2009

• Online reports
• Student level
• 2008 vertical scale score
• 2009 vertical scale score
• Projection (yes or no)
• Summary reports for campus, district, region, and
  state levels (Expected soon)
• Number and percent Met Standard
• Number and percent Met Standard with TPM
• Number and percent Met Standard or Met Standard
  with TPM

48
Vertical Scale(Reading a CSR in 2009)
Grade Level
Vertical Scale Score
Horizontal Scale Score and Met Std Indicator
TPM Projected to Meet Std
49
TPM Parent Script

• Your child has received Texas Projection Measure
  (TPM) scores in addition to her TAKS scores.
  These new TPM scores let us know if your child is
  on track to pass at grade __ (the next
  high-stakes grade, and grade 7 Writing).
• According to the TPM score on the
  Confidential Student Report, your child is
  projected to pass/not to pass. We compare the
  projected score to the standard for the grade and
  subject. For students who are not projected to
  pass, remediation will be offered, such as .
• Additional information is available on the
  TEA website. Computers are available on campus
  for parents at .

50
Reporting Spring 2010

• For grades 5 and 8, any student retesting will
  have projections recalculated for all subjects
• Grade 8 science will have a TPM
• TAKS-M students in grades 4, 7, and 10 will have
  projections

51
Students Who Retest

• When a student takes a retest, the students test
  score from the primary administration will be
  used in the TPM as long as the primary test and
  retest have been administered in the same
  language.
• For accountability calculations, the TPM from the
  higher of the two administrations (primary or
  first retest) will be used.
• Students that retest will receive new projections
  in all subject areas each time

52
Prediction Model for TAKS-M
Current Grade Prediction Grade Year Data Available on First Cohort First Year Equations Applied
3 5 2010 2011
4 5 2009 2010
5 8 2011 2012
6 8 2010 2011
7 8 2009 2010
8 11 2011 2012
9 11 2010 2011
10 11 2009 2010
11 N/A N/A N/A
52
53
Prediction Model for TAKS-Alt

• Transition Table divides each proficiency level
  into three sublevels
• A descriptive transition table that describes
  students progress relative to their progress
  expectations will be developed.
• Progress targets that require students below
  proficiency to reach proficiency by the next
  high-stakes grade level (5, 8, and 11) will be
  created.

53
54
Possible Transition Table for TAKS-Alt
Number of Years from Current Grade to Prediction Grade Previous Performance Level Previous Performance Level Number of Sublevels Improvement Needed to Achieve Proficiency Number of Years to Achieve Proficiency Progress Target
Number of Years from Current Grade to Prediction Grade Level Sublevel Number of Sublevels Improvement Needed to Achieve Proficiency Number of Years to Achieve Proficiency Progress Target
1 year Did Not Meet Standard Low 3 1 Students must increase 3 sublevels
1 year Did Not Meet Standard Middle 2 1 Students must increase 2 sublevels
1 year Did Not Meet Standard High 1 1 Students must increase 1 sublevel
2 years Did Not Meet Standard Low 3 2 Students must increase 2 sublevels one year and 1 sublevel the other
2 years Did Not Meet Standard Middle 2 2 Students must increase 1 sublevel each year
2 years Did Not Meet Standard High 1 2 Students must increase 1 sublevel either year
3 years Did Not Meet Standard Low 3 3 Students must increase 1 sublevel each year
3 years Did Not Meet Standard Middle 2 3 Students must increase 1 sublevel in 2 of the three years
3 years Did Not Meet Standard High 1 3 Students must increase 1 sublevel in 1 of the three years
55
Using and Interpreting TPM

• Reporting will allow focused interventions by
  campuses and districts
• Four groups of students will be clearly
  identified
• Met Standard/Made Growth (Yes/Yes)
• Met Standard/Missed Growth (Yes/No)
• Missed Standard/Made Growth (No/Yes)
• Missed Standard/Missed Growth (No/No)

56
Using Data from the TPM

• Focus on effects rather than statistics
• Campus
• Teacher
• Interventions
• Look at the efficacy of curriculum and/or
  instruction

57
(No Transcript)
58
Resources

• Texas Education Agency
• http//www.tea.state.tx.us/index3.aspx?id3688men
  u_id3793
• http//www.tea.state.tx.us/index3.aspx?id3818men
  u_id3793
• Texas Turnaround Center
• http//www.txturnaround.org/webinars.html