All You Ever Wanted to Know About Tests

1
All You Ever Wanted to Know About Tests and
More!
2
Outcomes

• Identify types of tests
• Understand types of scores
• Be able to interpret test reports

3
Common VocabularyCommon Understanding
Learning Gains
FCAT CRT
FCAT NRT
SSS
Scale score
Raw Score
NPR
Achievement Level
4
Two Major Types of Tests

• Norm-Referenced Test (NRT)
• Criterion-Referenced Test (CRT)

5
What is a Norm-Referenced Test (NRT)?

• A standardized assessment in which all students
  perform under the same conditions.
• It compares the performance of a student or
  group of students to a national sample of
  students at the same grade and age, called the
  norm group.

6
What is a Criterion-Referenced Test (CRT)?

• An assessment where a student's performance is
  compared to a specific learning objective or
  performance standard and not to the performance
  of other students.
• It tells us how well students are performing on
  specific goals or content standards rather than
  just telling how their performance compares to a
  norm group of students nationally or locally.

7
Summary NRT and CRT
8
Raw Score (RS)

• The number of items a student answers correctly
  on a test.
• John took a 20 item mathematics test (where each
  item was worth one point) and correctly answered
  17 items.
• His raw score for this assessment is 17.

9
Scale Score (SS)

• Mathematically converted raw scores
  
• that use a new scale to represent levels of
  achievement or ability.
• For FCAT-SSS, a computer program is used to
  analyze student responses and to compute the
  scale score. It reports test results on the
  students entire test.

10
Scale Score (SS)

• Higher scale scores indicate higher proficiency.
• On a continuous, vertical scale across grade
  levels you can track a students progress from
  lower to upper grade levels on one scale. Growth
  in scale score units indicates growth in
  proficiency.
• For FCAT-SSS, the Developmental Scale Score is
  used to determine a students annual progress
  from grade to grade.

11
Student Growth?
12
(No Transcript)
13
National Percentile Rank (NPR)

• Percentage of students in the norm group whose
  scores fall at or below a given students score.

• For example, a student scoring an NP of 71,
  scored as well or better than approximately 71
  percent of the students in the national norm
  group.

14
Percentile Score and NCE
Percentile 1 10
20 30 40 50 60 70
80 90 99

• Distances between points are not equal on a
• percentile scale.
• At ends of scale the distances are larger.

Normal Curve Equivalent 1 10
20 30 40 50
60 70 80 90
99

• On a NCE scale the distances are equal.

15
Normal Curve Equivalent (NCE)

• Normal Curve Equivalents (NCEs) are
  norm-referenced scores ranging from 1 to 99 and
  have an average score of 50.
• NCEs are an equal interval scale which allows for
  arithmetic calculations.
• Changes in academic achievement are usually
  measured through NCE gains.
• A student or group of students make an average
  years growth if they receive the same NCE score
  for two consecutive years.

16
True Score

• A score entirely free of error.
• Hypothetical value that can never be obtained by
  testing, since a test score always involves some
  measurement error.
• A students "true" score may be thought of as the
  average of an infinite number of measurements
  from the same or exactly equivalent tests,
  assuming no practice effect or change in the
  student during the testing.

17
Standard Error of Measurement (SEM)

• The amount a students score is expected to
  fluctuate around his or her true score.
• SEM is frequently used to obtain an idea of the
  consistency of a students score or to set a band
  around a score.
• For example, if a student scores 110 on a test
  and SEM6, we would say we are 68 confident the
  students true score was between (1101 SEM) and
  (1101 SEM) or between 104 and 116.

18
(No Transcript)
19
Learning Gains
The amount of progress a student makes in one
school year.
20
Learning Gains Who Qualifies?

• All students with matched, consecutive years
  (i.e. 2005 2006) FCAT SSS results, grades 4-10,
  who were enrolled in the same school surveys 2
  3 (FTE)
• including all subgroups (ESE, LEP, etc.).

21
Learning Gains Which Scores?

• Gains apply in reading and math, not writing.
• Previous years test score may be from same
  school, same district, or anywhere in the state.

22
Learning Gains What equals Adequate Yearly
Progress (AYP)?
A. Improve FCAT Achievement Levels from 2005 to
2006 (e.g. 1-2, 2-3, 3-4, 4-5) OR B. Maintain
satisfactory Achievement Levels from 2005-2006
(e.g. 3-3, 4-4, 5-5) OR C. Demonstrate more
than one years growth within Level 1 or Level 2
- determined by DSS Cut Points (not applicable
for retained students)
23
Learning Gains Retainees
A retained student can only be counted as making
adequate progress if he/she A. Moves up one
level. (e.g. 1-2, 2-3, 3-4, 4-5)
(Reason A) B. Maintains a level 3, 4, or 5.
(Reason B)
24
Developmental Scale Score Gains Table (DSS
Cut Points)
Students achieving within Level 1 (or within Level 2) for two consecutive years must gain at least one point more than those listed in the table in order to satisfy the making annual learning gains component of the school accountability system. Students achieving within Level 1 (or within Level 2) for two consecutive years must gain at least one point more than those listed in the table in order to satisfy the making annual learning gains component of the school accountability system. Students achieving within Level 1 (or within Level 2) for two consecutive years must gain at least one point more than those listed in the table in order to satisfy the making annual learning gains component of the school accountability system.
Grade Level Change Reading Mathematics
3 to 4 230 162
4 to 5 166 119
5 to 6 133 95
6 to 7 110 78
7 to 8 92 64
8 to 9 77 54
9 to 10 77 48
25
Learning Gains Activity

• Using the data on the following table, determine
• which students made a learning gain.
• what percentage of the teachers students made a
  learning gain.

26
Data Display for FCAT Reading Results
Student 04/05 Grade Level 05/06 Grade Level Pre-test Achievement Level Pre-test DSS Post-test Achievement Level Post-test DSS Learning Gain Determination
A 7 8 Level 1 Level 2 Yes or No Reason A, B, or C
B 7 8 Level 4 Level 4 Yes or No Reason A, B, or C
C 7 8 Level 2 1598 Level 2 1743 Yes or No Reason A, B, or C
D 8 8 Level 1 Level 2 Yes or No Reason A, B, or C
E 8 8 Level 3 Level 3 Yes or No Reason A, B, or C
F 8 8 Level 1 1486 Level 1 1653 Yes or No Reason A, B, or C
G 7 8 Level 5 Level 4 Yes or No Reason A, B, or C
27
Teacher Learning Gains Based on Data Display
Total Number of Students with a Pre and Post-test who qualify for learning gain calculations Reason A Increased 1 or more Achievement Levels Reason B Maintains satisfactory levels (3, 4, or 5) Reason C DSS Target Gain (More than a years growth)
7 2 2 1

• 5 out of 7 students made learning gains.
• 71 of this teachers students made learning
  gains and add points towards the schools grade.
• No points are given to the school for Student F
  because he was retained and stayed within level 1
  even though he made significant gains in DSS
  points.
• No points are given to Student G because he
  decreased a level.

28
Class Record Sheet for Learning Gains
29
STANFORD 10 Activity
30
(No Transcript)
31
References/Acknowledgements

• Bernhardt, Victoria L. Data Analysis for
  Comprehensive School Improvement, Eye on
  Education, Inc., 1998.
• Wahlstrom, D. (1999). Using Data To Improve
  Student Achievement, Virginia Beach, VA.
  Successline, Inc.
• Harcourt Brace Educational Measurement (2001).
  Glossary Of Measurement Terms. Internet document.
  San Antonio, TX.
• Ferrer,Wilma. Power Point development.
• Council for Educational Change, Student
  Performancec Snapshot