SCORING THE ALTERNATE ASSESSMENT

1
SCORING THEALTERNATEASSESSMENT
2
AGENDA

• Evidence Labels
• Scorers
• Rules of Scoring
• Scorer Recording Worksheet
• Data Folios Data Collection
• Post-test

3

• RELIABLE, ACCURATE, AND QUALITY
• SCORING
• WILL DEPEND ON
• RELIABLE, ACCURATE, AND QUALITY
• DATA

4
EVIDENCE LABELS
Nancy Drew
6
1.12.44.c
3
Qpoi Blask dgz,wcvn lhull sdort noun glud.
Blask dgz wcvn lhull spoi s ndorton gulud.
Bdgzl ask Qpoi wcvn lhull sd ortno wun jlud.
Snorg nkost sbeling tha ewak and boprog.
Crung swiple spid thew aspmg s woportsb ango meal.
5
EVIDENCE LABELSHandout 1

• Information provided on the evidence label is the
  reference point for scoring. It should help each
  scorer know the expectations for the individual
  students performance.
• All acronyms and abbreviations should be written
  out and/or explained.

6
EVIDENCE LABELS

• Across the sections of the evidence label, it
  might be necessary to highlight what makes each
  piece of data unique.
• For example, different settings, different
  times/days, different people, different task,
  different format, different materials

7
EVIDENCE LABELS
The Evidence Label provided by CETE has

• The students name
• The content area being assessed
• The students grade level
• The number of the evidence sample (1, 2, or 3)
• The extended indicator being measured

• The data collection setting
• The assessment task and materials
• Instructions that were given to the student
• Accommodations used on the activity, including
  acceptable supports

8
(No Transcript)
9
EVIDENCE LABELS
Each evidence label must be completed with the
following information

• The data collection setting
• The assessment task and materials
• Instructions that were given to the student
• Accommodations used on the activity, including
  acceptable supports

• The students name
• The content area being assessed
• The students grade level
• The number of the evidence sample (1,
  2, or 3)
• The extended indicator being measured

10
EVIDENCE LABELS

• a. Data Collection Setting
• Settings might be General Education Classroom,
  Resource Room, or in the Community.

11
(No Transcript)
12
EVIDENCE LABELS
Each evidence label must Include the following
information

• The students name
• The content area being assessed
• The students grade level
• The number of the evidence sample (1, 2, or 3)
• The extended indicator being measured
• The data collection setting

• The assessment task and materials
• Instructions that were given to the student
• Accommodations used on the activity, including
  acceptable supports

13
EVIDENCE LABELS

• b. Assessment Task and Materials
• It is critical to provide sufficient detail,
  such that all steps in the assessment task, as
  well as materials to be used, are clear.
• Remember! A scorer might not have any other
  knowledge of the student, task,
• and/or materials!

14
(No Transcript)
15
EVIDENCE LABELS
Each evidence label must be completed with the
following information

• The data collection setting
• The assessment task and materials
• Instructions that were given to the student
• Accommodations used on the activity, including
  acceptable supports

• The students name
• The content area being assessed
• The students grade level
• The number of the evidence sample (1, 2, or 3)
• The extended indicator being measured

16
EVIDENCE LABELS

• c. Instructions Given to the Student
• Clear specification of the exact directions
  provided for the student is critically important
  here.
• Details will assist the scorers to understand
• ? the demands of the task
• ? how the task is related to the
• indicator being assessed

17
(No Transcript)
18
EVIDENCE LABELS
Each evidence label must be completed with the
following information

• The data collection setting
• The assessment task and materials
• Instructions that were given to the student
• Accommodations used on the activity, including
  acceptable supports

• The students name
• The content area being assessed
• The students grade level
• The number of the evidence sample (1,
  2, or 3)
• The extended indicator being measured

19
EVIDENCE LABELS

• d. Accommodations, including acceptable supports
  used
• Relative to the student performing this task, in
• the setting where assessment is to occur
• ? Explain any accommodations that are
  typically provided to the student
• AND
• ? Provide explicit details re supports which
• are / are not acceptable for assisting the
  student

20

• Supports should be written to focus on the
  prompts expected to be provided at the time of
  assessment, not prompts, that might have been
  required in the past.

not
21
(No Transcript)
22
EVIDENCE LABELS

• An evidence label may need the connection between
  the task and the indicator explained, especially
  if this relationship would not be readily obvious
  to someone outside the field of SPED.

23
EVIDENCE LABELS

• Make sure that each piece of evidence is
  correctly labeled, and that each number matches
  the selected indicator.
• MAKE SURE THERE IS A COMPLETED EVIDENCE LABEL FOR
  EVERY PIECE OF EVIDENCE.

24
SCORERS
25
WHO WILL THE SCORERS BE?

• Each data folio will be scored independently
  by three different scorers
• one will be the students special education
  teacher of record
• the remaining two scorers will be determined
  locally (anyone chosen to score should be
  familiar with the Kansas curriculum standards,
  including the extended standards)

26
POSSIBLE SCORERS

• School psychologist
• School social worker
• Related service provider
• Another special education teacher
• Administrator
• General educator
• Licensed job coach
• A paraeducator may
• be a scorer

not
27
Entering the Scores

• Each scorer will independently determine their
  scores.
• Scores will then be sent to a Scorer Recorder to
  enter the scores onto CETE.
• Scores should be sent to the Scorer Recorder in a
  sealed envelope.
• (This is important to preserve the independence
• of the scoresand the technical
• integrity of the Kansas Alternate
• Assessment process.)

28
Entering the Scores

• The Score Recorder
• Can not have served as a scorer for any alternate
  assessment folio.

29
RULES OF SCORING
30
Entering the Scores

• If a student is absent you will finish their
  assessment when they return to school.

31
DATA FOLIO ELEMENTS REQUIRED FOR SCORING

• Table of contents a list of the indicators
  provided to CETE (printable from CETE website)
• Evidence label for each piece of evidence
  (printable from CETE website)
• The evidence itself, complete with the date on
  which each piece was collected
• If any one of these is missing, a scorer must
  attempt to locate it before proceeding with the
  scoring!

32
RULES OF SCORING

• Each scorer must
• Complete the scoring independently
• Refrain from comparing scores, or discussing the
  scoring with anyone else (including other
  scorers)
• Refrain from asking the students SPED teacher
  for assistance

33
RULES OF SCORING

• ALL scoring must be based solely on the evidence
  submitted to be included in the students
  alternate assessment data folio.
• The extended curricular standards for reading,
  mathematics, science, or history-government may
  be used as references during the scoring of the
• data folio.

34
RULES OF SCORING

• Scorers need to be objective and impartial
• The following cannot influence scoring
• Personal feelings
• Content presentation - neatness, art work, color,
  etc.
• Quality of photos or audiotapes
• Regulate time spent on scoringdont rush, but
  dont spend too much time
• Maintain the order of the data
  folio

35
RULES OF SCORING

• ALL data collection and scoring must be completed
  within the assessment window
• January 2, 2008
• (See Handout 2 Testing Schedule)

36
Testing Schedule
37
RULES OF SCORING

• The scorer must score each piece of evidence on
  the
• Skill Performance Rubric
• provided by the Kansas State Department of
  Education.

38
RULES OF SCORING
SKILL PERFORMANCE RUBRIC
No half points may be given..
39

• A scorer must rate what/how the student performs
  on the day of the assessment, and not what is
  his/her typical or best performance.

40
GUIDED PRACTICE

• Score Handout 4 using the Skill Performance
  Rubric.

41
RULES OF SCORING

• Only three pieces of evidence are to be scored
  for each selected extended indicator
• If more than three pieces of evidence are
  included for a single indicator, some will NOT be
  scored
• First scorer will select three pieces to be
  scored
• Remaining scorers must score same items
• Extra pieces of evidence should remain
• in the students folder

42
RULES OF SCORING

• What if there are two pieces of evidence, rather
  than three?
• Make an attempt to locate the missing piece of
  evidence.
• If you cant locate it, make a note in the data
  folio, then rate the two pieces provided.
• Score the missing piece as a 0.

43
RULES OF SCORING

• If no evidence is submitted, or the evidence
  included is not sufficiently complete to score,
  ask the teacher to see if the missing information
  can be located.
• If additional information does not become
  available, then the scorer is to score a 0 (for
  each piece of missing or unclear data).

44
RULES OF SCORING

• Each piece of evidence must include a minimum of
  five trials, or five opportunities for the
  student to respond.
• More than five trials (i.e., data points) MAY be
  included in a single piece of evidence, and WILL
  be considered in scoring.

45
RULES OF SCORING

• Five trials, or response opportunities, might
  look like
• five responses marked or -
• one worksheet of five computation problems
• one story retelling opportunity, which includes
  five key points
• videotape, including five questions from
  teacher, which student answers

46
RULES OF SCORING

• What if more than 5 responses are included in one
  piece of evidence?
• Score all the recorded responses within each
  piece of evidence as described on the data label.
• Calculate an average score accordingly.

47
GUIDED PRACTICE 5

48
RULES OF SCORING

• What if fewer than 5 responses are included in
  one piece of evidence?
• Make every effort to locate the missing
  information.
• If no additional information can be found, note
  this in the data folio, then score all the
  evidence that is provided.
• Use the score from the available evidence as the
  numerator of a fraction and a score of 5 for
  the denominator of the fraction to
  compute the overall score.

49
RULES OF SCORING

• It is critically important that the evidence
  provided matches the extended indicator selected
  for assessment,
• AND
• That each scorer be able to discern this match.

50
RULES OF SCORING

• What if the evidence provided does not seem to
  match the selected indicator?
• Ask the students SPED teacher, Is this the
  evidence intended for this indicator?
• Double check the evidence label to ensure the
  numbers match.
• Review the Clarifying Examples for the extended
  indicator being assessed.
• If the scorer continues to believe that there
  is an obvious mismatch between the
  evidence and the selected
  indicator, the evidence will
  be scored a 0.

51
RULES OF SCORING

• Each piece of evidence must have its own
  corresponding evidence label.
• Remember!
• This is the only way for the teacher or data
  collector to communicate with the scorers about
  the uniqueness of each piece of evidence in the
  data folio.

52
RULES OF SCORING

• What if an evidence label does not accompany each
  piece of evidence?
• Check with the students SPED teacher to see if a
  label exists.
• If no label can be located, then the students
  SPED teacher must create an evidence label.
• DO NOT PROCEED UNTIL AN EVIDENCE LABEL IS
  PROVIDED.

53
SCORER RECORDING WORKSHEET

• CETE

54
SCORER RECORDING WORKSHEET

• Download the Scorer Recording Worksheet from the
  CETE website.
• Make copies for each of the scorers. Circle the
  rater number for each of the scorers.

55
SCORER RECORDING WORKSHEET
56
SCORER RECORDING WORKSHEET

• Information included on the scorer recording
  worksheet
• Student name
• Content area
• Rater Number or Rater Name
• Indicators number 1 5
• Space for recording scores

57
SCORER RECORDING WORKSHEET

• Record a score for each piece of evidence on this
  worksheet.
• Send the score sheet to the person who will
  record scores in your district.
• The worksheets should be maintained and stored
  locally after the assessment is completed.

58
Learner Characteristics Inventory

• There Will Be
• No
• Learner Characteristics
• Inventory
• This Year

59
DATA FOLIOS AND DATA COLLECTION
60
A COMPLETE DATA FOLIO SHOULD INCLUDE

• Three pieces of evidence for each of the five
  selected extended indicators.
• The data folio for a reading alternate assessment
  will have 15 pieces of evidence (5 indicators X
  3 pieces of evidence per indicator).
• The data folio for a mathematics alternate
  assessment will have 15 pieces of evidence (5
  indicators X 3 pieces of evidence per indicator).
• The data folio for a science alternate assessment
  will have 15 pieces of evidence (5 indicators X
  3 pieces of evidence per indicator).
• The data folio for a history-government alternate
  assessment will have 15 pieces of evidence
  
  (5 indicators X 3 pieces of evidence
  per indicator).

61
A COMPLETE DATA FOLIO SHOULD INCLUDE

• The raw data for any graph or other summary
  report.
• A key or legend for any code used in reporting
  the data.
• The raw data for any rubric scoring used in the
  data folio.
• The teacher may not use the KSDE Skill
  Performance Rubric for the collection or
  reporting of the data included in the folio.

62
Handout 6
63
POST TEST

• Complete the post test on-site today.
• Score independently do not visit with anyone
  regarding your responses.
• Use the materials provided to you today as
  necessary.
• Leave the post-test with your on-site
  facilitator.

64
Questions?

• If additional questions come up after the
  training, please e-mail
• Debbie Matthews dmatthews_at_ksde.org
• 785-296-0916
• Joan Houghton
• jhoughton_at_ksde.org
• 785-296-2515