Checklists

1
Checklists Rating Scales

• Scoring Performances
• Modified from original presentation byGlen
  Hammond, RRC, CAE, CIE

2
Goals

• To explore the importance of
• grading performance appropriately
• understanding, spotting, compensating for usual
  errors
• appropriately using
• checklists
• rating scales

3
(No Transcript)
4
Principles of Effective Rating

• Characteristics should be
• educationally significant
• in harmony with vision, goals, objectives,
  outcomes.
• directly observable
• occur in the setting
• clearly visible to an observer (skills easier
  than interests, feelings, attitudes)

5
Checklists and Rating Scales

• MONITORING and RECORD-KEEPING devices
• Increased emphasis on self-reflection and
  self-assessment
• Individual students
• assess their own progress
• develop an improvement plan

6
Checklists Rating Scales

• Checklist Attitude Yes or No
• shows originality Yes No
• respects own work Yes No
• is easily motivated Yes No
• Rating Scale Attitude -- Degree
• shows originality SA A U D SD
• respects own work SA A U D SD
• is easily motivated SA A U D SD
• (SA Strongly Agree A Agree U Undecided D
  Disagree SD Strongly Disagree)

7
Constructing Checklists

• Base on course objectives.
• State in clear and precise terms.
• Avoid generalities focus on specifics.
• Write to be easily understood by the student.
• can be basis for instructor/student discussion.
• allows for self- and peer-evaluation.

8
Constructing Checklists

• Keep short - focus on a few specifics.
• Can be used
• for single evaluation or cumulative record
• for diagnostic, formative, summative evaluation
• as a basis for anecdotal comment
• as instructional tool for learner to follow in
  practicing the skill.

9
Differentiating Assessment Tools

• CHECKLISTS
• psychomotor social skills
• indicate presence or absence/mastery or
  non-mastery of skill
• may indicate sequence of actions in a performance

• RATING SCALES
• for psychomotor and affective skills
• indicate quality, degree, or frequency of skill
  or trait
• useful in summarizing general impressions

10
Checklist Preparation for Varnishing

• __ sands and prepares surface properly
• __ wipes dust from surface
• __ selects appropriate brush
• __ selects varnish and checks varnish flow
• __ pours needed amount of varnish into clean
  container

11
Rating Scale Job Attitude

• is enthusiastic G Av N Imp
• is willing to work cooperate G Av N Imp
• desires to improve G Av N Imp
• is a good team worker G Av N Imp
• is concerned with safety G Av N Imp
• works without supervision G Av N Imp
• aware of job requirements G Av N Imp

12
Why Use Observation Checklists

• Quick and easy
• Record skills/behaviors rarely assessed prior to
  summative exam
• Can show areas that need work, early
• can help and correct before student failure
• Provide opportunities to change gears
• Desired behaviour can be identified, taught, and
  checked.

13
Why Use Observation Checklists

• Focus on what is observable
• Useful for monitoring growth or need for
  improvement
• More meaningful and authentic
• Identify learners level and needs

14
Why Use Observation Checklists

• Useful to
• discover patterns
• assess progress
• make plans to help students continue their
  learning.

15
Rating Scale Basics

• Use for psychomotor and affective domains
• give quality, frequency, degree, or level
  information
• for processes or products

16
Principles of Effective Rating

• Characteristics points clearly defined
• 3 to 7 rating positions
• Combine ratings from several observers whenever
  possible (greater reliability)
• omit ratings you feel unqualified to judge

17
Sample Rating Scale Degree of Confidence

• 1 little or no 2 some 3 great confidence
• inquiry teaching 1 2 3
• instructional modules 1 2 3
• simulations 1 2 3
• lecture 1 2 3
• leading a discussion 1 2 3
• questioning 1 2 3

18
Rating Scale Limitations

• Bias leniency, strictness, modesty, halo effect
• Social expectations (faking/self-deception)
• Interpretation
• Criterion inadequacy

19
Common Errors in Rating

• Personal Biastendency to rate all at
  approximately the same position on the scale
• generosity error - rates all at high end
• severity error rates all at low end
• central tendency error rates all as average

20
Common Errors in Rating

• Halo Effectgeneral impression influences the
  rating of individual characteristics
• Logical Error two characteristics incorrectly
  related
• intelligence and achievement
• giftedness and poor social adjustment

21
Common Errors in Rating

• Raters Attitude
• Rating unimportant, not enough effort
• Not enough time allocated
• Insufficient opportunities for observation.

22
Precautions/Considerations

• Begin with a blueprint of the behaviours
• ensure valid sampling within the area
• Clearly define traits
• Use most behavioural description possible
• Divide the rating continuum
• Use as many points as needed for clarity
• usually between three and seven.

23
Precautions/Considerations

• Clearly define anchors on continuum
• No question as to what each rating means
• Train and motivate the raters
• Stress accuracy and objectivity

24
Defining Steps on a Scale

• Numeric anchors
• numbers, often accompanied by verbal cues
• Degree of Agreement anchors
• Completely Disagree to Completely Agree
• Never to Always
• Adjectival anchors
• Bipolar adjective endpoints with numbers in
  between
• valuable 7 6 5 4 3 2 1 worthless
• passive 1 2 3 4 5 6 7 active

25
What is QUALITY?

• If something occurred
• occasionally, it would occur __ of the time.
• sometimes, it would occur __ of the time
• frequently, it would occur __ of the time
• usually, it would occur __ of the time
• often, it would occur __ of the time

26
Overcoming Problems

• Proper design, training, and use of the scales.