Using Scoring Rubrics

1
Using Scoring Rubrics

• MAE Faculty Retreat
• October 20, 2006
• Dr. Judy Shoemaker
• Director, Office of Research Evaluation
• Division of Undergraduate Education

2
What is a scoring rubric?

• Scoring guide or scoring matrix
• Contains well-defined and systematically applied
  criteria
• Especially useful for assessing behavior or a
  product that has a strong subjective component
  (that is, cannot be scored objectively)
• Examples homework problems, oral presentations,
  essays, senior design projects, group skills,
  leadership.

3
What does a scoring rubric look like?

• Rows
• What is to be assessed
• Outcomes
• Columns
• Levels of performance for that outcome
• From acceptable to unacceptable
• Scores assigned to each level
• Cells
• Description (text) of what constitutes each level
  of performance for that outcome
• Rows and columns may be switched

4
Types of scoring rubrics

• Holistic (one dimension)
• To rate the overall process or product as a whole
• Uses a single scale
• Generates a single score
• Detailed or analytical (more than one dimension)
• To rate parts of the process or product
• Separate rating scales for each part
• Generates several scores
• Scores frequently summed

5
Scientific Report
6
Participation in Class Discussions
7
PO (c) Design a system, component or process to
meet desired needs
8
PO (g) Communicate effectively in a presentation
or report
9
Advantages of scoring rubrics

• Promotes consistency of ratings
• Across students
• Across raters
• Reduces rating errors
• Halo effect
• Central tendency errors
• Rater drift

10
More advantages

• Makes scoring easier and faster
• Useful as a learning tool
• Makes expectations clear to students
• Provides useful, detailed feedback to students
• Useful as a teaching guide
• Keeps focus on student learning

11
How to develop a scoring rubric

• Determine outcomes to be assessed
• Decide if holistic or detailed is more
  appropriate
• Select number of performance levels (3 to 5)
• Identify performance standard (for example, gt3)
• Create matrix
• Complete each cell in the matrix
• Start with end-points (excellent/poor
  acceptable/unacceptable)

12
How to develop.

• Try out and revise as needed
• Share scoring rubric with students
• Use scoring rubric
• Train raters, then check rater reliability
• Provide feedback to students

13
Resources

• Examples of engineering scoring rubrics
• www.ce.udel.edu/ABET/Current20Documentation/ABET_
  scoring_rubrics_index.html
• www.engr.sjsu.edu/assessment/topic/t1.html
• www.eng.auburn.edu/programs/chen/programs/accredit
  ation/assessment-rubrics.html

14
Resources

• Information about scoring rubrics
• Developing scoring rubrics for ABET outcomes
  assessent
• www.engr.iupui.edu/ece/assessment/scoringRubrics.h
  tml
• Scoring rubrics What, when and how?
• Pareonline.net/getvn.asp?v7n3
• Designing scoring rubrics for your classroom
• Pareonline.net/getvn.asp?v7v25