Assessment Tools Development: Rubrics

1
Assessment Tools DevelopmentRubrics

• Arman Abdullah
• FKKSA, UMP

2
Presentation Outline

• What is a Rubric? And Related Definitions
• Why Develop a Rubric?
• Reasons for Using Rubrics
• Questions Answered by Rubrics
• Advantages using Rubrics
• Characteristics of a Rubric.
• Elements of a Rubric
• Example of Rubric/Dimensions
• Rubric Development Guidelines/Protocol
• Rubrics Development - Degrees of Performance
• Scoring Rubric Options

3
What is a Rubric?

• A rubric is a printed set of scoring guidelines
  (criteria) for evaluating work (a performance or
  a product) and for giving feedback. http//www.ru
  brics.com/
• A rubric is a scoring guide that seeks to
  evaluate a students performance based on the sum
  of a full range of criteria rather than a single
  numerical score. (http//www.teachervision.fen.com
  )
• The established criteria, including rules,
  principles, and illustrations, used in scoring
  responses to individual items and clusters of
  items (AERA/APA/ NCME, 1999, p. 182).

4
Related Definitions

• Performance Element the major, critical
  attributes which focus upon best practice.
• Scale the possible points to be assigned (high
  to low).
• Criteria the conditions of a performance that
  must be met for it to be considered successful.
• Standard a description of how well the criteria
  must be met for the performance to be considered
  "good".
• Descriptors statements that describe each level
  of the performance.
• Indicators specific, concrete examples or
  tell-tale signs of what to look for at each level
  of the performance
• http//www.rubrics.com/4DACTION/W_ShowMemberArticl
  e/11

5
Rubric Development

• A BRAINSTORMING ACTIVITY
• Essay Examination

6
Why Develop a Rubric?

• How easy is it for you to justify the assessment
  or grade you have assigned to student if you had
  to defend it?
• Would you and other people say that the grading
  of your students' work or performance behavior is
  always accomplished objectively or subjectively?
• How clear are the assessment criteria you
  establish for each course you teach?

7
Why Develop a Rubric?(cont.)

• Many expert believe that rubrics improve
  students end products and therefore increase
  learning.
• When teachers evaluate papers or projects, they
  know implicitly what makes a good final product
  and why.
• When students receive rubrics beforehand, they
  understand how they will be evaluated and can
  prepare accordingly.

8
Reasons for Using Rubrics

• To focus instruction---intentionally.
• To guide feedback---descriptively.
• To characterize desired results---objectively.
• To operationalize performance standards
  (quality)---purposefully.
• To develop self-assessment competence---constantly
  .
• To involve students---thoughtfully.
• To quantify quality

9
Questions Answered by Rubrics

• A rubric answers such questions as
• By what criteria will the work be judged?
• What is the difference between good work and
  weaker work?
• How can we make sure our judgments (or scores)
  are valid and reliable?
• How can both performers and judges focus their
  preparation on excellence?

10
Advantages using Rubrics

• Teachers can increase the quality of their direct
  instruction by providing focus , emphasis and
  attention to particular details as a model for
  students.
• Students have explicit guidelines regarding
  teacher expectations.
• Students can use rubrics as a tool to develop
  their abilities .
• Teachers can reuse rubric for various activities.

11
Characteristics of a Rubric

• focus on measuring a stated objective
  (performance, behavior, or quality) -
  Performance Element
• use a range to rate performance - Scaling
• contain specific performance characteristics
  arranged in levels indicating the degree to which
  a standard has been met - Criteria

12
Elements of a Rubric

• A Rubric has Four important elements
• dimension
• definition and example of dimension
• scale
• standards of excellence

13
Elements of a Rubric Continued

• Dimension
• One or more dimensions (traits or features) that
  serve as the basis for judging the student
  response, performance or product.
• Definition and Example
• Definition and Example to clarify the meaning of
  each trait or dimension. Define the range of the
  trait or dimension to be assessed.

14
Elements of a Rubric Continued

• Scale
• A scale (the system of numbers, and their units
  of values) on which to rate each dimension.
  Scoring rubric rating scales may be numerical,
  qualitative, or a combination of the two.
• Standards of Excellence
• Standards of excellence for specified performance
  levels accompanied by models or examples of each
  level (Herman, Aschbacher, and Winters, 1992).

15
Example of Rubric

• Chocolate chip rubric
• The cookie elements chose to judge were
• Number of chocolate chips
• Texture
• Colour
• Taste
• Richness (flavor) .

16
Example of Rubric continued

• 4-Delicious
• Chocolate chips in every bite
• Chewy
• Golden brown
• Home baked taste
• Rich, creamy, high-fat flavor
• 3- Good
• Chocolate chips in about 75 of the bites taken
• Chewy in the middle, but crispy on the edges
• Either brown from overcooking or light from being
  25 raw
• Quality store-bought taste medium fat content

17
Example of Rubric continued

• 2-Needs Improvement
• Chocolate chips in 50 of the bites taken
• Texture is either crispy /crunchy from
  overcooking or doesnt hold together because it
  is at least 50 uncooked
• Either dark brown from overcooking or light from
  undercooking
• Tasteless
• Low fat content
• 1- Poor
• Too few or too many chocolate chips
• Texture resembles a dog biscuit
• Burned
• Store bought flavor with a preservative
  aftertaste- stale, hard, chalky Non fat contents
  

18
Example of Rubric continued
19
HANDS-ON Group Activity 1

• GROUP ACTIVITY Select a student performance you
  would like to evaluate.
• EXAMPLES
• An oral presentation
• A web page showing student research results
• A play
• A Sculptor or Art work
• A collaborative project to research a topic and
  produce a video to convey the information.
• TASK Use the sheet provided to brain-storm on
  identifying the four elements of a rubric

20
HANDS-ON Group Activity 2

• Developing A Rubric Template
• Examples
• Standards Rubric

21
HANDS-ON Group Activity 2 Continued

• Fill in the template with your criteria. Be sure
  to include the objective or behavior
  (categories), range/level, and the degree to
  which it has been met. Write specific
  descriptions of expected student performance for
  each level.
• Share your completed rubric with another group.

22
Examples of Dimensions
23
Examples of Dimensions continued
24
Rubric Template

• Description of identifiable performance
  characteristics reflecting a beginning level of
  performance. (Beginning)
• Description of identifiable performance
  characteristics reflecting development and
  movement toward mastery of performance
• (Developing)

• Description of identifiable performance
  characteristics reflecting mastery of
  performance.
• (Accomplished)
• Description of identifiable performance
  characteristics reflecting the highest level of
  performance. (Exemplary)

25
Rubric Development Guidelines

• Terms to use in measuring range/scoring levels
• Needs Improvement...Satisfactory...Good...Exemplar
  y
• Beginning...Developing...Accomplished...Exemplary
• Needs work...Good...Excellent
• Novice...Apprentice...Proficient...Distinguished
• (OR novice, basic, proficient, exemplary,).
• Numeric scale ranging from 1 to 5 (Example)

26
Rubric Development Guidelines

• Determine expected learning outcomes
• Use brief statements or phrases - Keep it short
  and simple (Include 4 - 15 items) -Should fit on
  one sheet of paper
• Focus on a different skill with each rubric item
• Focus on how students develop and express their
  learning
• Evaluate only measurable criteria
• Pilot test and Reevaluate the rubric (Did it
  work? Was it sufficiently detailed?)

27
Rubric Development Protocol

• Gather student work samples.
• Sort samples into 3 - 4 groups.
• Record your own descriptive statements.
• Categorize statements into critical performance
  elements.
• Write an operational definition of each element.
• Select the "best match" of student work per each
  level of performance--exemplary, proficient,
  basic, novice.
• Repeat steps . . . refining your rubric's
  elements, descriptors, and indicators.Sourcehttp
  //www.rubrics.com/4DACTION/W_ShowMemberArticle/1
  2

28
Rubrics Development - Degrees of Performance

• Concept Words for Conveying Degrees of
  performance
• Types Depth of Information...Breadth of
  Application...Quality...
• Scope...Extent...Complexity...
• Degrees...Accuracy
• Presence to absence
• Complete to incomplete
• Many to some to none
• Major to minor
• Consistent to inconsistent
• Frequency always to generally to sometimes to
  rarely

29
Scoring Rubric Options

• There are three options for selecting a scoring
  rubric
• Adopt
• Adapt
• Do it yourself

30
RESOURCES

• http//edweb.sdsu.edu/triton/july/rubrics/Rubric_G
  uidelines.html
• http//www.rubrics.com/
• http//www.rubrics.com/4DACTION/W_ShowMemberArticl
  e/12