Scaling

1
Scaling Grading in Examinations

• Presented by
• Dr. Manisha Taneja
• Lecturer in Education
• R.M.S. College of Education,
• Behrampur, Sec-74, Gurgaon

2
Adding marks to decide result

• Doctors measure several indicators of health such
  as height, weight, body temperature, blood
  pressure, etc, but do not add them to determine
  the overall health of a patient (why?).
• But, teachers add marks obtained by the students
  in different subjects in order to assess the
  overall school performance (how strange?).
• Doctors know that the measures they obtain assess
  different traits are not on the same scale, and
  hence, cannot be added or subtracted.

3
How can we add or compare?

• Only measurements on the same scales can be added
  or subtracted. We change inches, feet and yards
  to meters before performing arithmetical
  operations.
• We cannot even compare quantities of traits
  unless we convert them to a common scale.
• Converting raw marks to common scale is
  necessary.

4
Standard of an Evaluator

• Marks in different subjects vary in overall level
  (mean score) and spread (SD).
• The standard of marking of an evaluator is
  defined in terms of mean and SD of the raw scores
  awarded by him while evaluating a given set of
  answer books.
• When marks are combined, the SDs of different
  components play significant roles in the
  combination.
• The weight of a given component in the
  combination is proportional to its SD. The weight
  of a component with SD as 15 will be thrice the
  weight of the one with an SD of 5 points.

5
Weaknesses of Essay Examinations

• Low validity, low scorer-reliability, high
  subjectivity, low comparability, unfairness
• Optional questions, such as 5 out of ten, reduce
  comparability
• Fail pass, cut-scores for different
  divisions/categories are arbitrary.
• Combining marks by adding arithmetically is a
  highly unscientific.
• Before combining, marks the should be scaled or
  standardized

6
What is Scaling?

• Converting measurements taken on different scales
  to a common scale is scaling.
• The basic idea behind scaling is that the
  distance of a scaled score from scaled-score mean
  in terms of scaled-score SD equals the distance
  of the corresponding raw score from the raw-score
  mean in terms of raw-score SD.
• It is simply a linear transformation which does
  not change the original distribution of scores.
  The common examples of scaled scores are
  z-scores, T-scores and ETS-scores etc.

7
Comparing performance in two areas

• Hindi
  Maths
• Mean 50
  60
• SD 10
  12
• Student A 60
  72
• Student B 55
  60
• Student C 65
  65 (How do they compare ?)

8
Methods of Scaling

• There are a few practical methods of scaling. One
  is nomogram method, and the other is graphical
  method.
• The methods may be explained by the following
  example
• Head
  Assistant
• Mean 55
  63
• SD 10
  15
• Range 26 82
  30 87
• The assistants awards are to be scaled on heads
  awards which are assumed to be a standard.

9
Nomogram method

• Draw two parallel lines of about the same length
  and represent the lowest and the highest scores
  awarded by the Head at the end-points of one
  line then divide the line into parts to
  represent scores.
• Do the same thing for assistants lowest and
  highest awards but in reverse direction.
• Join the diagonally opposite points by
  intersecting straight lines the line originating
  at any un-scales score point through the point of
  intersection will meet the opposite line at the
  scaled score point.

10
Graphical Method -1

• Convert the lowest and highest score by the
  assistant to z-scores and multiply by heads SD
  and then add heads mean to each. This gives the
  scaled (to heads standard) scores of 33 and 71.
  
• Then select points (30, 33) and (87, 71) on the
  graph paper and draw a line joining them.
• This line may be used to find the scaled score
  for any un-scaled score. The equation of this
  line may also be used to compute scaled scores.

11
Graphical Method -2

• Alternatively, find the points on both the scales
  one SD below and one SD above the respective
  means. This would result in the pair of points
  (78, 65) and (48, 45), which may be plotted and
  joined to find out the line. These points may
  also be used to find out the equation of the line
  which may be used to calculate scaled scores.
• The scaled scores can then be combined by
  arithmetic operations like addition and used fro
  further analysis and reporting of results.

12
Grading System

• Traditional method of adding scores and placing a
  student in different performance
  categories/divisions is arbitrary.
• Standard errors in marking vary with subjects,
  teachers, and time. This is a measure of a
  chance-variation in marking behavior or
  subjectivity.
• Chance variation justifies grading
• A grade is a symbol associated with a score-range
  indicating more or less the same level of
  performance providing for random marking errors.

13
Standard Error of marking

• If the script of an examinee is examined by
  several examiners independently, there would be a
  wide variation in marks around the hypothetical
  true score.
• The difference between the true and obtained or
  awarded score is called the marking error.
• For each examiner there would be a marking error
  for a given answer book. The marking errors being
  random are likely to be normally distributed.
• The standard deviation of marking errors may be
  called the standard error of marking (SEM).
  Research studies conducted in 1960s showed that
  standard error of marking on essay tests was 5-7
  . In a more recent study, it was found to be
  about 12.

14
Interpretation of SEM

• If the obtained score of a person is 50, his true
  score may be anywhere between 29 and 71, if
  standard error of 7 is accepted. This shows that
  the concerned candidate may fail or may obtain a
  first division.
• This shows that the performance in the
  score-range 29 71 is more or less of the same
  level. This forms the basis for awarding grades,
  rather than marks.

15
Procedure of grading two approaches

• The grading process depends on factors like
  nature of the subject, difficulty of questions,
  and the quality of group being evaluated.
• First Approach Direct grading
• Assigning weights and Computation of GPA.
• Second Approach Grading by converting numerical
  scores into Letter grades or symbols.

16
Direct grading

• In direct grading, the evaluator assigns, by his
  own judgment, one of the several symbols to a
  given answer indicating its quality. If there are
  several answers, their grades are to be combined
  and GPA is reported.
• Sometimes, the number or percentage of students
  to be placed in each grade is decided in advance.
• For computing GPA, the grade/symbol assigned to
  each question/component is assigned a weight out
  of 5,4,3,2,1 for A,B,C,D,E respectively, and the
  sum is divided by the number of components.

17
Grading via Numerical Scores

• For this purpose two approaches are used
  absolute grading and relative grading.
• In absolute grading, the absolute
  quality/standard or level of performance (in
  terms of numerical score-ranges) is attached to
  each grading category. For example
• Grade
  Score-range (percent)
• A
  95 -100
• B
  85 95
• C
  75 84
• D
  65 74
• E
  Below 65

18
Absolute grading

• In this case, the number of persons to be placed
  in each grade is not specified in advance.
• This is significantly affected by difficulty
  level of the test and variability of the test
  scores.

19
Relative grading

• In relative grading also known as
  norm-referenced grading, relative positions of
  examinees are considered.
• This method is also known as grading on the
  curve because it assumes a distribution of
  scores normal or otherwise
• It also has two approaches pre-decided interval
  approach and pre-decided number or percentage
  approach.

20
Two approaches

• In the first approach, entire scale is divided
  into score intervals (not necessarily all equal)
  and number of persons to be assigned each grade
  is subsequently determined.
• In the other approach, the number or percentage
  of persons to be assigned each grade is fixed in
  advance and score-range for each grade is
  determined subsequently.

21
Limitations of grading system

• There are chances of misclassification, which
  increase with variability, specifically in the
  neighborhood of cut-scores.
• Subjects abler students to a disadvantage and
  poorer ones to an advantage, because of lumping
  them together.
• Has limited utility in making certain crucial
  decisions such as taking selection decisions.

22

• Thank You