Rasch analysis of multimark multipart items

1
Rasch analysis of multi-mark multi-part items

• CEM UK Rasch modelling day
• Tom Bramley
• 14/02/06

2
Rasch in practice

• Dichotomous item tests
• Likert scales
• UK Academic tests and examinations?
• Traditional item type creates problems
• Several parts to a question, variable number of
  marks per part
• Not designed with Rasch model in mind
  (unidimensionality, local independence, choice of
  mark tariff)

3
Vertical equating

• Main advantage of Rasch sample-free person and
  item calibration
• Item banking
• Tailored testing
• Possible to link tiered tests via common items
• Example Key Stage 3 Science (14 yr olds)
• Lower tier aimed at pupils at NC levels 3-6
• Higher tier aimed at pupils at NC levels 5-7
• Questions targetting levels 5 and 6 common to
  both.

4
Analysis issues

• Choice of level at which to model
• Sub-questions (mostly 1-mark, some 2 or 3)
• Whole questions (3-9 marks, mostly 4, 5, 6)
• Choice of software
• Dichotomous will probably get same result
• Partial Credit RUMM / WINSTEPS use different
  methods
• Choice of linking method
• Joint estimation single large data set with
  missing data
• Separate estimation manually calculate linking
  constant
• Choice of which (if any) common items to exclude

5
RUMM 2020

• Demo of sub and whole-question analyses

6
Common sub-questions
C 1.532 logits
7
Common sub-questions
C 1.586 logits
8
Common whole questions
C 1.178 logits
9
Common whole questions
C 1.166 logits
10
Equipercentile T36
11
Equipercentile T57
12
Equating results summary
13
What is a logit?
14
Check on equate