A method to estimate Differential Item Functioning in IRT models

1
A method to estimate Differential Item
Functioning in IRT models
Herbert Matschinger1, Oliver H. Günther2,
Hans-Helmut König2 1Evaluation Research and
Epidemiology 2Health Economics Research Unit
Department of Psychiatry University of
Leipzig e-mail Herbert.Matschinger_at_medizin.uni-le
ipzig.de
2
Health-Related Quality Of Life (HRQOL)

• 5 3 - categorical items to assess health related
  quality of life.
• dichotomized 1 2,3
• Mobility
• Self-care
• Usual (Daily) activities
• Pain/Discomfort
• Anxiety/Depression

3
Data

• European Study of the Epidemiology of Mental
  Disorders (ESEMeD)
• 2001 - 2003
• 6 European countries N 21425
• Belgium 2419
• France 2894
• Germany. 3555
• Italy 4712
• The Netherlands 2372
• Spain 5473

4
Possible Research Questions

• Do these 5 items measure one single dimension in
  the sense of a Rasch model ?
• Do the item difficulty parameters differ between
  countries ?
• Are the ICCs all parallel?
• Do the discrimination parameters differ between
  countries ?

5
Linear Mixed Modell
2 sets of predictors 1) X fixed
effects 2) Z random effects
De Boeck,P. and Wilson,M. (2004). Exploratory
Item Response Models A General Linear and
Nonlinear Approach. New York, Berlin Springer.
6
Generalised Random Intercept Modell
?pi log(ppi/(1-ppi) LogitPr(ypi 1
?p) - ßi ?p
random intercept ?p ? ?p0Zi0
fixe Effekte -ßi ?
LogitPr(ypi 1 ?p) Xpi ßi ?p
Xpi 1 - Parameter LogitPr(ypi 1 ?p)
Xpi ßi ?p Xpi ?i 2 Parameter
Skrondal,A. und Rabe-Hesketh,S. (2004).
Generalized Latent Variable Modeling Multilevel,
Longitudinal, and Structural Equation Models.
London, New York Chapman Hall/CRC.
7
Structure of the data (long format)
8
Logits For The First 3 Items Of Person p
LogitP( y11 ?1) ß11 ß20 ß30 ß40
ß50 ?1
ß1 ?1 LogitP( y12 ?1) ß10
ß21 ß30 ß40 ß50 ?1
ß2 ?1 LogitP( y13
?1) ß10 ß20 ß31 ß40 ß50 ?1
ß3
?1
9
Frequencies
10

1 - Parameter model for all 6 countries
Intercept (-ß)
stand. err item1 -3.528638 .043579
item2 -5.576689 .061522 item3
-3.851309 .045826 item4 -1.809436 .03351
6 item5 -4.449809 .050426 var(?)
6.4802599 .16565955 log likelihood
-31545.688
11
(No Transcript)
12
Relation Between Fixed and Random Parameters

• Effects of exogenous variables can be modelled
  via the fixed or the random part of the modell
• The fixed effects (intercepts) are the effects
  for the value 0 of the predictors of the random
  term
• The two models are identical if certain
  constraints hold for the fixed effects.

13
1 Parameter Model / Effect of Countries on T

• Generate two sets of (dummy) indicator variables
  for two different reference categories
• Estimate the model twice for two different
  reference categories
• Compare the two results with respect to the
  differences of the fixed parameters (item
  difficulties)

14
1 Parameter model effect on ? (reference group
is Belgium (1))
Intercept (-ß) s.e.
effect s.e.
Reference Belgium item1
-3.510938 .0803299 France
.54711696 (.0949265) item2 -5.55874
.0912916 Germany -.11129763
(.0931433) item3 -3.833834 .0815657
Italy -.19944865 (.08868224)
item4 -1.788838 .0753648 Netherlands
.14494306 (.10088877) item5 -4.432465
.0842277 Spain -.21903409
(.08672157) var(1) 6.4690974 (.16557056) LL
-31485.737
21
15
1 Parameter model effect on ? (reference group
is France (2))
Intercept (-ß) s.e.
effect s.e.
Reference France item1
-2.96341 .0696799 Belgium
-.54803974 (.09490968) item2 -5.01124
.0813064 Germany -.65890334
(.08620927) item3 -3.28631 .0709361 Italy
-.74705435 (.08142842) item4
-1.24127 .0651058 Netherlands
-.40267533 (.09432633) item5 -3.88495
.0737165 Spain -.76663319
(.07938433) var(1) 6.4690974 (.16557056) LL
-31485.737
16
Differences between fixed parameters (ß)
France -2.96341 -5.011237 -3.28631
-1.241268 -3.884948
Belgium Difference -3.510938
-5.55874 -3.833834 ?0.548 for each item
-1.788838 -4.432465
17
Systematics in differences

• The fixed parameter depend on the contrast
  employed for the predictor.
• The fixed parameter are the item difficulties
  for the reference category of the predictor.
• The difference in difficulties between the two
  estimates are the differences between the two
  reference categories (countries)
• These differences are the same for all items

18
(No Transcript)
19
Modeling Country Differences Via Fixed Effects
/ Effects on ?

• The fixed effects depend on the reference
  category
• Choose category 1 (Belgium) for reference
• Define all possible interaction effects between
  the dummies of the countries and each item
  (selection variables X)
• Constrain these interaction effects for each
  country to be equal for each item

20
Constraints on the Interaction Effects
Item1co2 Item2co2 Item3co2 Item4co2 Item5co2
Item1co3 Item2co3 Item3co3 Item4co3 Item5co3
Item1co4 Item2co4 Item3co4 Item4co4 Item5co4
Item1co5 Item2co5 Item3co5 Item4co5 Item5co5
Item1co6 Item2co6 Item3co6 Item4co6 Item5co6
Item1co2 Item2co2 Item1co3
Item2co3 Item1co4 Item2co4 Item1co2
Item3co2 Item1co3 Item3co3 Item1co4
Item3co4 Item1co2 Item4co2 Item1co3
Item4co3 Item1co4 Item4co4 Item1co2
Item5co2 Item1co3 Item5co3 Item1co4
Item5co4
21
Results
item1 -3.510822 .0803441
item2 -5.558637 .0913007 item3
-3.833719 .081579 item4 -1.788679
.0753829 item5 -4.432352 .0842393
France Ico2 .5468687 .0949383
Germany Ico3 -.1115037 .0931599
Italy Ico4 -.1996552 .0887003
Netherlands Ico5 .1447117 .1009002
Spain Ico6 -.2192302 .0867413
........................................... .....
......................................
I4co2 .5468687 .0949383
I4co3 -.1115037 .0931599
I4co4 -.1996552 .0887003
I4co5 .1447117 .1009002
I4co6 -.2192302 .0867413
Belgium
Item 1
Item 5
22
var(1) 6.4695502 (.16557072)LL
-31485.71012285193Compare with results from
slide 15
These two models are equal ! Both models assume
equal item functioning with respect to country
differences
23
(No Transcript)
24
Modeling Country Differences Without Constraints
on Fixed Effects

• Results for a model without these 20 constraints
  are completely different.
• LL -31115 compared to 31484 df(20)
• Deviations from Belgium are different for each
  item
• Item-difficulties are heterogeneous with respect
  to countries

25
Results
item1 -3.510822 .0803441
item2 -5.558637 .0913007 item3
-3.833719 .081579 item4 -1.788679
.0753829 item5 -4.432352
.0842393 France Ico2 .2672621
.1360669 Germany Ico3 .4485834
.1301335 Italy Ico4 -.3667433
.1284983 Netherlands Ico5 -.1668095
.1474595 Spain Ico6 .1085338
.122575 France I1co2 .1939196
.1898053 Germany I1co3 -.6497205
.1954673 Italy I1co4 -.3702901
.1787872 Netherlands I1co5 -.2827915
.2075728 Spain I1co6 -.1267169
.169957
Belgium
Item 1
Item 2
26
Results cont.
France I2co2 -.2079065 .141196
Germany I2co3 -.4445011 .1373677
Italy I2co4 -.51551 .1304282
Netherlands I2co5 .1836286 .1448181
Spain I2co6 -.1787188 .1247335
France I3co2 .6728065 .1156087
Germany I3co3 -.0709686 .1138256
Italy I3co4 -.1572684 .1080919
Netherlands I3co5 .5405529 .1222693
Spain I3co6 -.5646274 .1067354
France I4co2 1.640549 .1511489
Germany I4co3 -.7005679 .1700497
Italy I4co4 .4469239 .1478239
Netherlands I4co5 -.9392069 .196039
Spain I4co6 .1914548 .1461186
Item 3
Item 4
Item 5
27
(No Transcript)
28
2 - Parameter Model
Intercept SE item1
-4.885114 .1492511 item2 -8.501162
.3370249 item3 -7.840406 .3939964
item4 -1.630332 .0350471 tem5
-2.919229 .0394554 var(1) 1 (0)
loadings for random effect item1
3.8137476 (.13478418) item2 4.3643911
(.20426133) item3 5.9397444 (.32311677)
item4 2.1078878 (.05010409) item5
1.0385574 (.03638325) LL -30547.222
29
item 5
30
2 Parameter Modell effect on ?

• Now we choose the effect coding for the countries
  
• The effect for the reference category is the
  negativ sum of all the other effects
• The model was estimated twice
• Reference category 6 (Spain)
• Reference category 1 (Belgium)
• Fixed effects keep virtually the same
• Loadings keep almost the same

31
2 Parameter Modell effect on ?
Intercept SE
Loading SE item
1 -4.801975 .1460378 item 1 3.8015702
.13333549 item 2 -8.457937 .3378112 item
2 4.3896022 .20675589 item 3 -7.658351
.3787267 item 3 5.8764124 .31438171 item 4
-1.599355 .0351327 item 4 2.1281491
.05062713 item 5 -2.903158 .0392471 item
5 1.0433228 .03642365
Deviation SE (1)
Belgium .00116297 .0249041 LL-30506
(2) France .13703951 .0233686 (3)
Germany -.02622923 .0212044 (4)
Italy -.10642868 .0196428 (5)
Netherlands .06823874 .0252931 (6)
Spain -.07378098 .0181395
32
40 item 5
33
DIF for difficulties ß (i-contrast) LL-30161.21
34
Germany France Spain Belgium
Netherlands Italy
6
6
6
6
6
6
35
France Belgium Spain Netherlands
Italy Germany
6
6
6
6
6
6
36
Netherlands Belgium Spain France Germany Ital
y
6
6
6
6
37
France Netherlands Belgium Germany Italy Spain
6
6
6
6
6
38
France Italy Spain Belgium Germany
Netherlands
6
6
6
6
6
39
DIF for difficulty and discrimination (i-contrast)
eq discrimc item1-item5 _Ico2 - _I4co6
gllamm eurod item1- item5 _Ico2 - _I4co6
, link(logit) fam(bin) i(id) eqs(discrimc)
constr(2) frload(1) w(wt) nocons adapt dot
40
Discrimination parameters for Belgium
------------------------------ Coef.
Std. Err. ----------------------------- item
1 2.9217353 (.3955187) item 2 3.5756061
(.41036239) item 3 6.6334026 (.86959252)
item 4 2.0837394 (.14233753) item 5 .95331206
(.112838)
41
Item 1
_Ico2 -.97672244 (.46282706) France
_Ico3 -.14834368 (.53854614) Germany
_Ico4 1.1158482 (.60224724) Italy _Ico5
.32108006 (.66483885) Netherlands _Ico6
.00363324 (.47251395) Spain
42
Item 2
_I1co2 .05795306 (.57767685) France
_I1co3 .79687968 (.7296182) Germany
_I1co4 1.7103824 (.73387108) Italy _I1co5
1.0374762 (.86778915) Netherlands _I1co6
1.178677 (.5917116) Spain
43
Item 3
_I2co2 -1.6862693 (1.0787816) France
_I2co3 -1.2215711 (1.1674393) Germany
_I2co4 .12178561 (1.1594229) Italy _I2co5
-2.5445023 (1.0072729) Netherlands _I2co6
.7605634 (1.2962747) Spain
44
Item 4
_I3co2 .20873075 (.21891064) France _I3co3
.05125628 (.18909945) Germany _I3co4
.07868985 (.18044014) Italy _I3co5
-.12776638 (.20528291) Netherlands _I3co6
.15262695 (.17397662) Spain
45
Item 5
_I4co2 -.3214976 (.13368081) France _I4co3
.13574822 (.15949164) Germany _I4co4
.18829768 (.13714607) Italy _I4co5
-.14969769 (.18109017) Netherlands _I4co6
.50673288 (.13977853) Spain 47
46
6
6
6
6
6
6
47
6
6
6
6
6
6
48
6
6
6
6
49
6
6
6
6
6
50
item 5
6
6
6
6
6
51
Summary

• 1- and 2 parameter IRT models can be specified by
  random intercept models with logit link function
  and a binomial error
• Effects of individual characteristics can be
  modelled via fixed or random effects
• DIF can be modelled by employing the necessary
  interaction effects.
• gllamm within STATA is a very flexible tool for
  specifying IRT models
• The 5 Items of the HRQOL do not portray one
  single dimension
• Anxiety/Depression measures a different dimension

52
Thank you very much !!