Innovative Diagnosis Models II: The GDINA Model

1
Innovative Diagnosis Models (II)The G-DINA Model
Jimmy de la Torre Department of Educational
Psychology Rutgers, The State University of New
Jersey
2

• Fundamental difference between IRT and CDM A
  fraction subtraction example
• IRT performance is based on a unidimensional
  continuous latent trait
• Students with higher latent traits have higher
  probability of answering the question correctly
• CDM performance is based on binary attribute
  vector
• Successful performance on the task requires a
  series of successful implementations of the
  attributes specified for the task

3
Background

• Denote the response and attribute vectors of
  examinee i by and
• Each attribute pattern is a unique latent class
  thus, K attributes define latent classes
• Attribute specification for the items can be
  found in the Q-matrix, a J x K binary matrix
• DINA (Deterministic Input Noisy And gate) is a
  CDM model that can be used in modeling the
  distribution of Yi given

4

• Inherent limitation of the DINA Model
• Let the number of required attributes for item j
  be 3

5

• Because individuals in group have
  varying deficiencies, their probabilities of
  success may not be the same
• We will consider a generalization of the DINA
  model with more relaxed assumptions

6

• Generalizing the DINA model
• The generalization will be referred to as the
  generalized DINA (G-DINA) model
• The G-DINA model partitions the latent classes
  into latent groups ( is the number of
  required attributes for item j)
• Each latent group represents one reduced
  attribute vector
• Each latent group has its own associated
  probability of success, as in,

7
DINA
8
G-DINA
9

• In its most general formulation, the G-DINA model
  can be expressed as
• is the intercept
• is the main effect due to
• is the interaction effect due to
• is the interaction effect due to

10
Special Cases of the G-DINA Model (Reduced Models)

• When all but and are zero
• ? DINA

11
DINA
12
Special Cases of the G-DINA Model (Reduced Models)

• When all but and are zero
• ? DINA
• When
  
• ? DINO

13
DINO
14
Special Cases of the G-DINA Model (Reduced Models)

• When all but and are zero
• ? DINA
• When
  
• ? DINO
• When all interaction terms are zero
• ? additive CDM (A-CDM)

15
Additive CDM
16
Internal Engine of the G-DINA Model Framework
(Estimation Algorithm)

• The parameters of the G-DINA model can be
  estimated using the EM algorithm
• By using appropriate design matrix M, the
  parameters of the reduced models can be obtained
  from the G-DINA model parameters
• The weight matrix W can be used to account for
  the differential sizes of the latent classes
• The reduced model parameters are estimated via
  weighted least squares, and their standard errors
  via multivariate delta method

17
Model Comparison

• Assuming that the Q-matrix has been correctly
  specified, the saturated model will give the best
  model-data fit
• Statistical tests can be performed to examine
  whether a reduced model can be used in place of
  the saturated model
• Two tests developed for the G-DINA framework are
  the likelihood-ratio (LR) test and the Wald test

18
Simulation Study 1

• To examine the accuracy of the estimation
  algorithm for the G-DINA model framework
• Two factors were involved sample size (500,
  1000, and 2000), and type of reduced model (DINA,
  DINO, and A-CDM)
• Number of items and attributes 30 and 5
• 100 data sets were generated

19
A-CDM (Parameter Estimation)
20
Simulation Study 2

• To examine the Type I error of the LR and Wald
  test under the G-DINA model framework
• Same two factors were involved sample size (500,
  1000, and 2000), and type of reduced model (DINA,
  DINO, and A-CDM)
• Same number of items and attributes 30 and 5
• Nine significance levels were used .50, .40,
  .30, .20, .10, .05, .02, .01, and .005
• 1000 data sets were generated

21
A-CDM (Type I Error - 500 Examinees)
22
A-CDM (Type I Error - 2000 Examinees)

23
Summary and Discussion

• G-DINA is a general and interpretable CDM
• Reduced model parameters can be obtained from the
  G-DINA model parameters
• Reduced model parameter estimates and SE can be
  accurately estimated
• Tests can be done to determine whether reduced
  models are sufficient for a particular item
• Compared to LR test, Wald test is more promising
  -- it provides more accurate Type I error
  particularly when the sample size is large

24

• The G-DINA model framework can increase the
  flexibility and usefulness of CDMs
• CDM specification is not required a priori
• model specification can be verified
• multiple CDMs can be used in the same test
• cognitive processes and problem-solving
  strategies can be empirically studied using large
  samples
• Additional work is needed to better understand
  the properties of the G-DINA model
• Resource needs to be invested to properly
  construct cognitively diagnostic assessments