A UniverseDefined System of Arithmetic Achievement Items Hively, Patterson,

1
A Universe-Defined System of Arithmetic
Achievement Items (Hively, Patterson, Page,
1968)

• Robert J. Mislevy
• University of Maryland
• April 10, 2002

2
Psychological Perspective

• Written from an unabashedly behavioristic
  perspective.
• A thorough analysis of behavior is capable of
  yielding more than a long list of examples
  (p.276)
• What the organism has may be operationally
  defined as a functional relationship between
  certain classes of stimuli and classes of
  responses (p. 276)
• Classes of stimuli may be defined by stating
  sets of relevant and irrelevant properties
  (p.276) s
• Diagnose organisms discriminative repertoire

3
Recall Messicks Guiding Questions

• What complex of knowledge, skills, or other
  attributes should be assessed, presumably because
  they are tied to explicit or implicit objectives
  of instruction or are otherwise valued by
  society?
• What behaviors or performances should reveal
  those constructs?
• What tasks or situations should elicit those
  behaviors?
• (Messick, 1994)

4
Examples from the Subtraction Universe
(Hively, Patterson, Page, 1968, p. 281)
5
More on Hively et al.

• A hierarchy of forms based on TM variables
  constitutes a crude theory of the psychological
  structure of the subject matter -- a theory which
  may have very little to do with how people
  actually behave (pp. 287-288)
• No necessary connection with instructional
  theory.
• Work product per item,is right/wrong. Compare
  with VanLehns Mind Bugs.

6
Hivelys Item Forms are partial Task Models

• Argument back to why situations with those
  properties evoke behavior that evidence target
  skill/knowledge. YES
• Stimulus material specs. ALMOST
• Conditions for examinee interaction (rules,
  tools, context, instructions). NOT PART OF FORM
• Task-model variables. YES
• Ones with fixed or constrained values all tasks
  must have
• Ones that can vary, and affect evidentiary focus
  or difficulty
• One that can vary, and are incidental to the
  skills meant to be evoked
• Work product specs. NO