A Template Construction Path

1
A Template Construction Path
Updated 10/22/03 - C. Kennedy
2
How might a designer get started?
3
A Template Design Process- Overview -

• Determine the curricular goals (cognitive model,
  student learning objectives, progress maps,
  design patterns, etc.) for the instructional
  module.
• Determine the work products (evidence) from which
  inferences can be drawn regarding student
  knowledge relevant to the cognitive model.
• Determine how the evidence will be assessed the
  rubrics (categorizations of responses) that will
  be used relative to the cognitive model.
• Define the measurement model that transforms
  evidence to inferences.

4
1. Design the Cognitive Model
SM
5
2. Design the Evidence
SM
6
3. Design the Evaluation Procedure
SM
WORK PRODUCT(S)
SMV A
SMV B
ACTIVITY
OBS. VAR. 3
7
4. Design the Measurement Model
SM
EVALUATION PHASE(S)
WORK PRODUCT(S)
SMV A
ACTIVITY
OBS. VAR. 3
8
What is a template?
9
A template is an abstraction of a family of
assessment tasks.
10
Evaluation Phases transform work products of an
activity into observable variables.
EVALUATION PHASE(S)
11
Measurement Model Fragments connect observable
variables with student model variables.
MMF 1
MMF 2
MMF 3
12
Measurement Model Fragments connect observable
variables with student model variables.
Each MMF can have at most one OV
but may have one or many SMVs.
MMF 2
13
How is a template related to an assessment task a
student interacts with?
14
A fully specified template (aka task
specification) spawns an assessment task.
(This is done by the application.)
15
The work product is evaluated to produce
observable variables.
(This may be done by the application
automatically, or by human raters online or
offline.)
16
The complete measurement model is sent to Scoring
Engine.
(This is done by the application.)
Packaged in XML and sent to Scoring Engine
17
How do I get started?
18
A Template Development Process

• Design the Template - consider using template
  component summary areas to capture design ideas.
• Determine relevant Student Model Variables and
  the Student Model.
• Determine the Activities and Work Products.
• Determine the Evaluation Procedures and the
  Observable Variables they generate (both
  intermediate and final OVs).
• Determine the Measurement Models linking each
  final Observable Variable with associated
  Student Model Variables.

19

• Construct the Template
• Define
• Student Model Variables
• Student Models
• Work Products
• Observable Variables
• Define
• Evaluation Procedures (and Phases)
• Measurement Models
• Define Activities
• Define Template

20
Then, on to Task Model Variables and Presentation
specifications!