Using Assessment Blueprints to Ensure Alignment

1
Using Assessment Blueprints to Ensure Alignment

• Scott Marion Sue Stevens
• Wyoming Department of Education
• March 4-5, 2002
• www.measuredprogress.org/wycas/index.htm

2
What is Alignment?

• Matching the knowledge and skills called for in
  the standards with the knowledge and skills
  required on the assessments.
• Two-way process of alignmentwe need to work from
  the standards to the items and from the items to
  the standards.
• Alignment requires matching the level of
  cognitive complexity called for by the
  standard(s) so the full alignment picture is
  really three-way or three dimensional.

3
Alignment

• Minimal non-overlap among the assessments and the
  standards so that students are evaluated on
  standards and not on construct-irrelevant
  material.
• Alignment should occur at the benchmark level to
  make sure that the standards are appropriately
  represented, but never lose site of the
  performance standards.

4
Construct under-representation
5
Constructed irrelevant variance
6
Another view of construct under-representation
Well-sampled Standard
Not well-sampled
7
What is a test blueprint?

• It is tool for designing (building) tests to
  match the content and process specifications
  (e.g., standards and benchmarks) with test items
  and tasks to help avoid construct
  under-representation and construct irrelevance.
• Like a blueprint for building a house, a test
  blueprint needs to be completed PRIOR to building
  the test.
• If youve not followed the order described in
  Step 2, theres still hopeyou can use the
  blueprinting process to evaluate and revise your
  existing assessments.

8
What belongs in a blueprint?

• Content categories (e.g., objectives, benchmarks,
  standards)
• Process (e.g., cognitive demand) categories
• Number of items, tasks, or score points
• Remember items/tasks can assess multiple
  benchmarks, in fact some benchmarks CANNOT be
  assessed in isolation.

9
An Example
10
Moving to new forms of assessment

• The test blueprint was originally designed for
  tests with many short items such as
  multiple-choice
• How should we deal with extended tasks or other
  open-ended items?
• For multi-part tasks, we can treat each
  dimension as an item and/or
• We can treat each score point as a separate
  item
• The important point is that the blueprint
  provides a tool to help establish alignment.

11
Your task

• Use the packet of WyCAS released items, the
  blueprint template, and the standards document to
  categorize the WyCAS items into the appropriate
  content categories.
• Dont worry about cognitive demand nowthats for
  the next workshop.

12
Discussion

• Does every item match a benchmark?
• Are all of the benchmarks covered appropriately
  (remember, this is only a small sample of items)?
• Is the criterion for two-way alignment met? If
  not, what should be done to improve content
  alignment?