Assessing Your Standards: Standards and Assessment Alignment

1
Assessing Your Standards Standards and
Assessment Alignment
Adult Education Content Standards Consortia
Meeting

• Regie Stites
• July 22, 2005
• U.S. Department of Education, Division of Adult
  Education and Literacy
• American Institutes for Research

2
Assessment is ?
3
Assessment is

• Rocket science
• A critical component of effective teaching
• Not all that it needs to be to support
  standards-based educational improvement

4
True or False

• A test is a test is a test
• A score is a score is a score
• Two tests that measure the same thing can be
  equated
• Multiple choice questions only measure recall
• You can tell a good item by looking at it
• Multiple-choice test Standardized test
  High-stake test

5
Assessment Literacy

• All educational assessments are observations and
  interpretations of defined constructs (models of
  cognition)
• (A)n assessment result is an estimate, based on
  samples of knowledge and performance from the
  much larger universe of everything that a person
  knows and can do (from Knowing What Students
  Know The Science and Design of Educational
  Assessment, NRC, 2001, p. 37)

6
Best Assessment Practices

• Assessments should be used for the purpose they
  were designed to serve for different purposes
  use different tests
• Purpose is defined by when you give the test
  (before, during, or after instruction) and by how
  you will use the test results (formative or
  summative)

7
Formative Assessment

• Key purpose is to use test results (usually
  before or during instruction) to plan, monitor,
  and improve teaching and learning
• Should be curriculum-based and aligned with
  standards
• May or may not be standardized
• Can be selected or locally-developed

8
Summative Assessment

• Key purpose is to use results as evidence of
  achievement (usually pre/post or post
  instruction)
• May or may not be high-stakes for the learner
• Should be a standardized test to ensure
  reliability and fairness
• Can be used as partial evidence of instructional
  quality (and to guide improvement)

9
Seeing Standardized Tests In A New Light

• Recognize their true utility dont ignore
  results and dont over-interpret them
• Be patient - help everyone understand that
  standardized test results are just one of many
  sources of evidence of learning on standards
• Question the experts ask for the help you need
  to improve alignment of standards and assessments

10
Webbs Criteria

• Categorical concurrence
• Extent to which assessment items address same
  content as standards
• Depth-of-knowledge consistency
• Extent to which cognitive complexity of responses
  to assessment tasks matches what learners should
  know and be able to do as stated in the standard
• Range-of-knowledge correspondence
• Proportion of indicators of standards covered by
  assessment items
• Balance of representation
• Extent to which assessment items are distributed
  evenly across indicators

11
Alignment Resources

• Norman L. Webb (1999). Alignment of Science and
  Mathematical Standards and Assessments in Four
  States
• http//www.ccsso.org/content/pdfs/AlignmentPaper.
  pdf
• CCSSO Alignment Analysis Web page
• http//www.ccsso.org/Projects/Alignment_Analysis/
  

12
Guided Practice 1

• Select high priority standards that are most
  suitable for standardized testing (with the test
  you have)
• With the Webb criteria in mind evaluate the
  likely alignment of these standards to your
  current test and options for additional
  assessment if needed

13
Guided Practice 2

• Discuss the need to organize an assessment review
  or a formal alignment analysis of your state
  standards and the test you use for program
  accountability
• Map out plans for a review of assessment options
  or a formal alignment analysis