What are some fundamental assessment principles

1

• What are some fundamental assessment principles?

Rudner Schafer, 2002, pp. 6-20
2
What are some fundamental assessment principles?

• 1. Assessment is a process of professional
  judgment involving
• Development of measures
• Creating rubrics
• Grading participation
• Scoring essays, etc.
• Interpretation of scores
• Meaning
• Value
• Impact
• (see Ruder Schafer, 2002, pp. 6-11)

3
What are some fundamental assessment principles?

• 2. Assessment decision-making involves a series
  of tensions
• Learning vs. Auditing
• Formative vs. Summative
• Criterion-referenced vs. Norm-referenced
• Traditional vs. Alternative
• Authentic vs. Contrived
• Speeded tests vs. Power tests
• Standardized tests vs. Classroom tests

4
What are some fundamental assessment principles?

• 3. Assessment impacts student motivation and
  learning
• encourage students to be involved in the
  assessment process
• teach students about the assessment process
• develop a system that encourages students to what
  to learn
• assess what students know, not what they dont
  know
• allow students to see the progress they are
  making
• encourage successsuccess begets success

5
What are some fundamental assessment principles?

• 4. Assessment involves errors
• consider that scores are influenced by the
  standard error of measurement (SEM) which
• Is a means of accounting for possible variability
  in scores.
• Provides a range of where the true student score
  might be located.
• the smaller the SEM, the more confident we are of
  the accuracy of the score

6
What are some fundamental assessment principles?

• 5. Good assessment promotes good instruction
• informs the teacher of lesson effectiveness
• provides opportunities to alter instruction as
  needed

7
What are some fundamental assessment principles?

• 6. Good assessment is valid and reliable
• Validity the degree with which the inferences
  based on test scores are meaningful, useful, and
  appropriate (Messick, 1996).
• Does it measure what it says it measures?
• Is often determined through expert opinion.

8
What are some fundamental assessment principles?

• Traditional types of validity
• A. Criterion-related validity evidence
• refers to tests scores are systematically
  related to one or more outcome criteria by
  answering
• How well does the assessment measure a skill,
  trait, or domain when compared to other existing
  instruments?
• How well does the assessment measure a skill,
  trait, or domain when compared to a criterion?
• (Overton, 2003 Ruder Schafer, 2002, p. 12 )

9
What are some fundamental assessment principles?

• B. Content-related validity evidence
• refers to the extent to which the test questions
  represent the skills in the specified subject
  area by answering
• How well do the items on a test represent the
  content it purports to measure?
• (Overton, 2003 Ruder Schafer, 2002, p. 12 )

10
What are some fundamental assessment principles?

• C. Construct-related validity evidence
• refers to the extent to which the test measures
  the right psychological constructs by
  answering
• How well does the instrument measure the
  psychological constructs?
• (Overton, 2003 Ruder Schafer, 2002, p. 12 )
• A construct is a psychological trait, personality
  trait, psychological concept, attribute, or
  theoretical characteristic.
• (Overton, 2003, p. 153).

11
What are some fundamental assessment principles?

• What are some sources of invalidity?
• A. Construct under-representation
• Lacks the important dimensions or facets of the
  construct to be measured
• B. Construct-irrelevant variance
• Is too easy Information allows students to
  answer the question without assessing their true
  knowledge. Students score too high.
• Is too hard. Too much extraneous questions are
  provided. Students score too low.

12
What are some fundamental assessment principles?

• Reliability
• refers to the degree of score consistency
• can be viewed as an absence of error
• less error increases the reliability of
  assessment
• Reliable assessments
• produce consistent and dependable scores
• have minimal random error

13
What are some fundamental assessment principles?

• Sources of reliability error
• A. The test itself
• the selection of items to represent skills or
  domains
• effectiveness of test item distractors
• difficulty in relationship to students ability

14
What are some fundamental assessment principles?

• B. Student factors when taking the test
• attitudes
• health
• sleep
• carelessness
• test taking abilities

15
What are some fundamental assessment principles?

• C. Scoring factors
• incorrect scoring of multiple choice questions
• lack of objective criteria for evaluation
• inconsistent rater scoring
• biases

16
What are some fundamental assessment principles?

• Reliability coefficientsa number that indicates
  the reliability of an assessment. Different
  assessments allow for different reliability
  coefficients
• .60 and higher for group data used for
  administrative purposes
• .80 and higher for individual data that influence
  screening decisions
• .90 and higher or placement decisions,
  achievement tests, aptitude tests
• (Salvia Ysseldyke, 1995)

17
What are some fundamental assessment principles?

• How is reliability coefficients developed?
• A. Test-retest reliability
• developed by administering the same test twice
  and correlating the scores
• is not practical
• requires testing the same group twice
• is expensive
• is not a good use of time
• allows students to learn from the first test

18
What are some fundamental assessment principles?

• B. Split-half reliability
• is derived by dividing a test into halves and
  correlating scores on each half
• could divide into odd v. even questions
• better then test-retest
• is best if divided based on content
• is not appropriate for speed tests where all
  items may not be answered by students

19
What are some fundamental assessment principles?

• C. Internal consistency
• focuses on the degree to which individual items
  correlate to one another across the class
• referred to as homogeneity
• derives statistics
• Cronbachs alpha
• Kuder-Richardson Formula 21 (KR-21)
• requires only one test administration
• are better when a single skill area is being
  measured

20
What are some fundamental assessment principles?

• D. Alternate-form reliability
• refers to having different versions of tests that
  are matched in content and difficulty
• might involve having Form A and Form B of an
  assessment
• some variability due to being different
  assessments

21
What are some fundamental assessment principles?

• 7. Good assessment is fair and ethical
• fair and ethical assessment has
• an absence of bias
• equitable treatment
• equality in outcomes
• opportunity to learn
• Is detailed in the Fairness in Testing section
  of the Standards
• (Available at http//www.apa.org/science/standards
  .html by American Psychological Association)

22
What are some fundamental assessment principles?

• 8. Good assessments use multiple methods
• Uses the principle of Triangulation defined as
• using multiple sources of data to derive an
  accurate picture of students abilities

23
What are some fundamental assessment principles?

• 9. Good assessment is efficient and feasible
• provides teachers and administrators with the
  most bang for their buck
• is provided in a manner that uses district and
  school resources efficiently

24
What are some fundamental assessment principles?

• 10. Good assessment uses technology when
  appropriate
• uses computer-based assessment to free the
  teacher up to teach
• examples include
• item banks
• electronic grading
• computer-adapted testing
• computer-based simulations