Monitoring and Prescoring Activities

1
Monitoring and Prescoring Activities

• 2008 - 2009 Alternate and Alternative Assessments
  Administrators Update Workshop
• Virginia Department of Education
• August 2008

2
Agenda

• Monitoring
• Collections of Evidence (COEs)
• Course Work Compilations (CWCs)
• Prescoring COEs and CWCs
• Activity
• Questions

3
Monitoring

• Periodic and systematic review of evidence-based
  assessments in the development process with
  opportunities for feedback and intervention.

4
What Monitoring is Not

• Checking with the teacher
• Flipping through an evidence based assessment
• Review without feedback

5
Why Monitor?

• To create better COEs and CWCs.
• To increase the alignment between the students
  level of achievement and the evidence presented
  in the COE or CWC.
• To catch potential problems early.
• To prevent the mad dash before the COEs and
  CWCs are due.

6
Why COEs and CWCs Fail

• Missing evidence to support the SOL
• Missing SOL
• Use of unacceptable evidence
• Use of evidence not aligned with the SOL
• Ungraded or incorrectly graded evidence
• Missing or incomplete SEI tags

7
Monitoring Decisions

• Who will monitor?
• How will you train them?
• When will monitoring occur?
• Who will receive feedback?
• How will feedback be given?
• What are your intervention options?

8
Components of Monitoring

• Trained Reviewers
• Feedback Loops
• Intervention Options

9
Trained Reviewers

• Persons with Content Knowledge and Testing
  Program Knowledge
• School Test Coordinator
• Assigned or Hired Staff
• Building Level Teams
• Central Office Level Teams

10
Feedback Loops

• Teachers preparing collections
• Principals
• VAAP, VAAP, VSEP Leaders
• Trainers/Technical Assistance
• Providers

11
Intervention Options

• Individualized Consultation and Support
• Peer or small group sessions
• Attendance at Local Training
• Access to state and local SOL or assessment
  resources

12
A Sample Monitoring Plan

• School-based Review Teams created and trained
• Teachers turn in evidence-based assessments every
  nine-weeks
• All evidence-based assessments reviewed by team
  according to division pacing chart
• Team report shared with principal and central
  office
• Central office dispatches Content Teams

13
A Sample Monitoring Plan
Select And Train School-Based Monitoring Teams
Monitoring Teams review COEs/ CWCs every 9 weeks.
Monitoring Teams report to the Principal and
School Test Coordinator.
Interventions are scheduled as needed.
Feedback is provided to teachers.
14
Prescoring

• A detailed review of evidence-based assessments
  prior to submission
• Compared to scoring rules
• Prior to scoring
• Opportunity for feedback and intervention

15
What Prescoring is Not

• A mad dash to create a collection of evidence

16
Prescoring Decisions

• Who will prescore?
• How will you train them?
• When will prescoring occur in relation to the
  submission date?
• Who will receive feedback?
• How will feedback be given?
• What are your intervention options?

17
Why Prescore?

• To provide a last check to ensure that all the
  scoring rules have been followed
• To ensure that no last-minute detail has been
  overlooked
• To provide an opportunity to make changes before
  the COE/ CWC is submitted

18
Rule 1- Evidence Must Show Individual Student
Achievement

• Does the evidence show any level of achievement
  according to the SOL/ ASOL being defended?
• Is the evidence on grade level?
• Are all of the SOL addressed (stems and bullets)?

19
Rule 2- Evidence Must Be Student-Generated

• Has the evidence been copied from the blackboard,
  textbook, or computer?
• If the evidence is from a group project, is the
  achievement of the particular student clearly
  identified?
• If the evidence is a worksheet, are there
  examples or directions that provide answers?

20
Rule 3 - Evidence Must Be Labeled with SEI Tag

• Does each piece of evidence have a completed SEI
  tag with
• Content area
• SOL
• Bullet (s)
• Demonstrated or Inferred checked

21
Rule 4- Evidence Must Address Accommodations,
Captions and Grading

• If accommodations have been used, are they
  documented in the evidence?
• Does each picture have a caption that clearly
  address the students level of achievement?
• Has the work been graded accurately with correct
  and incorrect answers clearly identified?

22
Media Considerations

• Do videotapes and audiotapes have transcripts?
• With an SEI Tag
• Are videotapes and audiotapes clearly labeled
  with SEI tags?
• On the case, not on the DVD
• Is equipment/software needed to review the media
  clearly identified on the case?

23
Organizational Considerations

• Is the evidence organized according to the
  scoring worksheet?
• Are all required state and local forms completed?

24
Sample Prescoring Plan

• PHASE 1 - Building Level Review
• Submitting teacher review
• School-based oversight teams review
• Building Administrator review and sign-off

• PHASE 2 - Central Office Review
• Submitted from schools
• Reviewed by central office teams
• Return to schools to correct identified errors or
  omissions

25
Tools for Prescoring

• VGLA Implementation Manual
• Teacher Checklist for Collection of
  Evidence .54
• Administrator Checklist for Collections of
  Evidence...55
• VSEP Implementation Manual
• Teacher Checklist for Course Work Compilation
  ..75
• Administrator Checklist for Course Work
  Compilation.76
• Completed CWC Submission Checklist...78
• VAAP Implementation Manual
• Teacher Checklist for Collection of
  Evidence..89
• Administrator Checklist for Collections of
  Evidence..91

26
Monitoring Prescoring Results

• Reduces or Eliminates COEs and CWCs with
• Missing evidence
• Unacceptable evidence (copied from textbooks,
  homework, hand-over-hand)
• Inaccurate or ungraded evidence

27
Monitoring Prescoring Results

• Eliminates Surprises!
• Ensures collections required for students
  educated outside of the division are completed
• Prevents incomplete collections in the division
• Prevents collections not driven by IEP, 504 or
  LEP Student Assessment Participation Plans

28
Planning

• Use the Activity Sheet in your packet
• Work alone or with those sitting nearby
• Begin to plan your divisions monitoring and
  prescoring activities
• Use the Planning Sheet to continue to plan at
  your division.

29
Questions

• Division of Student Assessment and School
  Improvement
• [phone]
• Student_Assessment_at_doe.virginia.gov