Program Assessment Technical Assistance Meeting

1
Program AssessmentTechnical Assistance Meeting

• September-October, 2007

2
Accreditation Overview

• The accreditation system is a series of
  activities which, over time, give a clearer
  picture of an institution or program
  sponsorhistory, how it makes changes, how it
  stays current with the field (both standards and
  the schools they serve).

3
Accreditation Activities

• Ongoing Data Gathering Analysis
• Biennial Report
• Program Assessment
• Site Visit
• Follow Up

4
Ongoing Data Gathering Analysis

• Expectation that all approved programs are making
  programmatic improvements based on solid data
• Data Collection Process is a regular, routine
  part of the program
• Good programs use multiple sources of data to
  make programmatic decisions. No single bullet.

5
On-going Data Collection and Analysis

• Data on the effectiveness of candidates and the
  program in contributing to that effectiveness is
  a key focus of the new system
• Data collection alone is not sufficient
  Analysis and Improvements are all key to new
  accreditation system.

6
Biennial Reports

• Focus on candidate outcomes
• Each program sponsor will submit a report for
  each approved program that is offered that
  includes aggregated data, analysis, and program
  modification
• Two sections of the report Afor each program
  (see above) and Bfor overall trends and action
  plan submitted by the Dean.

7
Biennial Reports

• Submitted in years 1, 3 and 5
• CTC staff will review the reports
• News from the Pilot Programs coming soon

8
Program Assessment

• Where does it fit?
• Program Assessment is completed in Year 4 of the
  Accreditation Cycle
• ?This used to be part of the Site Visit, but in
  the revised system Program Assessment has been
  shifted to a different time in the continuing
  accreditation process.

9
Program Assessment

• Program Assessment asks institution to report on
  their ongoing alignment with the standards and
  improvement efforts at the program level.
• Institutions mirror the reflective practices that
  are taught to candidates.
• Program Assessment asks institutions to consider
  how they measure candidate competence and how
  those measures inform instruction, assessment and
  program design/implementation.

10
Program Assessment Purpose

• A professional dialogue between program sponsors
  and readers (facilitated through CTC staff) in
  order to know that the program is meets the
  adopted standards, how it documents candidate
  competence, and how the program makes ongoing
  improvement.

11
Program Assessment Document

• 3 parts
• Part I begins with the current program you are
  implementing.
• For some, it is a recently approved program
  document.
• For others, it is an updated version of the
  approved program document.

12
Ideas for ways to complete Part I

• Submit
• Document describing the current program and how
  it meets the adopted standards.
• Approved program document with updates embedded,
  but highlighted in some way so reviewers know
  what has changed.
• Other ideas?

13
Part I

• May outline why the program has evolved into its
  current form.
• Was there evidence of candidate competence that
  indicated a need to change?
• Were there changes in faculty that necessitated
  changes?
• Was there a change in the needs of surrounding
  districts?
• For future cohorts, this will be addressed in the
  Biennial Reports submitted 1 and 3 years prior to
  the Program Assessment

14
Part II

• ?Part II includes
• current course syllabi
• updated vitae for program faculty

15
Part III

• Documentation that supports the programs
  assessment tools as reported on in Biennial
  Reports
• assessments that are used to determine
  candidate competence
• scoring rubrics for those assessments
• scorer training information
• re-calibration activities.

16
Part III

• Part III will include only those assessments used
  at key points in the program in order to
  determine whether candidates can move to the next
  step or need remediation.
• When and if candidates are ready to assume
  fieldwork.
• How well candidates do in fieldwork.
• When candidates can be recommended for the
  credential.

17
Part III

• What about other types of assessments?
• If observations are used, how does the
  institution/program sponsor ensure that each
  observer is using the feedback form in the same
  manner?

18
Part III

• If rubrics are used, how are faculty trained to
  use the rubric? How often are they
  re-calibrated?
• If course assignments are used, how does the
  institution/program sponsor ensure that each
  section of the course is using the assignment and
  grading it in the same way?

19
Part III

• What should we say about the TPA?
• If a teacher preparation program is using the
  TPA there will not be a need to give the
  background on the development of the exam,
  validity and reliability information, etc.
• ? However, it will be important to note how
  assessors are trained in the particular area, how
  often the scoring is calibrated and the
  information as to how the TPA is administered.

20
Program Assessment

• How will Program Assessment documents be
  reviewed?
• Team of trained reviewers, with expertise in the
  program areas
• Protected time for training, then reading time

21
Program Assessment Review

• What readers will be looking for
• Alignment with program standardsHow are the
  changes linked to the standards? Parts I and III.
• What candidates are taught and how they are
  assessed. Found in Parts I and II of the Program
  Assessment document.

22
Program Assessment Review

• What data indicates about candidate competence.
  Found in Biennial Reports (not available in 2007)
  and Part III.
• Programmatic changeswhat is the data-based
  rationale for changes? How have the changes
  impacted (or are expected to impact) candidate
  competence? Biennial Reports and Part I

23
Program Assessment Review

• What happens if readers cant find something or
  need more information?
• CTC staff will communicate with the institution
  or program sponsor
• Program will be able to submit additional
  information

24
Preliminary Findings

• After a thorough review, the readers make a
  preliminary finding on each Program Standard.
  Findings on each standard may be
• Met
• Met with concerns
• Not met

25
After that?

• Preliminary findings are reported to the
  institution and to the Committee on Accreditation
  (COA).
• Preliminary findings determine the focus of the
  Site Visit. Will additional reviewers, or those
  with particular expertise be needed?
• ? Preliminary findings will be confirmed by the
  site visit team.

26
Program Assessment

• Yellow Cohort (first ones through, YEAH!)
  selected submission dates
• October 2007,
• January 2008 or
• March 2008
• Whichever date is selected, readers need to come
  to a decision about the Program Standards by
  January 2009.

27
Resources

• http//www.ctc.ca.gov/educator-prep/accred-impleme
  ntation.html
• Jo Birdsell jbirdsell_at_ctc.ca.gov
• Sign up for PSD News