New NCATE Program Review Process

1
New NCATE Program Review Process

• Margaret D. Crutchfield
• Wendy Wiggins
• February, 2005
• margie_at_ncate.org

2

• Accreditation process has two primary components
• Unit review
• Program Review

3
Units

• Unit School, College, or Department of
  Education
• Unit Standards are written by NCATE
• Units are reviewed by the NCATE Board of
  Examiners (BoE) during an on-site visit
• NCATE UAB makes an accreditation decision for the
  unit

4
Programs

• Program Specific Discipline Area
• Program review procedures are dependent on the
  State Partnership Agreement between the state and
  NCATE
• Standards written by SPAs

5

• SPA Specialized Professional Association

6
Principles for the Reform of the Program Review
Process

• Maintain focus on subject content preparation
• Continue to rely on expertise of SPAs
• Reduce costs and burdens for SPAs
• Reduce costs and burdens for institutions
• Use candidate performance evidence consistently

7
Purpose of the Program Review

• Determine whether or not the program has in place
  a limited number (6-8) of comprehensive
  assessments that demonstrate candidate mastery of
  the SPA standards.
• Candidate performance on these assessments is
  appropriate to demonstrate mastery.
• Provide information for unit to use to respond to
  Unit Standard 1

8
NCATE Unit Standard 1

• Candidates preparing to work in schools as
  teachers or other school personnel know the
  content of their fields, demonstrate professional
  and pedagogical knowledge, skills, and
  dispositions and apply them so that students
  learn. Assessments indicate that candidates meet
  professional, state, and institutional standards.

9
Design features of new process

• Web-based institution will log in to access and
  complete report form for each program report due
• Web-based form will include specific prompts for
  each entry, as well as examples of assessments
• Program report will be submitted electronically

10
6-8 Assessments The rules

• Institution must submit a minimum of six
  assessments, unless the SPA specifies more than
  six required assessments
• Institution may submit additional assessments
  when SPA does not specify all eight assessments
• Five specific types of assessments are required
  by all SPAs

11
Required Assessments

• State licensure exam for program area (if
  availableotherwise another content based
  assessment)
• Content Assessment
• Assessment of Planning (e.g., unit plan)
• Student teaching/internship assessment
• Assessment of candidate impact on student
  learning or providing a supporting learning
  environment

12
1 (Required)-CONTENT KNOWLEDGE

• Data from licensure tests or professional
  examina-tions of content knowledge.

13
2 (Required)-CONTENT KNOWLEDGE

• Assessment of content knowledge in the
  discipline to be taught, teaching field, or other
  education professional field.

14
Examples of content-based assessments

• Comp exams, research reports, child studies,
  action research (IRA)
• Comp exams, course grades from content fields,
  portfolio tasks (ACEI)
• Comp exams (when course meets standard) or test
  grades/unit assessments (NCSS)

15
3 (Required)-PEDAGOGICAL PROFESSIONAL
KNOWLEDGE, SKILLS, AND DISPOSITIONS

• Assessment that demonstrates candidates can
  effectively plan classroom-based instruction, or
  fulfill identified professional responsibilities
  in other professional education roles.

16
Example of planning assessments

• Individualized educational plans, needs
  assessments, practicum evaluation, intervention
  plans (TESOL)
• Differentiated unit plan (CEC)

17
4 (Required)- PEDAGOGICAL PROFESSIONAL
KNOWLEDGE, SKILLS, AND DISPOSITIONS

• Assessment that demonstrates candidates'
  knowledge, skills, and dispositions are applied
  effectively in practice.

18
5 (Required)-EFFECTS ON STUDENT LEARNING

• Assessment that demonstrates candidate
  effects on student learning, or on providing
  supportive learning environments for student
  learning.

19
Examples of student learning-based assessments

• Assessments based on student work samples,
  portfolio tasks, case studies, follow-up studies,
  and employer surveys (ACEI)
• Performances on a subset of items on a
  well-designed student teaching assessment
  instrument with science-specific items (NSTA)

20

• Evaluate student knowledge level
• Plan instruction
• Teach
• Analyze student learning
• Reflect

21
6 (Required)

• Additional assessment that addresses SPA
  standards.
• Examples of assessments include evaluations of
  field experiences, case studies, portfolio tasks,
  licensure tests not reported in 1, and follow-up
  studies.

22
7 (Optional for most SPAs)

• Additional assessment that addresses SPA
  standards.
• Examples of assessments include evaluations of
  field experiences, case studies, portfolio tasks,
  licensure tests not reported in 1, and follow-up
  studies.

23
8 (May be Optional)

• Additional assessment that addresses SPA
  standards. Examples of assessments include
  evaluations of field experiences, case studies,
  portfolio tasks, licensure tests not reported in
  1, and follow-up studies.

24
Components of the New Program Report

• Cover sheet
• Section I Contextual information
• Section II Assessments
• Section III Standards Assessment Chart
• Section IV Evidence of Meeting Standards
• Section V Use of Assessment Results to Improve
  Performance

25
Section IContextual Information

• Submit descriptions of the following
• Relevant state or institutional policies and
  practices affecting the program
• Field and clinical experiences
• Admission criteria, including GPA requirements
• The relationship of the program to the units
  conceptual framework
• The relationship of assessments used in the
  program to the units assessment system

26
Section I (continued)

• Attach the following
• The program of study
• Table with number of candidates and completers in
  the program
• Table on faculty expertise and experience in the
  specialty field and in schools

27
Section II Assessments Related Data
When Admin- istered
Scoring Guide/ Criteria
Type of Assess- ment
Name of Assessment
Data Table
Assess- ment
1. 2. 3. 4. 5. 6. 7. 8.
? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ?
28
Section III Alignment of SPA Standards
Assessments
ALIGNMENT with UNIT STANDARD 1
SPECIALTY STANDARD
RELATED ASSESSMENTS
? Content ? Professional Pedagogical KSD ?
Student Learning
Standard 1.1 Standard 1.2
?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8
? Content ? Professional Pedagogical KSD ?
Student Learning
?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8
29
Section IV Write the following in 2 pages or
less (for each assessment)

• Indicate the assessment(s) from Section II that
  provides information about the area and describe
  how it addresses the applicable program
  standards.
• 2. Summarize the data presented in the table(s)
  related to the assessment submitted in Section II
  and interpret the results in terms of the
  standards.

30
Section VUse of Assessment Results

• Evidence must be presented in this section
  that assessment results have been analyzed and
  have been or will be used to improve candidate
  performance and strengthen the program.

31
Features of Program Report

• Limited to 25-35 pages of text plus 3 attachments
  for each of the 6-8 assessments
• Consistent categories of evidence across programs
• Common report formats from institutions SPAs
• Will not include samples of candidate work

32
What else will change under the new program
review system?

• Timeframe for submitting program reports
• Program review decision framework

33
Timeframe for submitting reports

• Under old process 2-3 semesters before the
  NCATE visit
• Under new process one semester before the NCATE
  visit (if seeking first accreditation, two
  semesters before the visit has been proposed)

34
Review calendar

• Fall cycle program reports due September 15
• Spring cycle program reports due February 1

35
Program review decisions

• SPA reviewers will make one of the following
  decisions based on your program report
• The program is nationally recognized.
• The program is nationally recognized with
  conditions.
• A decision is deferred, pending submission of
  additional or clarifying information.
• The program is not nationally recognized a new
  program report may be submitted.

36
How much data is enough?

• Program reports submitted through Spring 2007
  will be eligible for full national recognition if
  they have at least one semesters data for at
  least 5 assessments.
• Program reports submitted through Spring 2007
  that do not meet the above criteria could be
  eligible for national recognition with conditions.

37

• After Spring 2007, reviewers will expect to see 3
  years of data

38
Who to Contact

• Margie Crutchfield margie_at_ncate.org
• Wendy Wiggins
• Resources www.ncate.org
• SPA web sites