Margie Crutchfield

1
First Time Program Reviewer Seminar

• Welcome!
• Margie Crutchfield
• Associate Vice President, Program Review
• Please remember to set-up your audio
• Go to tools, then audio, then audio set-up
  wizard
• For those of you who have a microphone, we will
  do a microphone sound check one by one,
  starting at 130
• 130 pm - First audio sound check
• 145 pm Second audio sound check
• 155 pm - Final audio sound check
• 200pm - Session will begin

2
Session Agenda

• Eluminate Product Features
• PRS Overview
• On-line Demonstration
• Writing the Report
• Timelines for Fall 08
• Q and A

3
(No Transcript)
4

• Applaud Agree Disagree

5
ICON REVIEW

• Ask a
• question Polling Talk live

6
Key Resources

• www.ncate.org (Program Reviewers)
• NCATE Guidelines for Reviewers
• NCATE Guidelines for Electronic
  Submission
• Recognition Report Templates
• Mini-Videos

7

• I have been assigned a review
• now what?

8
What should I expect from the lead reviewer?

• It is the lead reviewers responsibility to
  contact co-reviewers and set up time line for
  completing work.
• BUT if you dont hear from the lead reviewer
  within a week, you should contact them.
• If you still have problems contacting your team
  members contact your NCATE staff person

9

• In PRS open up your assigned program reviews
• Check to make sure all components of the report
  are present
• If you have concerns or questions about the
  report talk to your lead reviewer and/or your
  NCATE staff person

10

• Prepare at least a draft of your report
• Discuss your report with your team members
• Lead reviewer has information about setting
  conference call
• Be sure and submit your report (otherwise you get
  a lot of annoying e-mails)

11

• Lead reviewers must post the final team
  recognition report.
• This final report will be reviewed by the audit
  team.
• The audit teams report will be posted in PRS
  for you and your team to see

12

• Some items to note

13
Important Reminders

• You must have Version 7 or higher of Adobe
  Acrobat Readerfree download available on PRS
• Mac usersmake sure you review the document on
  the PRS log-in page

14
Lets walk through PRS
15
Logging into PRS

• To get started you will need to follow these
  steps
• Step 1. Go to the URL http//aims.ncate.org,
• Step 2. Enter your ID (e-mail)
• Step 3. Enter your password

16

• When you go to the URL,
• you will see the log-in screen

17
(No Transcript)
18
(No Transcript)
19

• Once in the site
• You will see your name and program assignments

20
(No Transcript)
21
Lets go live
22
Linking

• Programs that have identical assessments but have
  some significant difference (degree level, etc)
• Must have a shell in PRS for each program but
  these can be linked (as evidenced by the color
  coding)

23

• Compilers submit all information in first report
  and submit it.
• When they open second (and subsequent) linked
  reports all information and attachments from the
  first report is copied into the second report.
• The compiler changes any pertinent information on
  the second report and submits it.

24
Reviewers do the same

• Open up the first report, evaluate the report.
• Open the first recognition report template, input
  text and information for all programs.
• Submit the first report.
• Open the second recognition report, change
  information on the cover sheet and submit the
  report.

25
Data Rule

• Data Rule For Fall 08
• Two years of data for all assessments (with
  understanding of programs context)
• If a key assessment is in a course that is taught
  once a year then one year of data one semester
  of data
• If program is brand new or going through program
  review for the very first time, one year of data
  is required

26
TIMELINESUMMARY
27
NCATE Deadline

• Final Posting to Institution
• By 2/1

28
Making Decisions Standards

• Each standard Met, Met with Conditions (except
  for NCTM and NASP), or Not Met
• If standard is Met, you are not required to fill
  in Comment section
• If standard is Not Met or MET with Conditions,
  you MUST include a comment

29
Making the Final Decision

• Follow your SPAs decision rule
• But in general

30

• Recognized
• Program isnt perfect, but is well on its way,
  understands performance assessments, alignment of
  standards and assessments, etc.
• Recognized with condition
• Program understands performance assessment and
  alignment, but may have some serious deficiencies
  in some scoring guides and/or some assessments
• May not have sufficient dat

31

• Further development required/recognized with
  probation/not recognized
• NCATE staff will determine which of the above
  applied
• Program really misses the mark, little or no
  alignment of the standards and the assessments

32

• If the decision is National Recognition with
  Conditions

33

• THE CONDITIONS BOX
• MUST BE FILLED OUT (Part G)

34

• Write in the SPECIFIC conditions that need to
  be met by the institution on the next report
  submission.

35

• This report functions as a contract between
  NCATE-SPA and the Institution.
• In order for an Institution to retain Recognition
  after a Recognized with Conditions decision has
  been granted

36

• The revised team that looks at this when it is
  re-submitted will zero in on this conditions
  section to determine whether they were met or not
• High Stakes!

37
An example of a well-written CONDITION

• Assessments 1 and 2 The program needs to
  address candidate knowledge of content by
  standard (academic discipline) and by category
  (sub scores) on the Praxis II exam. They must
  also present aggregated candidate data (grades)
  in a table or chart of candidate scores from high
  to low for each standard.
• Assessment 5 Candidate impact on student
  learning needs to be addressed by the degree to
  which secondary students learned from candidate
  instruction. The unit plan assignment could well
  yield such data if the candidate taught it in a
  secondary classroom and conducted something as
  simple as a pre and post test on the unit.
• Standard 3.2 The program must identify the
  instructor(s) of the Social Studies Methods
  course(s) and indicate their backgrounds in
  social studies education or in one of the
  disciplines.
• Data from all assessments for the next year must
  be collected and analyzed.
• Additional procedures for evaluating post
  baccalaureate candidates need to be implemented
  that ensure standards and indicators are met.
• Concerns cited under Standard 1 must be
  addressed.

38

• Characteristics of a well-written Conditions
  statement
• CLEAR
• PRECISE
• OBJECTIVE
• STANDARDS BASED
• EVIDENCE BASED

39
A note about tone

• Reviewer attitude in report decisions is
  critical---
• Remember that you want to be as helpful as
  possible to the institutionstating the problem
  in clear objective languageavoid a God-like tone

40

• Reviewers are asked to accept the integrity of
  the program and review it for the purpose of
  determining if the information submitted meets
  the SPA standards
• Do not try to redesign the programs
• Objectively evaluate the QUALITY of the
  performance assessments and rubrics with respect
  to the alignment w/ SPA standards
• POSITIVE PERSPECTIVE
• Should include strengths and weaknesses
• Adapted from IRA- Guidelines for IRA/NCATE
  Program Reviewers October 4, 2006

41

• The BIG question!

42

• DOES THIS PROGRAM MEET THE STANDARDS
• ?

43
Reviewing Revised Reports

• Submitted when program was previously not
  approved.
• You have access to original program report and
  recognition report in PRS
• You ONLY review and make decisions on standards
  that were previously Not Met.

44
Response to Conditions Report

• You have access to original program report and
  recognition report in PRS
• You ONLY review and make decisions on conditions
  listed in Part G of the original recognition
  report.
• If conditions are now met, you must recognize the
  program.

45
Q A

• To pose a question
• Click hand icon
• Push talk icon to begin and end audio
• Please state name and SPA affiliation
• Questions will be taken in the order they are
  received

46
Conclusion

• Web-Conference Archive
• Mini-Videos
• Available on NCATE web site (www.ncate.org),
  click on Program Reviewers, scroll down to
  Resources

47
NCATE Staff

• Margie Crutchfield, margie_at_ncate.org
• Robin Marion, robin_at_ncate.org
• Sabata Morris, sabata_at_ncate.org
• Monique Thomason, monique_at_ncate.org
• Tech Support Cora Mak, cora_at_ncate.org