September%2020,%202005%20POLS%204300.06/PUBL%206800.03

1
September 20, 2005POLS 4300.06/PUBL 6800.03

• Questions from last week?
• Rigor versus relevance in P. E.
• Maximum value for investment
• Limits of rational comprehensiveness
• Strategy for focusing the issues
• Lines crossing on the graph
• An applied, not a basic, social science

2
September 20, 2005 (2)POLS 4300.06/PUBL 6800.03

• P.E. and Organizational Improvement
• Creating and Sustaining an Evaluation Habit of
  Mind
• Positive versus Negative Effects of P.E. on Staff
• The Manitoba School Improvement Program Inc.
  (MSIP) as a Case Study
• Motivation to Improve Human Services?

3
September 20, 2005 (3)POLS 4300.06/PUBL 6800.03

• Evaluators Handbook, chap 1
• Page 10 Forms of Program Evaluation
• Stages of the Program Cycle to be Evaluated
• The Request or Requirement What Is Called for?
• Formative and Summative Assessments
• Quantitative and Qualitative Approaches

4
September 20, 2005 (4)POLS 4300.06?PUBL 6800.03

• Evaluators Handbook, chap 1 (continued)
• Deciding what to observe and/or measure
• Processes, outcomes, goals, decisions
• Contexts, costs and constraints
• User orientation and responsiveness
• Central to ongoing needs assessments
• The key to successful implementations
• P.E. as mainly social and political

5
September 20, 2005 (5)POLS 4300.06/PUBL 6800.03

• Evaluators Handbook, chap 2
• Formative and/or Summative A matter of different
  but overlapping emphases
• Phase A Set the Boundaries of the Evaluation
  (8-10 tasks, pp. 37-32)
• Phase B Select Appropriate Methods (10-12 tasks,
  pp. 32-39)
• Phase C Collect and Analyze Information

6
September 20, 2005 (6)POLS 4300.06/PUBL 6800.03

• Phase D Report Findings
• Table 4 Timeline of Formative Evaluators
  Responsibilities (p. 40)
• Table 5 Differences Between Formative and
  Summative Evaluation Reports (p. 41)
• Structure in Context the Essential Key to
  Responsible Program Evaluation between Rigor and
  Relevance

7
September 20, 2005 (7)POLS 4300.06/PUBL 6800.03

• Stecher and Davis, How to Focus an Evaluation
  (pp. 9-61)
• Focus Clarify and Sharpen
• Elements of the Focusing Process
• Beliefs/Expectations of Evaluator Client
• Information Gathering Activities
• Formulating an Evaluation Plan (phases)
• 5 Approaches to Evaluation (pp. 22-42)

8
September 20, 2005 (8)POLS 4300.06/PUBL 6800.03

• What is the Program being Evaluated?
• Why precisely is it being evaluated?
• What constraints limit the evaluation?
• Information gathering using each method
• Table 3 Questions to guide information gathering
  (pp. 58-61) VERY USEFUL!
• Questions and Comments?