Evaluation: Full Circle

1
Evaluation Full Circle

• CHSC 433
• Module 7/Chapter 14
• UIC School of Public Health
• L. Michele Issel, PhD, RN

2
Objectives

1. Discuss the relevance of the Evaluation Standards
   to the quality of evaluations
2. State under what conditions informed consent is
   needed
3. Write an Executive Summary and appropriate
   Recommendations

3
Approaches to Assessing Quality of Evaluations

• Product-based view looks at what is produced, not
  preferences of evaluation stakeholders
• Manufacturing-based view looks at whether met
  contract and specifications
• User-based view is from the point of view of
  evaluation users

4
Standards and Principles

• The professional evaluation association has
  established both standards for high quality
  evaluations, and principles for conducting
  evaluations.
• These are values that guide the practice of
  evaluation.

5
4 Standards for Evaluation

• Utility standards
• Feasibility standards
• Propriety standards
• Accuracy standards

6
Principles for Evaluation

• Systematic inquiry
• Competence
• Integrity and Honesty
• Respect for People
• Responsibilities for General and Public Welfare

7
Ethics and IRBs

• Ethics is the discipline concerned with just,
  equitable, and moral decision making
• IRB is the administrative body that oversees the
  ethical conduct of research

8
IRBs and Informed Consent

• Informed consent is a process whereby an
  individual agrees to be in a study after
  receiving all the information about what being in
  the study entails.
• Program evaluations that include a research
  element must get IRB approval and use informed
  consents

9
Informed Consent

• Has 8 elements required by Federal Regulations
• Need to obtain if the evaluation
• Findings will be disseminated beyond the program
• and
• The evaluation is being done to generate new
  knowledge

10
Evaluation CQI/TQM

• Effect of Quality Improvement initiatives on
  evaluation
• Staff have expectations of involvement
• Easier to develop program theory because know and
  can use applicable TQM techniques
• Still need outcome evaluations in tax supported
  organizations

11
Performance Measures

• Terminology varies across sectors and disciplines
• Federal lingo is not evaluation lingo
• Level of specificity and measurement varies with
  terminology
• Performance Measures generally refer to
  indicators of either process or effect.

12
Making Recommendations (Hendricks Papagiannis,
1990)

• The following are recommendations about making
  recommendations based on the evaluation

13
Recommendations

• Consider all issues in the evaluation as fair
  game for recommendations
• Think about recommendations as soon as begin
  collecting data
• Draw recommendations from a wide variety of
  sources
• Work closely with agency personnel throughout
  the process

14
Recommendations

• Offer realistic recommendations
• Decide on how specific recommendations will be
• Think twice about recommending fundamental
  changes
• Show future implications of recommendations

15
Recommendations

• Make recommendations easy for decision makers to
  understand
• Stay involved after your recommendations have
  been accepted
• Look for other opportunities to make a repeat
  recommendation if an important one was not
  accepted

16
Executive Summary

• Provides brief, but comprehensive summary of the
  project and evaluation
• Gives highlights and recommendations
• Ranges from one to several pages
• Is often distributed in place of the full report

17
Grant Writing Tips

• Write for the reviewers minimize jargon,
  maximize explanations, keep the grammar easy to
  read
• Follow the instructions! Down to the font size,
  margin specification and lines per inch

18
More Grants Tips

• Match idea with funding priorities keep looking
  for a good match or change the idea
• Make contact with the program officer it is
  their job to get grants submitted, hence their
  helpfulness

19
Recommendations Across the Pyramid
Consider whether recommendations for each level
is appropriate Consider policy recommendation Reco
mmendations at same level are the program,
support or enhance the program