Focused Program Evaluation

1
Focused Program Evaluation
Randolph R. Randy Weigel Susan James EPIC
2004 Lusk, WY September 21
2

• Your tax dollars support this learning
• experience either directly or indirectly.
• Do you want your tax dollar to continue
• supporting this type of program?
• Yes
• No
• Robert J. Fetsch, Extension Spec.
• Colorado State University

3
Definition Of Evaluation
To evaluate is to make an explicit judgment
about the worth of all or part of a program by
collecting evidence to determine if acceptable
standards have been met. Andrews Warner
(1988)
4
Types of Evaluation

• Process (Formative)
• Impact (Summative)

5
When should we conduct an evaluation?
Before? During? After?
6
When should we conduct an evaluation?

• Project design
• Start up
• In-progress
• End of program
• Follow-up

7
When should we NOT conduct an evaluation?

• No one cares
• Not going to change the program
• One time program
• No time, , resources or energy
• No goals or objectives
• To satisfy your performance

8
Evaluability Assessment
What are you going to evaluate? What is the
purpose of the evaluation? Who will use the
evaluation What resources do you have and
need? What questions will the evaluation hope to
answer?
9
Knowledge Item
How important is nitrogen fertilizer management
for maintaining our water quality?
Farmers proper management of nitrogen
fertilizer applications is essential to maintain
good water quality. TRUE FALSE DONT KNOW
10
Attitude Item
How important is nitrogen fertilizer management
for maintaining our water quality?
Farmers proper management of nitrogen
fertilizer applications is essential to maintain
good water quality. Strongly Strongly agree
Agree Disagree disagree
11
Experience Item
How important is nitrogen fertilizer management
for maintaining our water quality?

• Do you follow recommendations for applying
• nitrogen fertilizer when the recommendations are
• based on deep soil sampling?
• No
• YesHow much do you apply
• 1. About 20 less than the amount recommended
• 2. About 10 less than the amount recommended
• 3. The exact amount recommended
• 4. About 10 more than the amount recommended
• 5. About 20 more than the amount recommended

12
Feeling Item
How important is nitrogen fertilizer management
for maintaining our water quality?
How do you feel when others imply that
farmers must better manage nitrogen fertilizer
applications to maintain good water quality?
Extremely Extremely angry ___ ___
___ ___ ___ calm
13
Collecting the Data What sources will you use?

• Existing Information
• People
• Observations
• Pictorial Records

14
What data collection method(s) to use?

• Which method will give you the information you
• need?
• Which method is appropriate for participants?
• Which method is least disruptive?
• Which method can you afford and manage?

15
Common Types of Data Collection
Surveys Interviews Observation Case
Study Group Techniques Tests Photos
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Common Types of Data Collection (cont.)
Document Review Testimonials Expert
Panel Simulations Journals Unobtrusive
measure Triangulation
20
Using the Information
How will the data be analyzed? How will the
information be interpreted? Who will receive the
evaluation? How will the information be
disseminated?
21
Avoid These Mistakes in Your Program Evaluation
22
http//uwacadweb.uwyo.edu/Research/
23
Did todays in-service provide you with
information to review and if necessary improve
your program evaluation skills? 1. Yes 2. No
24
Do you believe LISA research should or should not
be funded by CSREES/USDA? 1. Should 2.
Should not 3. Undecided or unsure
25
Please interpret each of these words below as
a percentage. Example Average
50 Frequently Occasionally Seldom Sometimes
26
How much profit did your business earn
in 2003? _____________ Dollars
27
http//www.extension.psu.edu/evaluation/
28
http//www.uwex.edu/ces/pdande/evaluation/index.ht
ml
29
http//cyfernet.ces.ncsu.edu/cyfres/browse_2.php?s
earchEvaluation
30
http//cru23.cahe.wsu.edu/lifeskills/
31
http//www.four-h.purdue.edu/fourfold/main.htm
32
http//www.sagepub.com/
33
http//www.uwyo.edu/CES/LIFE/Program_Planning_Main
.html