Tuesday Topics for State Extension Specialists

1
Tuesday Topicsfor State Extension Specialists
Extension Reporting System June 21, 2005
2
Thank youfor turning off cell phones

• Extension Reporting System
• June 21, 2005

3
Setting Measurable Outcome Indicators

• Joseph L. Donaldson
• Extension Specialist

4
Important Reminders

• Thank you, Specialists
• Thank you, Dr. Norman
• Thank you, Shirley Irwin
• Thank you, Cadre5

5
Federal Process

• You submit impact statements
• I compile these statements
• Submission
• Review Team
• Response

6
Celebrate Yourself

• strong and comprehensive report
• Many strong and well-documented impact
  statements
• is very helpful and adds increased
  credibility...

7
Setting Measurable Outcome Indicators

• Joseph L. Donaldson
• Extension Specialist

8
Our Collections

• MIS Impact Statements
• TNCEP
• TN 4-H Life Skills Evaluation System
• Master Beef Producer
• Master Gardener

9
Todays Outcomes

• Understand the role of outcome indicators in our
  new Extension Reporting System
• Construct measurable outcome indicators for
  statewide Extension programs

10
Demand for Outcomes

• Integration of program performance with public
  funding
• USDA-CSREES
• TN Performance-Based Budgeting
• Outcomes-Based Evaluation

11
Basic Problem

• Using different outcomes across the state
• Inability to aggregate data
• Growing need for accountability measures

12
Outcomes

• What happened as a result of this program?
• How are people better off?
• Financially?
• Quality of Life?
• Environment?

13
State A (19 outcomes)

• Number of people who are involved in addressing
  significant community issues
• Number of people who increase their knowledge of
  governmental process

14
State B (44 teams)

• (Describe) programs developed and outcomes
  reached by the coalitions/networks and or
  individual school, agency or organization
  partnerships that increase healthful eating,
  physical activity, and lifestyle choices, in an
  impact statement (lt500 words).

15
Language

• Outcomes
• Outcome indicators
• Outcome targets

16
Outcomes What Results

• What happens to the people we serve as a result
  of their participation in our programs.
• Tennessee beef cattle producers increase
  profitability.

17
Outcome Indicators

• A measure that indicates the degree to which the
  outcome was achieved.
• Number of beef producers who market their cattle
  collectively.

18
Outcome Targets

• The goal, or anticipated number of clients who
  will achieve the outcome indicator.
• 25 of 50 beef cattle producers will market their
  cattle collectively.

19
How will it work?

• Outcomes
• Outcome indicators
• Outcome targets

20
The Process

• Identify programs
• Identify outcomes
• Identify outcome indicators
• Identify tools to measure the indicators

21
Identify Programs

• Statewide importance
• Need to be collected statewide by agents
• Need to know outputs and outcomes for this program

22
ID Outcomes/Indicators

• Not a list one mile long
• Quality is superior to quantity
• 3-7 indicators should do the job
• Identify an outcome chain a line of logic

23
Outcome Chains

• Short-term Number of producers who gain
  knowledge of group marketing.
• Intermediate Number who market their beef cattle
  collectively.
• Long-term Tennessee beef cattle producers
  increase profitability.

24
Outcome Chains

• Short-term Number of forest landowners who
  increase knowledge of BMPs.
• Intermediate Number who implemented at least two
  practices to reduce nonpoint source pollution.
• Long-term Reduction in nonpoint source pollution
  from silviculture operations.

25
Outcome Chains

• Short-term Number of TN parents who gain
  awareness of accidental poisoning prevention.
• Intermediate Number who now secure poisonous
  products in the home.
• Long-term The rate of TN children involved in
  accidental poisonings will decrease.

26
Outcome Reminder

• The outcome is not necessarily the same as the
  long-term outcome indicator.

27
Outcome

• By building youth-adult partnerships through 4-H,
  we will increase opportunities for positive youth
  development in TN communities.

28
Outcome Chains

• Short-term Number of youth and adults who
  increase leadership skills.
• Intermediate Number of youth and adults
  conducting local events collaboratively.
• Long-term Number of youth and adults in equal
  partnership on a local board or committee.

29
Maybe No Chain

• Number of volunteer hours contributed by number
  of MG
• Number of MG projects completed
• Number of hours of continued education per MG
• Total value of MG volunteer service

30
Identify Evaluation Tools

• Lots of ways to get this job done
• I will help you
• Possible that we will load your outcomes August 1
  and work on evaluation tools later
• Lets look at examples..

31
Food Safety Observation

• ? Uses separately labeled or color-coded cutting
  boards
• ? Changes disposable gloves between tasks
• ? Washes hands before and after, when working
  with raw foods
• ? Cleans and sanitizes food contact surfaces that
  touch contaminated food before they come in
  contact with cooked or ready to eat food
• ? Stores raw or contaminated food below cooked or
  ready to eat foods to prevent them from touching
  or dripping fluids

32
Air Quality Post-Then-Pre
Please think about your indoor air practices now
and before this program. Please check which, if
any practices you do now (left) and then check
which, if any, practices you were doing before
this program (right). BEFORE NOW TH
IS PROGRAM ____ 1. I use a plastic mattress
cover. ____ ____ 2. I dust weekly. ____ ____
3. I change air filters monthly. ____ ____ 4.
I open a bathroom window to reduce
moisture.____ ____ 5. I vacuum weekly. ____
33
Must Limit Outcomes

• Agents should not drown
• Too many will mean little if any use at all
• Statewide importance
• Not fishing for impacts
• Must be intentional

34
Money Matters

• For every 1 invested in the UT Extension
  ________ program, 5 was returned to the local
  community.

35
Multiple Elements

• People
• Acres
• Livestock
• Receipts
• Money Saved
• Bushels

36
Format

• Dont worry too much about format
• Looking for what you need to know
• I will place all in a consistent format
• Let me share two examples..

37
Format A

• ___ of ___ forest landowners used a professional
  forester to help them make management decisions
• Need to know number who adopted
• Need to know number served, surveyed

38
Format B

• This performance measurement was conducted with
  ___ clients
• Indicate survey, focus group, observation,
  interview, etc.
• ___ participants saved
• ___ participants invested
• ___ participants adopted

39
Limit to Three Elements

• People, possession, money
• _____ of _____ beef producers in the Extension
  xyz program marketed _____ head of cattle for a
  increase of _____.

40
Indicate Funding Streams

• Indicate these if necessary beside the outcome
• Number of consumers who now preserve food safely
  (EFNEP TNCEP)

41
Tuesday Topics June 28

• No Meeting

42
Tuesday Topics July 5

• Work Session for Outcome Indicators
• Due July 26
• 156/157 Plant Biotech

43
Tuesday Topics July 12

• Work Session for State Action Agendas (formerly
  called State Plans of Work)
• State Action Agendas only for State Priority
  Programs
• 156/157 Plant Biotech

44
Reporting System Committee

• Ray Burden
• Clyde Chesney
• Sharon Davis
• Shirley Hastings
• Connie Heiskell
• George Hopper
• Sachiko Hurst

• Alice Rhea
• Nancy Rucker
• Anne Sortor
• Jim Stewart
• John Toman
• Shawn Weir
• Dena Wise

45
Thank you!
46
Setting Measurable Outcome Indicators

• Joseph L. Donaldson
• Extension Specialist

47
References

• Bennett, C. Rockwell. K. (1995). Targeting
  outcomes of programs (TOP) An integrated
  approach to planning and evaluation. Retrieved
  August 29, 2003, from University of Nebraska TOP
  Web site http//citnews.unl.edu/TOP/english/index
  .html.
• Diem, K.G. (2003, Feb.) Program development in a
  political world its all about impact. Journal
  of Extension, 41(1). Retrieved from
  http//www.joe.org/joe/2003february/a6.shtml.
• Hatry, H. and Lampkin, L. (2003). Key steps in
  outcome measurement. The Urban Institute.
  Available at http//www.urban.org/UploadedPDF/310
  776_KeySteps.pdf
• Smart, D. (July 7, 2004). Issues and challenges
  of outcome measures Keeping your outcome
  measures simple. Retrieved June 20, 2005 from the
  National Research Center Best of the Best Website
  at http//www.ccfbest.org/outcomemeasurements/iss
  uesandchallenges.htm
• Taylor-Powell, E. (2002). Program development in
  UW-Extension. Retrieved August 28, 2003, from
  University of Wisconsin-Extension-Cooperative
  Extension, Program Development and Evaluation
  Unit Web site http//www1.uwex.edu/ces/pubs/pdf
  /G3658_1.PDF
• UW-Extension. (n.d.) Program development in
  UW-Extension Getting started in Extension
  programming. University of Wisconsin Distance
  Education/Digital Media Unit.
• Wholey, J.S., Hatry, H.P., Newcomer, K.E., Eds.
  (2004). Handbook of Practical Program Evaluation.
  San Francisco Jossey-Bass.