Outcome Measurement Training OM102: Choosing Indicators and Tools

1
Outcome Measurement TrainingOM102 Choosing
Indicators and Tools
2
The Agenda

• Refresh
• New Terms
• How to Write Indicators
• Data Collection
• Review Questions

3
Outcomes and Indicators
OUTCOMES
What changed?
Indicators How do we know it changed?
4
Key Terms - Quiz
1. Resources dedicated to or consumed by the
program(s) within an agency and constraints
Logic Model Inputs Activities Outputs Outcomes
2. Benefits or changes to individuals during or
after participating in program activities
3. The direct products of agency services the
results of the process
4. What the agency does with the inputs to
fulfill its mission the program services
5. An agreed upon path that links inputs,
activities, outputs, and outcomes in a logical
fashion.
5
Outcomes Important to Measure

• They are related to the core mission of the
  program.
• Others are interested in knowing about them.
• Data on them would be useful for making changes
  in how the program is being implemented.

6
Logic Model Puzzle

• Label your flip chart with the following headings
• Inputs
• Activities
• Outputs
• Initial Outcomes
• Intermediate Outcomes
• Long Term Outcomes
• 2. Tape your cards down to create a Logic Model
  
• 3. Draw arrows between your outcomes to show the
  logical connections

7
Road Trip
How do you know if the kids are behaving?
8
Terms

• Indicators
• The specific characteristics or behaviors
  measured to track a programs success in
  achieving its outcomes.
• It explains how you know if the outcome was
  achieved.

9
Characteristics of an Effective Indicator
10
Outcome Indicators - Example

• Teen Mother Parenting Education Program
• Long Term Outcome Teen mothers deliver healthy
  babies.
• Outcome Indicators
• Number and percentage of teen mothers who deliver
  a baby weighing 5.6 lbs or more at end of
  pregnancy
• Number and percentage of teen mothers that
  deliver babies during or after 3rd trimester

11
Steps to Writing a Good Indicator

• Identify exactly who is hoped to benefit (WHO?)
• Identify specific, observable change or
  accomplishment (WHAT?)
• Determine when the outcome is expected to occur
  (BY WHEN?)

12
Writing Indicators - Activity

• Look over the logic model your group created
• Choose two outcomes you want to measure.
• Write 1 indicator for each outcome

13
Characteristics of an Effective Indicator
14
Targets Benchmarks

• Target
• a numerical objective for a programs level of
  achievement on an indicator a
  projection
• Benchmark
• performance data used for comparison past
  agency data or industry standard

15
Targets Benchmarks - Example

• Outcome Teen mothers deliver healthy babies
• Outcome Indicators
• Number and percentage of teen mothers who deliver
  a baby weighing 5.6 lbs or more at end of
  pregnancy
• Target
• Next year, 35 (75) of participants will deliver
  a baby weighing 5.6 lbs or more at end of
  pregnancy
• Benchmark
• In our agency over the past 5 years, 70-72 of
  participants have delivered a baby weighing 5.6
  lbs or more at end of pregnancy

16
Data Source
17
Data Collection Methods

• Review Program Records
• Self-Administered Questionnaire
• Staff Assisted Questionnaire
• Interview
• Rating by Trained Observer
• Report from External Party

• Better descriptions include
• description of the method or tool used to gather
  data
• how the tool is administered, and
• the frequency data is collected.

18
Sample Framework
19
Create a Framework
Using one of your own programs as an example
complete an Outcome Measurement Framework for at
least 2 outcomes
20
Final Thoughts
Outcomes are difficult to measure when programs
have these characteristics

• Participants are anonymous
• Program has limited contact with participant
• Participant responses are not reliable
• Programs are focused on prevention

There are usually lots of indicators for the same
outcome. Dont be afraid to use proxies. Make
choosing targets and benchmarks an informed
decision. Remember you may have to negotiate.
Everything can be measured!