Outcome Chain

1
Thinking Strategically About KIDS
COUNT Performance Measures
Anne Gienapp, MPA Sarah Stachowiak, MPA ANNIE E.
CASEY FOUNDATION KIDS COUNT ANNUAL
CONFERENCE September 26, 2008 206.728.0474
www.organizationalresearch.com
1
2
WHAT ARE PERFORMANCE MEASURES?
2
3
Performance Measures

• Include two key areas
• Activities/Outputs Things that you can document
  or count that are associated with your key
  activities e.g.
• What staff, volunteers, partners do
• The amount, dose, duration, breadth, or depth of
  activities
• Outcomes Information about the short-term
  changes that occur among individuals, families,
  organizations, or systems because of your
  activities

3
4
Examples of Performance Measures

• Ex. Prepare press release(Activity)
• Ex. Identify and convene key strategic partners
  (Activity)
• Ex. of meetings held or of people in
  attendance (Output)
• Ex of media outlets that received press
  release (Output)
• Ex. of media pieces echoing message/frame
  (Outcome)
• Ex. increased awareness of issue among key
  groups (Outcome)

4
5
Outcome-Focused Performance Measures Change
Models
5
6
Connecting Performance Measures with Your Theory
of Change

• What will help demonstrate that you are using
  your funding strategically and that your efforts
  will help work toward the ultimate goals?

6
7
Sample Community Organization
7
8

• Output Measure
• of events and meetings
• of people reached

Sample Community Organization
8
9

• Output Measure
• of events and meetings
• of people reached

Sample Community Organization
Outcome Measure 70 of participants increase
knowledge
9
10

• Output Measure
• of events and meetings
• of people reached

Sample Community Organization
Outcome Measure 70 of participants increase
knowledge
Outcome Measure 30 of targeted leaders take a
public action to support High School Redesign
10
11

• Output Measure
• of events and meetings
• of people reached

Sample Community Organization
Outcome Measure 70 of participants increase
knowledge
Outcome Measure 30 of targeted leaders take a
public action to support High School Redesign
Output Measure At lease 5 leaders from each
sector (grassroots, community organizations,
businesses, colleges / universities, parents)
participate in a one-on-one meeting about High
School Redesign
11
12
TYPES OF KIDS COUNT PERFORMANCE MEASURES
12
13
Types of KIDS COUNTPerformance Measures

• Measures of data/policy products reaching
  intended audiences
• Measures of data/policy products sending primary
  messages
• Measures of work influencing policy
  makers/decision makers
• From Application for Continued KIDS COUNT
  Support, Nov 2007, Question 5

13
14
Sample KIDS COUNT Performance Measures from 2008
Proposals
Measures of data/ policy products reaching
intended audiences
14
15
Sample KIDS COUNT Performance Measures from 2008
Proposals



Measures of data/ policy products sending primary
messages
15
16
Sample KIDS COUNT Performance Measures from 2008
Proposals

Measures of work influencing policymakers/
decisions
16
17
What is a Strong KIDS COUNT Performance Measure?
See handout.
17
18
Case ExampleGeorgia Family Connection
Partnership

• GA FCP participated in pilot of advocacy/policy
  evaluation framework
• Developed theory of change and outcome maps for
  each key strategy, integrating KIDS COUNT work
• Prioritized areas for measurement focus
• Operational Measures Measures describing
  activities and outputs
• Effectiveness Measures Measures describing
  desired outcomes

18
19
Georgia Family Connection Partnership
Georgia Family Connection Partnership Theory of
Change Outcome Map Strategy 3
See handout.
19
20
GA FCPExamples of Performance Measures
20
21
PRIORITIZING PERFORMANCE MEASURES
21
22
General Guidance for Prioritization

• Performance Measures should be realistic and
  meaningful measures to which you will hold
  yourself accountable. Be thoughtful when setting
  targets.
• A set of Performance Measures should include both
  output and outcome measures.
• Limit the number of performance measures you
  commit to. The total number should depend on the
  scope and scale of your project. We recommend
  KIDS COUNT grantees consider 4-10 measures.

22
23
Considerations for Prioritization

• What are the key strategies or products you will
  be producing? What will help you know you are
  doing the work with quality and that they are
  having the impact you expect to achieve?
• What will be most strategic, impactful and/or
  important for you to learn about from your work
  on an annual basis?
• What will be a reasonable expectation of change
  within the timeframe you are reporting on?
• What is your organizations capacity for data
  collection and analysis?
• What will give you information to help improve
  your work along the way?
• What will help you tell your story to funders and
  other constituents?

23
24
Measuring Performance Measures

• Outputs
• You probably already have processes or systems or
  documenting your activities and outputs
• Examples
• Number of databooks distributed
• Number of attendees at events
• Tracking software for emails or website traffic
• Will the outputs you capture help you be
  accountable to your funding and let you know
  about the quality of your implementation?

24
25
Measuring Performance Measures

• Outcomes
• You may or may not already capture outcome data.
• You may be able to adapt existing processes to
  capture data on outcomes
• Example add questions about changes in
  knowledge or awareness to a satisfaction survey.

25
26
Measuring Performance Measures

• There are example tools and methodologies
  available you can use or adapt to collect data on
  a select number of key outcomes
• KIDS COUNT Network TA Resources
• http//www.kidscount.org/kcnetwork/resources/tools
  .htm
• ORS Handbook of Data Collection Tools
• http//www.organizationalresearch.com/publications
  /a_handbook_of_data_collection_tools.pdf
• New methods described in Julia Coffmans
  Advocacy Evaluation Trends and Practical Advice
• http//depts.washington.edu/mlcenter/assets/docs/a
  isle/Coffman.TP.2.pdf

26
27
Sample Outcome Measurement Tool
27
28
Sample Outcome Measurement Tool
Example questions for a survey of constituents
receiving issue alerts
1. Please indicate much you agree or disagree
with the following statements.

• 2. Due to an Issue Alert I received, I have
  (check all that apply)
• Made a donation to support issue
• Read a report or additional material about
  issue
• Emailed a legislator about issue
• Called a legislator about issue
• Attended an event related to issue
• Volunteered, e.g. collected signatures for ballot
  initiative, made calls, done other volunteering
  for an organization that supports issue
• Forwarded an Issue Alert to other individuals
• Encouraged other individuals to take action
  regarding issue
• Other (please describe)

28
29
Case ExampleChildren First For Oregon

• CFFO staff began working with ORS in early 2008
  to pilot the advocacy/policy evaluation framework
• ORS and CFFO staff decided to focus both on
  developing an evaluation approach for one
  campaign area while also working on measuring
  performance measures for the KIDS COUNT reporting
• Work has focused on prioritization of performance
  measures and implementation of one outcome
  measure for 2008.

29
30
Wrapping Up

• Questions?
• A-has based on your experiences collecting and
  reporting on performance measures?

30
31
Contact Information

• Anne Gienapp
• agienapp_at_organizationalresearch.com
• Sarah Stachowiak
• sarahs_at_organizationalresearch.com

31