Evaluating Advocacy: A Model for Public Policy Initiatives

1

• Evaluating Advocacy A Model for Public Policy
  Initiatives

2
Who We Are Innovation Network

• National nonprofit organization
• Committed to evaluation as a tool for empowerment
• Work with nonprofits at the grassroots level
• Build evaluation capacity of nonprofits and
  funders so they can better serve their
  communities
• Practice a participatory approach
• Innovation Network [phone]
• 1625 K St. NW www.innonet.org
• Washington, DC 20006 info_at_innonet.org

3
Overview/Agenda

• Discuss challenges inherent to advocacy
  evaluation
• Review traditional evaluation models and their
  fit for evaluating advocacy
• Share lessons learned from our work
• Evaluation planning
• Data collection strategies
• Feedback loops and reflection
• Shift in emphasis

4
A Case Study

• Coalition for Comprehensive Immigration Reform
  (CCIR)
• Evaluation funded by The Atlantic Philanthropies
• Began the evaluation in Fall 2005

5

• THE CHALLENGES

6
The Challenges

• I continue to find my greatest pleasure, and so
  my reward, in the work that precedes what the
  world calls success.
• -- Thomas Edison

7
The Challenges

• Success is to be measured not so much by the
  position that one has reached in life as by the
  obstacles which he has overcome while trying to
  succeed.
• -- Booker T. Washington

8
The Challenges

• Ability is nothing without opportunity.
• -- Napoleon Bonaparte

9
The Challenges

• If you don't risk anything, you risk even more.
• -- Erica Jong

10

• TRADITIONAL MODELS

11
Continuous Learning Cycle
Logic Model
Analysis, Reflection Improvement
Evaluation Planning
Data Collection
12
The Logic Model
13
Chain of Outcomes
Residents improve basic job skills and identify
jobs career paths of interest KNOWLEDGE/
ATTITUDE
Participants use post-training services, e.g. job
placement, child care, transportation BEHAVIOR
Increased number of community residents obtain
and retain jobs that support families CONDITION
Closer in Time Easier to
Measure More Attributable to
Project
More Distant in Time Harder to Measure Less
Attributable to Project
14

• LESSONS LEARNED

15
Continuous Learning Cycle
Planning
Real-Time Feedback and Reflection
Data Collection
16
8-Step Advocacy Evaluation Plan

1. Locate issue on the policymaking continuum
2. Identify type of policy change
3. Understand role of the group/network in
   policymaking
4. Clarify dimensions of success relevant to
   stakeholders

1. Develop chain of outcomes, leading to the
   intended change
2. Identify indicators measurement systems
3. Identify most effective strategies
4. Prioritize

17
Spheres of Influence
18
Continuous Learning Cycle
Planning
Real-Time Feedback and Reflection
Data Collection
19
Data Collection Strategies

• Mixed-method approach
• Deeper focus on real-time data
• Observations/monitoring
• Frequent opportunities for data collection
• De-brief protocol
• Opportunity windows/intense periods
• Capturing multiple perspectives

20
Continuous Learning Cycle
Planning
Real-Time Feedback and Reflection
Data Collection
21
Reporting and Feedback Loops

• Frequent, informal reporting
• Learning briefs research
• Approximate real-time
• Transparency/inclusion
• Face-to-face

22

• SHIFT IN EMPHASIS

23
Before
Outcomes
Outputs
24
Now
Outputs
Outcomes
25
Finding Balance
Outputs
Outcomes
26
Advocacy Evaluation Project

• Online resources
• Indicators for advocacy
• Piloting tools
• Advisory committee

www.innonet.org/advocacy
27
Any Questions?

• www.innonet.org
• Jennifer Bagnell Stuart
• jabstuart_at_innonet.org
• Ehren D. Reed
• ereed_at_innonet.org

28
Thanks for Your Participation!
Measure results. Make informed decisions.
Create lasting change.
Innovation Network, Inc. 1625 K St. NW, 11th
Floor Washington, DC 20006 (202)
728-0727 www.innonet.org